Phase 7: AI-assisted query authoring (autocomplete, explain, fix, optimize, NL translation)

Adds a self-hosted (Ollama, qwen2.5-coder) model provider abstraction
with a pluggable opt-in cloud adapter, schema grounding, and a shared
cost/safety guard every AI-suggested query is assessed against --
compiling to and executing through the same unchanged Phase 2 IR/
compiler and Phase 4 tenant scoping as a hand-written query, no
parallel execution path.

Track A (built into the query bar): inline ghost-text autocomplete,
"Explain this query", "Fix this query" with a diff view, and a
rule-based "Optimize" suggestion. Track B: natural-language-to-query
translation, always a separate review step from execution, with
`sentryctl query --nl` requiring explicit confirmation to run.
Every accepted/dismissed translate-fix-optimize interaction is logged
into the same append-only audit_log table Phase 4 built.

Two real product bugs were found and fixed via live browser
verification (a Svelte effect re-running on every keystroke that
silently cancelled the ghost-text debounce; a ghost-text widget
positioned at document offset 0 instead of the cursor), and a real
costguard logic bug (unbounded-aggregation vs. raw-row) was caught by
its own test suite. New integration tests wire a real Ollama client
through the real HTTP handler against a mock server matching Ollama's
wire contract (hack/mock-ollama), keeping model-quality verification
out of CI as a disclosed, periodic human-run check instead.

See /docs/phase-7-ai-design.md and /docs/phase-7-runbook.md.

docs/phase-7-runbook.md

@@ -0,0 +1,224 @@
+# Phase 7 runbook
+
+Extends `/docs/phase-0-runbook.md` through `/docs/phase-5-runbook.md`
+(Phase 6 had no runbook of its own — a compliance audit, not a running
+system). Read those first. Phase 7 adds one new component category (an
+AI model provider) and touches `api`, `enterprise/`, `web`, and `cli` —
+see `/docs/phase-7-ai-design.md` for the full design record; this
+document is verification only.
+
+## What's actually been verified
+
+Every AI operation (`complete`, `explain`, `fix`, `optimize`,
+`translate`, and the audit-logging endpoint behind it) has been run
+end-to-end against a real `docker compose` stack — real HTTP requests
+into the real `sentry-api` container, through the real
+`api/ai/provider/ollama.Client`, over a real network call, into a real
+process answering Ollama's actual `/api/chat` wire contract. **No real
+model weights are used anywhere in this verification** — see
+"Why a mock provider, not a real model" below for why that's a
+deliberate, disclosed choice rather than a shortcut. Two real product
+bugs were found and fixed via live browser verification of the frontend
+half (`QueryEditor.svelte`'s ghost-text autocomplete) that neither
+`svelte-check` nor `npm run build` caught — see
+`/docs/phase-7-ai-design.md`'s Track A section for the full writeup;
+this runbook doesn't repeat it.
+
+Also verified in this pass, against the same live stack's real
+Postgres: the Phase 4 `audit_log` table's `event_type` CHECK constraint
+was extended with migration 0036, and both new
+`enterprise/internal/audit` tests (`TestAIInteractionLoggerWritesAttributed
+ToContextIdentity`, `TestAIInteractionLoggerRefusesWithoutIdentity`)
+passed against it — a real row lands with `event_type='ai_interaction'`,
+correctly attributed to the tenant/user identity in context, with
+`detail` carrying the operation/confidence/accepted/edited fields as
+JSON.
+
+**Not verified, disclosed rather than silently skipped**: this
+environment has no GPU and no downloaded model weights, so the actual
+quality of `qwen2.5-coder:7b`'s output — whether it reliably produces
+valid pipe syntax for realistic questions, how well-calibrated its
+self-reported confidence is, whether Explain's prose actually reads as
+useful — has never been checked here. See
+`/docs/phase-7-ai-design.md`'s "Integration tests and CI testability"
+section for why that's kept as a periodic human-run checklist item
+rather than something this runbook or CI can cover.
+
+## 1. Bring up the stack
+
+```sh
+docker compose up -d --build
+cd web && npm run dev   # localhost:5183, talks to localhost:8080/8081 by default
+```
+
+No new required services — `docker compose ps` shows the same set as
+Phase 5. AI routes are off by default: with no `OLLAMA_BASE_URL` set,
+`api`/`enterprise-api` never register `/ai/*` at all (confirmed live in
+this pass — `curl -X POST localhost:8080/ai/translate` returns a plain
+`404`, not a 500 or a hang against an unreachable `localhost:11434`).
+
+## 2. Enable AI routes against the committed mock provider
+
+`hack/mock-ollama` (new this phase) answers Ollama's real `/api/chat`
+wire contract with fixed, deterministic canned responses picked by
+inspecting the system prompt's opening line — enough to exercise every
+real code path (`ollama.Client`'s HTTP call, JSON parsing,
+`planner.Compile`, `costguard.Assess`, the HTTP response shape) without
+needing model weights, a GPU, or non-deterministic output. This is the
+same technique `api/ai/aiapi/integration_test.go` uses in Go directly;
+this tool is for manual/browser verification, where an in-process fake
+isn't an option.
+
+Run it as a container on the compose network with a network alias of
+`ollama` (so `api`'s container can resolve the hostname), then point
+`OLLAMA_BASE_URL` at it via a throwaway compose override:
+
+```sh
+docker run -d --rm --name sentry-mock-ollama --network sentry_default --network-alias ollama \
+  -v "$(pwd)/hack/mock-ollama:/src" -w /src golang:1.25-alpine \
+  sh -c "go build -o /tmp/mock-ollama . && /tmp/mock-ollama"
+
+cat > /tmp/docker-compose.ai-verify.yml <<'EOF'
+services:
+  api:
+    environment:
+      OLLAMA_BASE_URL: "http://ollama:11434"
+      OLLAMA_MODEL: "test-model"
+EOF
+
+docker compose -f docker-compose.yml -f /tmp/docker-compose.ai-verify.yml up -d api
+```
+
+For `enterprise-api` instead of core `api` (needed to also exercise
+task 12's real audit-log write, since core has no `InteractionLogger`
+wired in), override that service's environment instead, same shape.
+
+Verify the routes are live:
+
+```sh
+curl -s -X POST localhost:8080/ai/translate -H 'Content-Type: application/json' \
+  -d '{"nlQuery":"errors in the last hour"}'
+# {"query":"earliest=-1h severity=ERROR","confidence":"high","compiles":true,"blocked":false}
+```
+
+**Clean up afterward** — don't leave the mock provider or the override
+wired into a stack anyone else might reach:
+
+```sh
+docker compose up -d api   # drops back to the plain env, no -f override
+docker rm -f sentry-mock-ollama
+rm /tmp/docker-compose.ai-verify.yml
+curl -s -o /dev/null -w '%{http_code}\n' -X POST localhost:8080/ai/translate -d '{}'
+# 404 -- confirms AI routes are unregistered again
+```
+
+## 3. Track A — Explain / Fix / Optimize / ghost-text
+
+With AI routes enabled (step 2) and the web dev server running against
+`localhost:8080`, open the Search page's query bar:
+
+- Type a partial query and pause — ghost text should appear inline
+  after ~300ms; Tab accepts it. Stop `sentry-mock-ollama` and confirm
+  ghost text just silently stops appearing (no error toast, no
+  console noise) — this is the "graceful degradation" requirement,
+  not incidental behavior.
+- Run a query that produces a parse or execution error, click "Try AI
+  fix" — the diff view should show the current vs. suggested query, and
+  Accept should replace the query bar's content without running it.
+- Run `severity=ERROR | stats count by host` (an unbounded aggregation
+  against the real seeded ClickHouse data from
+  `hack/benchmark-fixture`, per Phase 2/5's runbooks) — the inline cost
+  warning should appear, and clicking "Optimize" should show the real
+  mechanical rewrite (`earliest=-1h ` prepended).
+- Click "Explain this query" on any query — confirm the modal shows
+  prose, not raw JSON (a genuine model would return prose here too;
+  the mock's canned Explain response is deliberately plain text for
+  exactly this reason).
+
+## 4. Track B — natural-language translation
+
+Type a natural-language-shaped question into the query bar (4+ words,
+no `|`/comparison operator/`:`) — e.g. "show me errors from the last
+hour grouped by service". The "Interpret as natural language" affordance
+should appear; clicking it opens the translate modal, auto-translates,
+and shows both the generated query (editable) and an auto-fetched
+explanation. Confirm "Use this query" replaces the query bar content
+**without running anything** — no results table should appear until you
+separately click "Run query".
+
+CLI:
+
+```sh
+cd cli && go run ./cmd/sentryctl query --nl "errors in the last hour" --api http://localhost:8080
+# prints the translated query and, in an interactive terminal, prompts y/N before running
+```
+
+## 5. Audit logging (task 12)
+
+Requires `enterprise-api` (not core `api`) — core has no
+`InteractionLogger` wired in by design (see the design doc's "off unless
+configured" reasoning). With `enterprise-api` running against the
+compose profile that includes it and AI routes enabled per step 2's
+pattern applied to that service instead:
+
+1. Accept or dismiss a Fix/Optimize/Translate suggestion in the web UI.
+2. Confirm a row landed in `audit_log`:
+   ```sh
+   docker exec sentry-metadata-postgres psql -U sentry -d sentry_metadata \
+     -c "SELECT event_type, query_text, detail FROM audit_log WHERE event_type='ai_interaction' ORDER BY id DESC LIMIT 5;"
+   ```
+   `detail` should show `operation`/`accepted`/`edited` matching what you
+   just did in the UI.
+
+This exact path (minus the browser click, using the adapter directly)
+is what `enterprise/internal/audit/integration_test.go`'s
+`TestAIInteractionLoggerWritesAttributedToContextIdentity` already
+proves automatically — see "Running the automated suite" below to run
+it yourself instead of clicking through the UI.
+
+## 6. Running the automated suite
+
+```sh
+# api module -- includes the new mock-Ollama-backed integration tests
+# (api/ai/aiapi/integration_test.go), no live infra needed
+cd api && go build ./... && go vet ./... && go test ./...
+
+# enterprise module -- same, plus the live-Postgres audit tests (skipped
+# automatically unless AUDIT_TEST_POSTGRES_ADDR is set)
+cd enterprise && go build ./... && go vet ./... && go test ./...
+
+# live-Postgres audit tests specifically, against the real dev stack:
+docker run --rm --network sentry_default -v "$(pwd):/src" -w /src/enterprise \
+  -e AUDIT_TEST_POSTGRES_ADDR=metadata-postgres:5432 \
+  -e AUDIT_TEST_POSTGRES_PASSWORD=audit-writer-dev-only \
+  -e AUDIT_TEST_ADMIN_PASSWORD=sentry-dev-only \
+  golang:1.25-alpine go test ./internal/audit/... -v
+
+# cli module
+cd cli && go build ./... && go vet ./... && go test ./...
+
+# web
+cd web && npm run check && npm run build
+```
+
+All of the above pass in this environment as of this runbook. The first
+three don't need Docker or a live database at all except where noted —
+that's deliberate, see the design doc's CI-testability section.
+
+## Why a mock provider, not a real model
+
+Testing against a real Ollama server running the actual pinned
+`qwen2.5-coder:7b` needs a multi-gigabyte model download and either a
+GPU or a slow CPU-bound wait per request — infeasible for both this
+environment and, more importantly, for CI, and non-deterministic enough
+even at temperature 0 that a failing test wouldn't reliably mean a real
+regression. `hack/mock-ollama` and `api/ai/aiapi/integration_test.go`'s
+in-process equivalent both trade away model-quality coverage for
+plumbing coverage that's actually fast and deterministic enough to run
+every time — the same tradeoff this project already made for
+ClickHouse/Postgres-backed pieces of Phase 4 that only "compile and are
+unit-tested" in environments without live infrastructure. Model-quality
+verification (does the real model produce good translations for real
+questions) is real, disclosed future work — a periodic, human-run
+checklist item against a real local Ollama with the pinned model before
+a release, not a CI gate.