Phase 2: unified query language spanning ClickHouse and Tantivy
Replaces the separate SQL-only /query and text-only /search endpoints with one pipe-syntax query language (plus raw SQL escape hatch) that compiles to a single IR and execution plan across both backends, so a query like `message:"connection refused" | stats count by host` runs as one request instead of two disjoint tools. - api/internal/querylang: lexer -> ast -> parser -> ir -> planner -> executor, each layer independently tested. - Execution generalizes Phase 1's proven Tantivy-prefilter pattern into a 4-way routing table (pure ClickHouse / text-only / text + aggregation / raw SQL passthrough). - Unified web query page and `sentryctl query`, both hitting the same POST /query endpoint. - Benchmarked against a real 1,022,000-row dataset (hack/benchmark-fixture); caught and fixed a real bug where the Tantivy prefilter cap (10,000) produced an IN-clause exceeding ClickHouse's default max_query_size -- lowered to 5,000, documented in docs/query-language-design.md and docs/phase-2-runbook.md. - docs/query-language-reference.md: customer-facing syntax reference.
This commit is contained in:
+8
-6
@@ -1,7 +1,8 @@
|
||||
// Command api is Sentry's query API: POST /query (raw SQL, SELECT-only)
|
||||
// and POST /search (free-text, via the search service). See
|
||||
// internal/queryapi for why these are plain REST rather than the pinned
|
||||
// gRPC+gateway pattern.
|
||||
// Command api is Sentry's query API: a single POST /query endpoint
|
||||
// accepting either the pipe syntax or raw SQL, compiled and routed
|
||||
// across ClickHouse and search by internal/querylang. See
|
||||
// internal/queryapi and /docs/query-language-design.md for why this is
|
||||
// plain REST rather than the pinned gRPC+gateway pattern.
|
||||
package main
|
||||
|
||||
import (
|
||||
@@ -16,6 +17,7 @@ import (
|
||||
"github.com/ClickHouse/clickhouse-go/v2"
|
||||
|
||||
"github.com/sentry/sentry/api/internal/config"
|
||||
"github.com/sentry/sentry/api/internal/querylang/executor"
|
||||
"github.com/sentry/sentry/api/internal/queryapi"
|
||||
"github.com/sentry/sentry/api/internal/searchclient"
|
||||
)
|
||||
@@ -58,8 +60,8 @@ func main() {
|
||||
}
|
||||
defer search.Close()
|
||||
|
||||
exec := queryapi.NewExecutor(conn)
|
||||
handler := queryapi.NewHandler(logger, exec, search, cfg.QueryTimeout, cfg.CORSAllowedOrigin)
|
||||
sqlRunner := executor.NewChRunner(conn)
|
||||
handler := queryapi.NewHandler(logger, sqlRunner, search, cfg.QueryTimeout, cfg.CORSAllowedOrigin)
|
||||
|
||||
srv := &http.Server{
|
||||
Addr: cfg.HTTPListenAddr,
|
||||
|
||||
Reference in New Issue
Block a user