Skip to content

clickhouse: type inference via the xqlc analysis core#4521

Draft
kyleconroy wants to merge 3 commits into
mainfrom
claude/clickhouse-xqlc-core
Draft

clickhouse: type inference via the xqlc analysis core#4521
kyleconroy wants to merge 3 commits into
mainfrom
claude/clickhouse-xqlc-core

Conversation

@kyleconroy

@kyleconroy kyleconroy commented Jul 22, 2026

Copy link
Copy Markdown
Collaborator

Motivation

sqlc already has a working ClickHouse parser (internal/engine/clickhouse, backed by doubleclick), but no analysis for it: the ClickHouse catalog was a stub, the engine was not wired into the compiler, and there was no EngineClickHouse. The missing layer is a typed catalog + a query analyzer.

This PR merges the xqlc analysis core into sqlc as the future analysis engine, and makes ClickHouse the first engine to run on it, end to end. ClickHouse is a natural first target because it has a parser and zero existing analysis — this is net-new, with no regression surface for the other engines.

What's here

sqlc generate now works for ClickHouse, resolving column and parameter types entirely through the xqlc core.

1. core: land xqlc catalog + analyzer on sqlc's AST

  • internal/core/ — xqlc's SQLite-backed catalog (sql_* tables modeled on the Postgres system catalogs).
  • internal/core/analyzer/ — xqlc's dialect-neutral analyzer, repointed off xqlc's copy of the AST onto sqlc's internal/sql/ast. Single AST — no second copy, no converter.

2. clickhouse: analyze queries through the core catalog + analyzer

  • seed.go (ClickHouse dialect + built-in types) and schema.go (DDL → core catalog).
  • Fixes the ClickHouse converter to render nested type parameters (Nullable(String), Array(UInt64), Decimal(18,4)) instead of dropping them as TODO.

3. clickhouse: wire end-to-end sqlc generate onto the core

  • config: the clickhouse engine constant.
  • compiler: NewCompiler builds a core catalog seeded with the ClickHouse dialect; parseCatalog applies schema DDL to it; a dedicated parseQueryCore resolves columns/params via core/analyzer and assembles *compiler.Query, reusing only the shared query-metadata parsing.
  • codegen: a core-catalog → plugin.Catalog projection for models/enums, plus a ClickHouse → Go type map (Nullable(T)*T, the integer ladder, DateTimetime.Time, …).
  • clickhouse parser: computes statement byte-spans (doubleclick reports statement starts but not ends), so -- name: annotations fall inside each statement.
  • An endtoend case (clickhouse_select) exercises the full pipeline; its generated Go is committed as golden output.

Example generated output for SELECT id, name, tag, amount, created FROM events over id UInt64, name String, tag Nullable(String), amount Float64, created DateTime:

type ListEventsRow struct {
	ID      uint64
	Name    string
	Tag     *string      // Nullable(String)
	Amount  float64
	Created time.Time
}

Design decisions

  • One AST — sqlc's internal/sql/ast; the analyzer was repointed, not duplicated.
  • One catalog — the core catalog is the single source of truth for analysis and codegen on this path; the in-memory internal/sql/catalog is not used for ClickHouse.
  • No analyzer seam, no legacy analyze step — ClickHouse never enters analyzeQuery/inferQuery/outputColumns or the analyzer.Analyzer interface.

Follow-ups

  • Query coverage beyond SELECT in core/analyzer (INSERT/UPDATE/DELETE) and bind-parameter detection for ClickHouse ? placeholders.
  • Richer Go types (decimal.Decimal, uuid.UUID, netip.Addr, json.RawMessage) — these need import wiring; they currently resolve to string. Array-ness is not yet modeled on catalog attributes.
  • A testgen-style conformance suite against a real ClickHouse.

Still a draft: this is the first engine on the new core and the coverage above is intentionally scoped. When other engines migrate, they follow the same path and the legacy analyze machinery can be retired.

go build ./... and go vet pass; new unit tests and the clickhouse_select / parse_basic/clickhouse endtoend cases are green.

🤖 Generated with Claude Code

https://claude.ai/code/session_01XTGxNHW6v1S1YyC9FDSgrK

kyleconroy and others added 3 commits July 7, 2026 07:10
Port xqlc's core catalog (SQLite-backed sql_* catalog) and its
dialect-neutral query analyzer into internal/core, repointing the
analyzer from xqlc's copy of the AST onto sqlc's internal/sql/ast so
there is a single AST. No converter and no second AST package.

A smoke test drives the analyzer with sqlc's own PostgreSQL parser to
prove the repointed analyzer resolves columns, types, star expansion,
and aliases end-to-end against internal/sql/ast.

This is the first step of merging xqlc back into sqlc as the future
analysis core; ClickHouse will be the first engine wired onto it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XTGxNHW6v1S1YyC9FDSgrK
Wire ClickHouse onto the merged core: a dialect seed registering the
built-in ClickHouse types, and a DDL handler that populates the core
catalog from CREATE TABLE using sqlc's existing ClickHouse parser. A
smoke test proves the full vertical path — ClickHouse SQL -> sqlc's
ClickHouse parser -> internal/sql/ast -> core catalog + analyzer ->
PrepareResult — resolving column names, types, nullability, source
bindings, and star expansion, with none of the legacy compiler analyze
step involved.

Also fix the ClickHouse converter to render nested type parameters
(the inner type of Nullable(T)/Array(T), Decimal precision, etc.) into
TypeName.Name instead of dropping them as TODO nodes, so wrapped types
resolve to their effective scalar type.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XTGxNHW6v1S1YyC9FDSgrK
Add the EngineClickHouse engine and a dedicated compile path so
`sqlc generate` produces Go for ClickHouse entirely through the xqlc
core, bypassing the legacy compiler analyze step and the in-memory
sql/catalog:

- config: add the "clickhouse" engine constant.
- compiler: NewCompiler builds a core.Catalog seeded with the ClickHouse
  dialect; parseCatalog applies schema DDL to it; a new parseQueryCore
  resolves each query's columns and parameters via core/analyzer and
  assembles *compiler.Query, reusing only the shared query-metadata
  parsing. The legacy analyzeQuery/inferQuery/outputColumns path and the
  analyzer.Analyzer seam are never entered.
- codegen: project the core catalog into plugin.Catalog for model/enum
  generation, and add a ClickHouse -> Go type map (Nullable(T) -> *T,
  the integer ladder, Float32/64, String, DateTime -> time.Time, ...).
- clickhouse parser: compute statement byte-spans with a running offset
  and a semicolon scan (doubleclick reports statement starts but not
  ends), so leading "-- name:" annotations fall inside each statement.

An endtoend case (clickhouse_select) exercises the full pipeline and its
golden Go output is committed. Updating parse_basic/clickhouse's golden
reflects the corrected statement spans and now-detected query name/cmd.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XTGxNHW6v1S1YyC9FDSgrK
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant