Skip to content

Add semantic-layer agent: SMQ compiler and semantic analysis strategy - #1026

Draft
yvonneyx with Copilot wants to merge 2 commits into
aifrom
copilot/paper-reproduction-implementation
Draft

yvonneyx with Copilot wants to merge 2 commits into
aifrom
copilot/paper-reproduction-implementation

Conversation

Copilot AI commented Oct 16, 2026

Copy link
Copy Markdown
Contributor

Reproduces the semantic-layer-mediated NL2SQL agent from the paper. Semantic models and joins are defined explicitly. A deterministic compiler turns Semantic Model Queries (SMQ) into trusted SQL, and a constrained Think–Act loop composes the final query from that SQL. direct and loop are unchanged, so the three can be compared.

ava.semantic({
  models: [{
    name: 'orders', table: 'orders',
    dimensions: [{ name: 'channel', column: 'channel' }, { name: 'status', column: 'status' }],
    measures: [{ name: 'revenue', agg: 'sum', column: 'amount' }],
  }],
  joins: [], // { from, to, keys: [{ from, to }], type?: 'left' | 'inner' }
});

// Pure and deterministic: no LLM, no DB
const { sql } = ava.compileSMQ({
  metrics: ['orders__revenue'],
  filters: ["orders__status = 'paid'"],
  group_by: ['orders__channel'],
});

await ava.analyze('Paid revenue by channel', { strategy: { type: 'semantic', maxSteps: 20 } });
  • Semantic layer (src/semantic/)
    • Models are validated with zod. Elements are referenced as model__name. YAML is not supported; configs are an object or a JSON string.
    • JoinGraph finds the shortest join path between models (BFS) and orients the join keys along it.
    • SemanticRegistry.validateAgainstSchema checks at analyze time that every model's table exists in the engine schema.
  • Compiler (compiler/)
    • compileSMQ supports metrics, filters, group_by, order_by and limit.
    • Filters use a small fixed grammar: comparisons, IN, LIKE, BETWEEN, IS NULL and AND/OR/NOT. Subqueries and ; are rejected.
    • Metric filters go to HAVING. Mixing dimensions and metrics in one filter is an error.
    • Dialects are pluggable (DuckDB, PostgreSQL, ClickHouse, MySQL, ANSI). The default is inferred from engine.language.name.
  • semantic strategy (analysis/semantic.ts)
    • Each step is one JSON tool call: getModelDataElements, convertSmqToSql (with a 5-row preview) or execute.
    • execute is terminal and runs at most once. A failed run is not retried.
    • execute is rejected, without running, if no SMQ has compiled successfully yet, or if the final SQL uses a table that no compiled SMQ introduced.
    • maxSteps defaults to 20. A last-step hint tells the model to execute.
    • Compile and protocol errors are fed back to the model as observations.
  • API and wiring
    • AVA.semantic(), AVA.compileSMQ(), semantic on AnalysisRuntime, the strategy union entry and the exported types.
    • The README has a short section on the new strategy.

For review

  • Tool calls: the tool calls use a JSON text protocol through generateText, not native AI SDK tool calling. This keeps one call per step and makes the loop easy to mock. I can switch to native tools if you prefer.
  • Table check: the "only compiled tables" check is a textual scan of FROM/JOIN names. It handles CTEs and EXTRACT(… FROM …) but is not a SQL parser.
  • Join fan-out: metrics aggregated across a one-to-many join can be double-counted. There is no guard yet.
  • Not included: the CLI and the eval harness don't expose semantic, and there is no real-LLM accuracy comparison against direct/loop yet.

@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

✅ Preview is ready!

PR preview ✅ Ready ✅ Ready
🔗 Preview https://antvis-AVA-preview-pr-1026.surge.sh
📝 Commit5750455
⏱️ Build time0.01s
📦 Size9.6 MB (+188.5 KB ⬆️) · 171 files
🪵 LogsView logs
📱 MobileScan to open preview on mobile

↩️ Previous: ⚡️ 5750455 · antvis-AVA-preview-pr-1026.surge.sh (open ↗) · 2026-10-20 09:20:05 UTC

🤖 Powered by surge-preview

@yvonneyx
yvonneyx force-pushed the copilot/paper-reproduction-implementation branch from 1eaa20a to d2da641 Compare October 19, 2026 06:19
Co-authored-by: yvonneyx <55794321+yvonneyx@users.noreply.github.com>
@yvonneyx
yvonneyx force-pushed the copilot/paper-reproduction-implementation branch from d2da641 to ccc93cb Compare October 19, 2026 08:28
- Added support for YAML configuration in semantic models.
- Introduced `validateSemanticQuery` method in DuckDB engine for semantic validation.
- Created a new `semantic.ts` file to handle semantic query parsing and validation.
- Updated semantic model definitions to include metrics alongside dimensions and measures.
- Enhanced SQL dialect support to include BigQuery and Snowflake.
- Improved error handling for ambiguous references in semantic queries.
- Refactored the semantic registry to accommodate new metric types and validation logic.
- Updated the planner to handle new join types and conditions.
- Adjusted the schema validation to enforce stricter rules on model definitions.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants