[akrish1982]

Summary

Adds ICEBERG to the Ossie Dialect enum, closing out #142's proposal to
support Apache Iceberg's native expression language as a "vendor" dialect,
similar to the existing ANSI_SQL / OSSIE_SQL_2026 handling.

ICEBERG expressions are boolean predicate strings in the form accepted by
PyIceberg's expression parser (pyiceberg.expressions.parser.parse), e.g.
order_date >= '2026-01-01' AND region = 'EMEA'. That parser is the
reference reader for the dialect, so consumers have something to validate
against. Function calls, including partition transforms such as
month(col) or bucket(16, col), are explicitly not part of the dialect:
the parser rejects them, and expressions are predicates only.

Changes:

• core-spec/ossie-schema.json: add ICEBERG to $defs.Dialect.enum
• core-spec/spec.yaml: regenerated via python3 scripts/generate-spec-types.py --apply
• core-spec/spec.md: add ICEBERG row to the Dialects table
• core-spec/expression_language.md: new "The ICEBERG Dialect" section
  documenting the supported predicates, operand rules, what is not
  supported, and examples (each form checked against PyIceberg 0.12.0)
• scripts/generate-spec-types.py: add description for ICEBERG
• python/src/ossie/models.py: add OssieDialect.ICEBERG
• validation/validate.py: skip ICEBERG from SQL parsing/validation
  (not SQL text), same treatment as MDX/TABLEAU/MAQL/SIGMA/THOUGHTSPOT/DAX
• Tests added in python/tests/test_models.py and validation/tests/test_validate.py

Related Issues

Fixes #142

Checklist

Specification

• Spec changes are included in core-spec/ and follow the existing structure
• Spec changes have been discussed on the mailing list or in a linked issue
• Breaking changes to the spec are clearly called out in the summary (none — purely additive enum value)

Ontology

• Ontology changes in ontology/ are consistent with spec changes (n/a — dialect enum only, no ontology impact)
• New or modified terms are defined and documented

Converters

• Converter logic in converters/ is updated to reflect spec or ontology changes (n/a — no converter currently targets Iceberg; dialect is pass-through for existing converters)
• New converters include tests under the converter's test directory
• If adding a new converter, .github/labeler.yml is updated with the new path

Validation

• Validation rules in validation/ are updated if the spec changed
• New validation cases are covered by tests

Documentation

• docs/ is updated to reflect any user-facing changes (documented in core-spec/expression_language.md and spec.md, which are the canonical spec docs)
• New features or behaviors are documented with examples where appropriate
• CONTRIBUTING.md is updated if the contribution process changed (n/a)

Examples

• examples/ are added or updated for any new spec constructs or converter support (n/a — no new top-level construct, just a dialect value)

Tests

• All existing tests pass (pytest / CI green) — ran pytest python/tests/test_models.py (42 passed) and pytest validation/tests/test_validate.py (68 passed)
• New functionality is covered by tests

Compliance

• ASF license headers are present on all new source files (no new files added, only edits to existing licensed files)
• No third-party dependencies are added without PMC/IPMC approval
• If third-party source code is vendored/copied (not just declared as a dependency), NOTICE and/or LICENSE have been updated per ASF policy (n/a — no vendored code)

[github-actions]

Thank you for your first contribution to Apache Ossie (incubating)!

A committer will review your PR. In the meantime, please ensure:

• All CI checks pass
• ASF license headers are present on new source files
• Tests cover new functionality

No ICLA is required to contribute. If you are elected as a committer in the future, you will need an Individual Contributor License Agreement (ICLA) on file with the ASF.

Questions? Ask in the PR comments or join us on Slack.

We appreciate your contribution!

[christianeu-db]

Thanks for submitting this PR. This would be one to discuss in the dev list. We have a number of non-SQL dialects but they don't have dedicated reference guides in the core spec. It's probably a good idea for vendors to put more details on their dialects and even provide a reference parser / SQL compiler.

As we think about portability, as the number of vendor-specific dialects Ossie increases, the greater chance for behavioral divergence as consumers need to build their own parsers / compilers.

[akrish1982]

Thanks for the feedback, @christianeu-db — makes sense on both points.

On process: I hadn't yet posted this to dev@ before opening the PR; I'll start a  [DISCUSS]  thread on  dev@ossie.apache.org  proposing the  ICEBERG  dialect and link back to this PR and #142, per the spec-change process in CONTRIBUTING.md. Happy to leave this PR open as the concrete artifact for that discussion, or convert it to draft until the discussion period settles — whichever the community prefers.

On the divergence concern: agreed that an enum value plus a function-name table isn't enough for a non-SQL dialect. I'd like to propose closing that gap for  ICEBERG  specifically (and potentially as a pattern for the existing non-SQL dialects) by:

• Expanding  expression_language.md 's  ICEBERG  section into a fuller reference (grammar/BNF for the expression strings, exhaustive transform/predicate list, worked examples)
• Providing a small reference parser/validator (similar to how  validate.py  uses  sqlglot  for SQL dialects) so implementations aren't each writing their own from scratch

Would that scope address your concern, or would you prefer this land as a smaller "propose the dialect name" PR now with the reference guide/parser as clearly-tracked follow-up work? Also open to folding this into a broader dev@ discussion about a documentation/parser bar for all non-SQL dialects, if that's the direction the community wants to take.

[kayemkim]

Thanks for picking up #142 with all six places that mirror the dialect list moving together (schema, generated spec.yaml and spec.md, models.py, generate-spec-types.py, validate.py); the merged tree passes the validation workflow including the spec.yaml sync check, and the only CI item left is the title check, which wants docs(core-spec): ... or Core-spec: ... rather than a free-form title.

One input for the dev@ thread you are planning. The month(order_date) / equal(col, v) strings in the new section are the Java Expressions factory method names written out as text, and as far as I can find nothing in Iceberg parses that form. What Iceberg does define is the partition-spec transform strings in the table spec (identity, bucket[16], truncate[4], year, month, day, hour, void, applied to a source field rather than written as month(col)), the REST catalog JSON for predicates ({"type":"eq","term":"name","value":25}, transform terms as {"type":"transform","transform":"bucket[100]","term":"id"}, type names eq, not-eq, lt-eq, starts-with, is-nan, ...), and PyIceberg's SQL-like string parser (foo is not null and (foo < 5 or 10 < foo)), which rejects function calls. So ICEBERG would be the first row in the dialect table whose text form has no reference reader or writer on either side, which is Chris's divergence point in concrete terms. Anchoring the dialect on the REST JSON form, or defining it as the string PyIceberg's parser accepts, would give consumers something to validate against.

[akrish1982]

Thanks @kayemkim and @christianeu-db. I've pushed an update that addresses both points.

Dialect is now anchored on a reference reader. @kayemkim, you were right that nothing in Iceberg parses the month(order_date) / equal(col, v) form. I've redefined ICEBERG as the predicate string accepted by PyIceberg's expression parser (pyiceberg.expressions.parser.parse), as you suggested. A string is a valid ICEBERG expression if and only if it parses there.

The rewritten section in core-spec/expression_language.md now:

• lists the supported predicates: comparisons, BETWEEN, IS [NOT] NULL, IS [NOT] NAN, [NOT] IN, [NOT] LIKE 'prefix%', and AND / OR / NOT with parentheses
• spells out the operand rules: one column against one literal, no column-to-column comparisons, no typed literals such as DATE '...'
• states explicitly that function calls are not supported, including partition transforms such as month(col) or bucket(16, col), and that expressions are predicates only

Every form and example in the section was checked against PyIceberg 0.12.0.

On divergence. @christianeu-db, I hope this addresses your concern for this dialect: consumers have an existing open-source parser to validate against instead of writing their own. Since the dialect is predicate-only, it applies only where a boolean expression is expected (filters), not to value-computing fields or metrics. I'm happy to make that scoping more prominent in the spec if that would help.

Housekeeping.

• Title changed to docs(core-spec): add ICEBERG expression dialect for the title check.
• Merged current main and resolved the conflict in validation/tests/test_validate.py.
• scripts/generate-spec-types.py --check passes, as do the Validation CI commands locally.
• The workflows on the latest commits are waiting on approval. Could a committer approve the CI run?

dev@ discussion: https://lists.apache.org/thread/bowhhbbh93jp04x1m1y1ypymppvcjwbh

I'll keep this PR open as the concrete artifact for that thread and follow the spec-change process from there.

[christianeu-db]

Thanks @kayemkim and @christianeu-db. I've pushed an update that addresses both points.

Dialect is now anchored on a reference reader. @kayemkim, you were right that nothing in Iceberg parses the month(order_date) / equal(col, v) form. I've redefined ICEBERG as the predicate string accepted by PyIceberg's expression parser (pyiceberg.expressions.parser.parse), as you suggested. A string is a valid ICEBERG expression if and only if it parses there.

The rewritten section in core-spec/expression_language.md now:

• lists the supported predicates: comparisons, BETWEEN, IS [NOT] NULL, IS [NOT] NAN, [NOT] IN, [NOT] LIKE 'prefix%', and AND / OR / NOT with parentheses
• spells out the operand rules: one column against one literal, no column-to-column comparisons, no typed literals such as DATE '...'
• states explicitly that function calls are not supported, including partition transforms such as month(col) or bucket(16, col), and that expressions are predicates only

Every form and example in the section was checked against PyIceberg 0.12.0.

On divergence. @christianeu-db, I hope this addresses your concern for this dialect: consumers have an existing open-source parser to validate against instead of writing their own. Since the dialect is predicate-only, it applies only where a boolean expression is expected (filters), not to value-computing fields or metrics. I'm happy to make that scoping more prominent in the spec if that would help.

Housekeeping.

• Title changed to docs(core-spec): add ICEBERG expression dialect for the title check.
• Merged current main and resolved the conflict in validation/tests/test_validate.py.
• scripts/generate-spec-types.py --check passes, as do the Validation CI commands locally.
• The workflows on the latest commits are waiting on approval. Could a committer approve the CI run?

dev@ discussion: https://lists.apache.org/thread/bowhhbbh93jp04x1m1y1ypymppvcjwbh

I'll keep this PR open as the concrete artifact for that thread and follow the spec-change process from there.

Responded to your thread on the list. If the ICEBERG dialect is in its own separate file (possibly out the core spec folder to avoid a vote every time you need to change it), I'm happy to see this merged