This is an automated email from the ASF dual-hosted git repository.
rdblue pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/iceberg.git
The following commit(s) were added to refs/heads/main by this push:
new 14ddc5f611 Docs: Add Expression spec to list of specs (#17908)
14ddc5f611 is described below
commit 14ddc5f611a5227da5d1a4c9cd93e977a99372b8
Author: Alex Stephen <[email protected]>
AuthorDate: Wed Oct 7 13:45:09 2026 -0700
Docs: Add Expression spec to list of specs (#17908)
---
format/expressions-spec.md | 14 --------------
site/docs/expressions-spec.md | 1 +
site/mkdocs-dev.yml | 1 +
site/nav.yml | 1 +
4 files changed, 3 insertions(+), 14 deletions(-)
diff --git a/format/expressions-spec.md b/format/expressions-spec.md
index 2975648a25..bd01378ac1 100644
--- a/format/expressions-spec.md
+++ b/format/expressions-spec.md
@@ -24,7 +24,6 @@ This document defines the structure and behavior of
expressions for use in Icebe
Stored expressions are needed for use cases like data validations (`CHECK`
constraints) and default values (for instance, `current_timestamp()`).
Expressions are exchanged in use cases like server-side scan planning in the
catalog protocol.
-
## Overview
The goal of this specification is to define a simple expression structure and
avoid complexity.
@@ -39,7 +38,6 @@ This specification covers the structure of Iceberg
expressions and includes appe
[udf-spec]: https://iceberg.apache.org/udf-spec
-
## Structure
Iceberg expressions have two types:
@@ -47,19 +45,16 @@ Iceberg expressions have two types:
* **Value expressions** represent data values and transformations of values
(function calls) that produce any Iceberg type
* **Predicates** represent comparisons of value expressions as well as
combinations of predicates with boolean logic (and, or, not)
-
### Value expressions
A value expression is an expression that produces a typed value.
Value expressions can be one of three types: a constant value, a field
reference, or a function applied to zero or more value expressions.
-
#### Constant values
A constant or literal is the simplest type of value expression that represents
a specific typed value.
-
#### Field reference
A field reference represents the value of a specific field in a row. When an
expression is evaluated on a row, it returns the value of the field.
@@ -72,7 +67,6 @@ Named references are used when the identity of the column is
determined when the
The context in which an expression is used determines the type of references
that are valid. Iceberg specifications should document whether ID references,
named references, or both are allowed.
-
#### Apply function
An apply expression represents the result of a function applied to (or called
on) zero or more values produced by child value expressions or predicates.
@@ -93,7 +87,6 @@ Engines may document and use a catalog name to identify their
built-in functions
Function references are unambiguous and are not interpreted using session
context. Producers are responsible for resolving catalog, namespace, and name
if the session is relevant. For example, if a SQL engine uses its current
catalog and namespace to find a function, the resolved catalog and namespace
must be used to produce an unambiguous function reference.
-
#### Value expression types
The type produced by a value expression may change. For example, an ID
reference may produce a widened type after the underlying column's type is
promoted.
@@ -104,7 +97,6 @@ Function calls may produce different types when function
definitions change, and
If types are incompatible at runtime, implementations binding or evaluating
expressions may apply type promotion to align types for predicates and to
resolve functions. Implementations may choose when to promote values to
accommodate engines that differ in casting behavior. However, implementations
must fail rather than insert unsafe casts.
-
### Predicates
A predicate is a boolean expression that produces true or false.
@@ -113,7 +105,6 @@ Predicates can be constants (true or false), tests of a
value expression, compar
Value expressions are not valid predicates, even when the expression is
expected to return a boolean value. Value expressions must be compared or
tested to produce a predicate. For example, `is_empty(str_col)` is not a valid
predicate because it may produce `null`, but `is_empty(str_col) = true` is a
valid predicate.
-
#### Tests
Tests are predicates that test a single value expression, optionally using a
constant or set of constants. Constants must all have the same type and must be
non-null and non-NaN. Tests are:
@@ -129,7 +120,6 @@ Tests are predicates that test a single value expression,
optionally using a con
| `IN (constant set)` | any primitive | same as value | true iff the value
is equal to any constant |
| `NOT IN (constant set)` | any primitive | same as value | true iff the value
is not equal to any constant |
-
#### Comparisons
Comparisons are predicates that compare two value expressions with the same
primitive type.
@@ -184,7 +174,6 @@ For floating point values, comparison with NaN behaves
similarly to comparison o
* `a < b` and `a > b` are false when either operand is NaN; otherwise the IEEE
754 order is used
* `a <= b` is `(a = b) OR (a < b)`; `a >= b` is `(a = b) OR (a > b)`; both are
true when both operands are NaN and false when only one operand is NaN
-
#### Boolean logic
Predicates must use 2-valued boolean logic. Evaluation of all predicates must
produce `true` or `false`.
@@ -193,7 +182,6 @@ Engines that implement SQL 3-valued boolean logic must add
`IS NULL` and `IS NOT
Logical combinations are boolean operators applied to predicates. `AND` and
`OR` are binary operations and `NOT` is a unary operation. `AND`, `OR`, and
`NOT` do not accept null values because predicates cannot produce them.
-
### Compatibility with REST catalog expressions
Prior to this spec, REST APIs used a more restrictive, term-based form of
predicates and references. Those forms are now deprecated, but should be
supported for backward compatibility to allow older clients to interact with
newer REST catalog services.
@@ -208,7 +196,6 @@ Both server-side scan planning and the report endpoint
should continue to accept
Residuals passed from services back to clients that do not use the new syntax
would cause clients to fail. Services are allowed to omit the residual so that
it is calculated on the client side (intended to avoid duplicating large IN
filters). For compatibility, REST services should omit residuals from tasks,
but may include them if the service detects support for newer predicates (for
example, via client version).
-
## Appendix A: Iceberg functions
This section defines the functions in the `iceberg_functions` reserved catalog
name.
@@ -241,7 +228,6 @@ Note that `year`, `month`, and `hour` transforms produce
ordinal values and not
[bucket-ref]: spec/#bucket-transform-details
[truncate-ref]: spec/#truncate-transform-details
-
## Appendix B: JSON serialization
Iceberg expressions are serialized as JSON objects in table, view, and UDF
metadata, and in the REST protocol for catalogs.
diff --git a/site/docs/expressions-spec.md b/site/docs/expressions-spec.md
new file mode 120000
index 0000000000..bb3115a53e
--- /dev/null
+++ b/site/docs/expressions-spec.md
@@ -0,0 +1 @@
+../../format/expressions-spec.md
\ No newline at end of file
diff --git a/site/mkdocs-dev.yml b/site/mkdocs-dev.yml
index dc05e7ed94..48afbc8b92 100644
--- a/site/mkdocs-dev.yml
+++ b/site/mkdocs-dev.yml
@@ -108,6 +108,7 @@ nav:
- AES GCM Stream spec: gcm-stream-spec.md
- UDF spec: udf-spec.md
- Mumbling spec: mumbling-spec.md
+ - Expressions spec: expressions-spec.md
- Implementation status: status.md
exclude_docs: |
diff --git a/site/nav.yml b/site/nav.yml
index 2e52c16faf..ec9f59a753 100644
--- a/site/nav.yml
+++ b/site/nav.yml
@@ -127,4 +127,5 @@ nav:
- AES GCM Stream spec: gcm-stream-spec.md
- UDF spec: udf-spec.md
- Mumbling spec: mumbling-spec.md
+ - Expressions spec: expressions-spec.md
- Implementation status: status.md
