RussellSpitzer commented on PR #17822: URL: https://github.com/apache/iceberg/pull/17822#issuecomment-6140668969
I’m not sure I see the complexity here. We have lots of operations that produce a snapshot without changing data. It feels like the writer producing the snapshot knows this state and otherwise I believe it would require two separate updates in the rest spec. One adding the snapshot and one updating the constraint with information about that snapshot. On Mon, Oct 12, 2026 at 11:52 PM Anoop Johnson ***@***.***> wrote: > ***@***.**** commented on this pull request. > ------------------------------ > > In format/spec.md > <https://github.com/apache/iceberg/pull/17822#discussion_r4253036087>: > > > +To express SQL `CHECK` semantics for an optional field, the stored expression must make the null case explicit. For example, SQL `CHECK (price >= 0)` for an optional `price` field is stored as the expression for `price >= 0 OR price IS NULL`. This is unnecessary for required fields, which can never be null. > + > +#### Constraints and Schema Evolution > + > +A constraint references fields by ID, so schema changes interact with constraints as follows. The referenced fields of a `check` constraint are the field IDs in its `expression`; the referenced fields of a `unique` or `primary-key` constraint are its `field-ids`. > + > +* Renaming or reordering a referenced field is allowed; the constraint continues to apply to the same fields. > +* Every field referenced by a constraint must exist in the table's current schema. A schema change that removes a referenced field must be rejected unless the constraint is removed in the same change. > +* The type of a field referenced by a `check` constraint must not be changed, even for type promotions that are otherwise allowed, because the constraint's `expression` contains literals that are bound to the field's type. The type of a field referenced by a `unique` or `primary-key` constraint may be changed by a type promotion, which preserves values and therefore preserves distinctness. > +* A field referenced by a `primary-key` constraint must not be made optional, and a struct that contains such a field must not be made optional, because a `primary-key` requires its fields to be non-null. > + > +Only a constraint's `name` and `enforced` fields may be changed in place, and a writer that changes either must update `timestamp-ms`. Changing the `expression` of a `check` constraint or the `field-ids` of a `unique` or `primary-key` constraint changes what the constraint requires, so it must be done by removing the constraint and adding a new one with a new `constraint-id`, so that statuses recorded for the old definition are not read as applying to the new one. > + > +#### Constraint Validation > + > +Enforcement and constraint status are tracked separately. Whether writers must verify the rows that they add is a property of a constraint, tracked by `enforced`. Whether a table is known to satisfy a constraint is a property of a table's data, tracked per snapshot by `constraint-statuses`. > > I have a design question. Why is the constraint status attached to the > snapshot and not the constraint itself? I missed the syncs on constraints, > so I've probably missed a prior discussion on this topic. I looked at new > constraints API in Spark 4.2 and looks like it's attached > <https://github.com/apache/spark/blob/26e6dc00ebc8b3f5f2242760956288a4da624b76/sql/catalyst/src/main/java/org/apache/spark/sql/connector/catalog/constraints/Constraint.java#L48> > to the constraint itself. From a cursory Google search, Snowflake > information_schema can be used > <https://docs.snowflake.com/en/sql-reference/info-schema/table_constraints> > the constraint properties, but not their validation status. > > The per snapshot model is a bit complex. For instance, if an engine wants > to record a new status, it needs to produce a new snapshot. I've seen a > couple of places in the code bases that reject empty writes (e.g. > BaseRewriteFiles). Writers need to carry forward the statuses. > > But it has one advantage: it is per-branch and time-travel validity. If > that's the motivation, there seems to be a gap: constraint definitions > aren't versioned the way schemas are. A snapshot records schema-id and old > schemas are retained, but for constraints a snapshot only records statuses > keyed by constraint-id, and the definitions are just the current set. So a > status recorded on a branch or an older snapshot can reference a constraint > ID whose definition has since been dropped or replaced (changing a > predicate or key is drop + re-add with a new id), with no way to recover > what it was. So the status is per-branch but the definition isn't. > > So my question is what use case drove attaching status to the snapshot? > > - If it's per-branch / time-travel validity, should constraint > definitions be versioned or retained like schemas so a status is always > interpretable where it was recorded? > - If the branching use case isn't a near-term goal, would moving > validation status onto the constraint be simpler and sidestep the question > above? > > — > Reply to this email directly, view it on GitHub > <https://github.com/apache/iceberg/pull/17822?email_source=notifications&email_token=AADE2YMRJUYBBGDWV7SMKQD5TVOEHA5CNFSNUABKM5UWIORPF5TWS5BNNB2WEL2QOVWGYUTFOF2WK43UKJSXM2LFO4XTKNBZGY4DEMJSGE22M4TFMFZW63VHNVSW45DJN5XKKZLWMVXHJLDGN5XXIZLSL5RWY2LDNM#pullrequestreview-5496821215>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AADE2YMG37UEDMZYO4M63RD5TVOEHAVCNFSNUABFKJSXA33TNF2G64TZHMYTKOBSGU3DINZZHNEXG43VMU5TKMRVGQYTMOJZGQ4KC5QC> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> > -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
