sungwy commented on code in PR #13879:
URL: https://github.com/apache/iceberg/pull/13879#discussion_r3461722855
##########
open-api/rest-catalog-open-api.py:
##########
@@ -1525,6 +1673,40 @@ class AddSchemaUpdate(BaseUpdate):
)
+class ReadRestrictions(BaseModel):
+ """
+ Read restrictions for a table.
+ A reader evaluates the row filter against original, untransformed column
values, then applies required-column-projections to the surviving rows. Each
action must produce a value of the same type as the input column. If a reader
cannot apply any returned restriction (a filter expression or an action), it
must fail the query and must not silently return raw, partial, or empty results.
+ If a projection targets a field, that action governs every value reachable
through it; other projections in the same object that target a descendant of
that field have no effect.
+ An empty ReadRestrictions object (no required-column-projections and no
required-row-filter) imposes no restrictions and is equivalent to the field
being absent from the response.
+
+ """
+
+ required_column_projections: (
+ list[
+ MaskAlphanum
+ | MaskToFixedValue
+ | ReplaceWithNull
+ | ShowFirst4
+ | ShowLast4
+ | TruncateToYear
+ | TruncateToMonth
+ | Sha256Global
+ | Sha256QueryLocal
+ ]
+ | None
+ ) = Field(
+ None,
+ alias='required-column-projections',
+ description='A list of columns that require specific actions to be
applied when reading. A server must not return an action for a column whose
type is not listed in that action\'s "Applicable to" set. If absent or empty,
no required actions apply; columns not listed are not subject to any required
action.\n1. For each column listed, the reader must apply the specified action
before\n returning values for that column.\n\n2. The reader must replace all
output references to the column with the result\n of the action, presenting
the result under the original column name. For\n example, if the action for
column \'cc\' is mask-alphanum, the reader must\n return the masked value as
\'cc\' in the query output.\n\n3. A column must appear at most once in
required-column-projections.\n',
Review Comment:
Do we want to consider excluding `map` key from the list of columns that can
apply column projections?
Iceberg Spec doesn't seem opinionated on whether map column's keys are
unique or not, but our library implementations assume that they are unique.
Applying something like `MaskToFixedValue` or `ShowLast4` on a map key column
will result in errors.
##########
open-api/rest-catalog-open-api.py:
##########
@@ -1525,6 +1673,40 @@ class AddSchemaUpdate(BaseUpdate):
)
+class ReadRestrictions(BaseModel):
+ """
+ Read restrictions for a table.
+ A reader evaluates the row filter against original, untransformed column
values, then applies required-column-projections to the surviving rows. Each
action must produce a value of the same type as the input column. If a reader
cannot apply any returned restriction (a filter expression or an action), it
must fail the query and must not silently return raw, partial, or empty results.
+ If a projection targets a field, that action governs every value reachable
through it; other projections in the same object that target a descendant of
that field have no effect.
+ An empty ReadRestrictions object (no required-column-projections and no
required-row-filter) imposes no restrictions and is equivalent to the field
being absent from the response.
+
+ """
+
+ required_column_projections: (
+ list[
+ MaskAlphanum
+ | MaskToFixedValue
+ | ReplaceWithNull
+ | ShowFirst4
+ | ShowLast4
+ | TruncateToYear
+ | TruncateToMonth
+ | Sha256Global
+ | Sha256QueryLocal
+ ]
+ | None
+ ) = Field(
+ None,
+ alias='required-column-projections',
+ description='A list of columns that require specific actions to be
applied when reading. A server must not return an action for a column whose
type is not listed in that action\'s "Applicable to" set. If absent or empty,
no required actions apply; columns not listed are not subject to any required
action.\n1. For each column listed, the reader must apply the specified action
before\n returning values for that column.\n\n2. The reader must replace all
output references to the column with the result\n of the action, presenting
the result under the original column name. For\n example, if the action for
column \'cc\' is mask-alphanum, the reader must\n return the masked value as
\'cc\' in the query output.\n\n3. A column must appear at most once in
required-column-projections.\n',
Review Comment:
nit (I feel it's best to move away from `column name` and use `field-id`
instead):
```suggestion
description='A list of columns that require specific actions to be
applied when reading. A server must not return an action for a column whose
type is not listed in that action\'s "Applicable to" set. If absent or empty,
no required actions apply; columns not listed are not subject to any required
action.\n1. For each column listed, the reader must apply the specified action
before\n returning values for that column.\n\n2. The reader must replace all
output references to the column with the result\n of the action, presenting
the result under the original field-id. For\n example, if the action for
column \'cc\' is mask-alphanum, the reader must\n return the masked value as
\'cc\' in the query output.\n\n3. A column must appear at most once in
required-column-projections.\n',
```
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -3632,6 +3641,301 @@ components:
additionalProperties:
type: string
+ ReadRestrictions:
+ type: object
+ description: >
+ Read restrictions for a table.
+
+ A reader evaluates the row filter against original, untransformed
column
+ values, then applies required-column-projections to the surviving
rows.
+ Each action must produce a value of the same type as the input
column.
+ If a reader cannot apply any returned restriction (a filter
expression
+ or an action), it must fail the query and must not silently return
raw,
+ partial, or empty results.
+
+ If a projection targets a field, that action governs every value
reachable
+ through it; other projections in the same object that target a
descendant
+ of that field have no effect.
+
+ An empty ReadRestrictions object (no required-column-projections and
no
+ required-row-filter) imposes no restrictions and is equivalent to the
+ field being absent from the response.
+ example:
+ required-column-projections:
+ - field-id: 4
+ action: show-last-4
+ - field-id: 6
+ action: replace-with-null
+ - field-id: 8
+ action: truncate-to-year
+ - field-id: 10
+ action: sha-256-global
+ - field-id: 12
+ action: mask-alphanum
+ required-row-filter:
+ type: eq
+ term: region
+ value: US
+ properties:
+ required-column-projections:
+ description: >
+ A list of columns that require specific actions to be applied when
reading.
+ A server must not return an action for a column whose type is not
listed in
+ that action's "Applicable to" set. If absent or empty, no required
actions
+ apply; columns not listed are not subject to any required action.
+
+ 1. For each column listed, the reader must apply the specified
action before
+ returning values for that column.
+
+ 2. The reader must replace all output references to the column
with the result
+ of the action, presenting the result under the original column
name. For
+ example, if the action for column 'cc' is mask-alphanum, the
reader must
+ return the masked value as 'cc' in the query output.
Review Comment:
nit:
```suggestion
of the action, presenting the result under the original
field-id. For
example, if the action for field-id `9` is mask-alphanum, the
reader must
return the masked value as field-id `9` in the query output.
```
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -3632,6 +3641,301 @@ components:
additionalProperties:
type: string
+ ReadRestrictions:
+ type: object
+ description: >
+ Read restrictions for a table.
+
+ A reader evaluates the row filter against original, untransformed
column
+ values, then applies required-column-projections to the surviving
rows.
+ Each action must produce a value of the same type as the input
column.
+ If a reader cannot apply any returned restriction (a filter
expression
+ or an action), it must fail the query and must not silently return
raw,
+ partial, or empty results.
+
+ If a projection targets a field, that action governs every value
reachable
+ through it; other projections in the same object that target a
descendant
+ of that field have no effect.
Review Comment:
I personally stumbled on the phrase `governs every value reachable through
it` and thought it implied that the action was being inherited by the children
in a recursive way. What do you think about this sentence instead?
```suggestion
If a projection targets a parent field, that action overrides any
projection
that targets a descendant of that field.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
