singhpk234 commented on code in PR #11041:
URL: https://github.com/apache/iceberg/pull/11041#discussion_r4589397916


##########
format/view-spec.md:
##########
@@ -160,7 +179,95 @@ Each entry in `version-log` is a struct with the following 
fields:
 | _required_  | `timestamp-ms` | Timestamp when the view's 
`current-version-id` was updated (ms from epoch) |
 | _required_  | `version-id`   | ID that `current-version-id` was set to |
 
-## Appendix A: An Example
+#### Storage Table Identifier
+
+The table identifier for the storage table that stores the precomputed results.
+
+| Requirement | Field name     | Description |
+|-------------|----------------|-------------|
+| _required_  | `namespace`    | A list of strings for namespace levels |
+| _required_  | `name`         | A string specifying the name of the table |
+| _optional_  | `catalog`      | A string specifying the name of the catalog. 
When `null` or not set, the catalog in which the materialized view is stored is 
used |
+
+### Storage table metadata
+
+This section describes additional metadata for the storage table that 
supplements the regular table metadata and is required for materialized views.
+The `refresh-state` property is set on the [snapshot 
summary](https://iceberg.apache.org/spec/#snapshots) property of a storage 
table snapshot to provide information about the state of the precomputed data.
+
+| Requirement | Field name      | Description |
+|-------------|-----------------|-------------|
+| _optional_  | `refresh-state` | A [refresh state](#refresh-state) record 
stored as a JSON-encoded string |
+
+Snapshots written by a refresh operation must set `refresh-state`.
+
+#### Refresh state
+
+Query engines maintain the precomputed data in the storage table by executing 
the materialized view’s query. During a refresh, the engine records the state 
of the view’s dependencies in the refresh state. Which dependencies are 
available depends on how source materialized views are treated, and the writer 
chooses which of them to record: all, a subset, or none. See [Appendix 
B](#appendix-b-example-freshness-criteria-and-recorded-dependencies) for 
examples. Each recorded dependency appears in `source-states` as a table entry 
or a view entry. A source materialized view may be recorded as a view entry for 
its definition and, optionally, a table entry for its storage table.
+
+The refresh state has the following fields:
+
+| Requirement | Field name                   | Description |
+|-------------|------------------------------|-------------|
+| _required_  | `refresh-start-timestamp-ms` | A timestamp of when the refresh 
operation was started |
+| _required_  | `view-version-id`            | The `version-id` of the 
materialized view when the refresh operation was performed |
+| _required_  | `source-states`              | A list of [source 
state](#source-state) records capturing the dependencies the writer chose to 
record; may be empty |
+
+When writing the refresh state, writers may leave source-states empty (e.g., 
when sources are non-Iceberg tables or a mechanism outside this spec determines 
whether the storage table reflects the result of the view query).
+
+##### Source state
+
+Source state records capture the state of objects referenced by a materialized 
view. Each record has a `type` field that determines its form:
+
+| Type    | Description |
+|---------|-------------|
+| `table` | An Iceberg table — either a source table in the dependency graph, 
or the storage table of a source materialized view |
+| `view`  | An Iceberg view in the dependency graph |
+
+A source materialized view may be recorded as a `view` entry referencing its 
view metadata and one or more `table` entries referencing its storage table or 
other source tables. These source table entries might be determined by 
recursively expanding its own dependencies.
+
+##### Source table state
+
+A source table record captures the state of a source table (including a source 
materialized view's storage table) at the time of the last refresh operation.
+
+| Requirement | Field name    | Description |
+|-------------|---------------|-------------|
+| _required_  | `type`        | A string that must be set to `table` |
+| _required_  | `name`        | A string specifying the name of the source 
table |
+| _required_  | `namespace`   | A list of strings for namespace levels |
+| _required_  | `catalog`     | A string specifying the name of the catalog. |
+| _required_  | `uuid`        | The uuid of the source table |
+| _optional_  | `snapshot-id` | The snapshot-id of the source table that was 
read during the refresh operation. Omitted when the source table had no 
snapshots at refresh time (e.g. a newly created, empty table) |
+| _optional_  | `ref`         | Branch or tag name of the source table being 
referenced in the view query |
+
+When `ref` is set, the source table is tracked against that branch or tag. 
When `ref` is not set, the source table is pinned to the recorded `snapshot-id` 
(e.g. `FOR VERSION AS OF <snapshot-id>`). Writers that read a branch or tag 
should record its name in `ref`.
+
+##### Source view state
+
+A source view record captures the state of a source view at the time of the 
last refresh operation.
+
+| Requirement | Field name   | Description |
+|-------------|--------------|-------------|
+| _required_  | `type`       | A string that must be set to `view` |
+| _required_  | `name`       | A string specifying the name of the source view 
|
+| _required_  | `namespace`  | A list of strings for namespace levels |
+| _required_  | `catalog`     | A string specifying the name of the catalog. |
+| _required_  | `uuid`       | The uuid of the source view |
+| _required_  | `version-id` | The version-id of the source view that was read 
during the refresh operation |
+
+#### Storage table creation and configuration
+
+When processing a `CREATE MATERIALIZED VIEW` statement, query engines should:
+
+1. Create the storage table as a regular Iceberg table with any specified 
configurations (partitioning, sort order, compression, etc.).

Review Comment:
   how are we planning to take these configuration and store it .. do we want 
these configuration to be applied per refresh ?
   
   How about things like encryption, one would want to store the underlying 
storage always encrypted 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to