singhpk234 commented on code in PR #16859:
URL: https://github.com/apache/iceberg/pull/16859#discussion_r3814630901
##########
core/src/main/java/org/apache/iceberg/MetadataTableUtils.java:
##########
@@ -70,6 +70,8 @@ private static Table createMetadataTableInstance(
return new SnapshotsTable(baseTable, metadataTableName);
case METADATA_LOG_ENTRIES:
return new MetadataLogEntriesTable(baseTable, metadataTableName);
+ case TABLE_PROPERTIES_LOG:
+ return new TablePropertiesLogTable(baseTable, metadataTableName);
Review Comment:
I don't hold a very strong opinion here, I am fine either ways :), but
sharing my thoughts here I feel as far as adding new columns to existing tables
is concerned we have done that couple of times !
partitions
- spec_id — https://github.com/apache/iceberg/pull/4516
- last_updated_at, last_updated_snapshot_id —
https://github.com/apache/iceberg/pull/7581
- total_data_file_size_in_bytes — https://github.com/apache/iceberg/pull/7920
- rename of last-updated timestamp column —
https://github.com/apache/iceberg/pull/8003
manifests / all_manifests
- content + added/existing/deleted_delete_files_count —
https://github.com/apache/iceberg/pull/4764
- reference_snapshot_id (all_manifests) —
https://github.com/apache/iceberg/pull/4847
we can also just say its null ... if the file is missing, also our default
max version is 100, i don't expect that much of an overhead !
> Yea my concern is mainly if there's something sensitive in table
properties that we'd have to guard more now
+1, infact in EMR we had to gaurd this all the way up to these metadata
tables :
https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-lf-limitations-cont.html
, even paths are sensitive as sometime we have partition_col in the file name
... which can reveal which partition are there, but i feel this is more of the
platform responsibility, from the glance of the table props defined nothing
sensistive comes up to me, but folks can always add **anything** in the table
prop, platforms may feel maintain an allowlist the props they wanna show though
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
