szehon-ho commented on code in PR #7581:
URL: https://github.com/apache/iceberg/pull/7581#discussion_r1231401197
##########
core/src/main/java/org/apache/iceberg/PartitionsTable.java:
##########
@@ -49,6 +49,16 @@ public class PartitionsTable extends BaseMetadataTable {
new Schema(
Types.NestedField.required(1, "partition",
Partitioning.partitionType(table)),
Types.NestedField.required(4, "spec_id", Types.IntegerType.get()),
+ Types.NestedField.required(
+ 9,
+ "last_updated",
+ Types.TimestampType.withZone(),
+ "Partition last updated timestamp"),
+ Types.NestedField.required(
+ 10,
+ "last_updated_snapshot_id",
+ Types.LongType.get(),
+ "Partition last updated snapshot id"),
Review Comment:
Nit: can we improve the doc? (Id of snapshot that last updated this
partition)
##########
core/src/main/java/org/apache/iceberg/PartitionsTable.java:
##########
@@ -49,6 +49,16 @@ public class PartitionsTable extends BaseMetadataTable {
new Schema(
Types.NestedField.required(1, "partition",
Partitioning.partitionType(table)),
Types.NestedField.required(4, "spec_id", Types.IntegerType.get()),
+ Types.NestedField.required(
+ 9,
+ "last_updated",
Review Comment:
nit: how about 'last_updated_ms'? (matches Table metadata and also as
below, we have a suffix for snapshot_id)
##########
core/src/main/java/org/apache/iceberg/PartitionsTable.java:
##########
@@ -142,13 +158,14 @@ private static StaticDataTask.Row
convertPartition(Partition partition) {
private static Iterable<Partition> partitions(Table table, StaticTableScan
scan) {
Types.StructType partitionType = Partitioning.partitionType(table);
PartitionMap partitions = new PartitionMap(partitionType);
-
- try (CloseableIterable<ContentFile<?>> files = planFiles(scan)) {
- for (ContentFile<?> file : files) {
+ try (CloseableIterable<ManifestEntry<? extends ContentFile<?>>> entries =
planEntries(scan)) {
+ for (ManifestEntry<? extends ContentFile<?>> entry : entries) {
+ Snapshot snapshot = table.snapshot(entry.snapshotId());
+ ContentFile<?> file = entry.file();
Review Comment:
This is not done yet, right? (Something equivalent to
ManifestReader.liveEntries())
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
