yangshangqing95 commented on code in PR #17134:
URL: https://github.com/apache/iceberg/pull/17134#discussion_r3554363153
##########
spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestMetadataTables.java:
##########
@@ -483,10 +496,14 @@ public void testAllFilesPartitioned() throws Exception {
Schema entriesTableSchema =
TypeUtil.selectNot(
- Spark3Util.loadIcebergTable(spark, tableName +
".entries").schema(),
+ Spark3Util.loadIcebergTable(
+ spark, tableName + "." +
MetadataTableType.ENTRIES.tableName())
+ .schema(),
Set.of(DataFile.FIRST_ROW_ID.fieldId()));
Schema filesTableSchema =
- Spark3Util.loadIcebergTable(spark, tableName +
".all_data_files").schema();
+ Spark3Util.loadIcebergTable(
Review Comment:
Since reading the metadata tables does not require reading any actual files,
the file path would remain unchanged regardless of how many commits are made.
Therefore, this case needs to be handled separately.
##########
spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestMetadataTables.java:
##########
@@ -483,10 +496,14 @@ public void testAllFilesPartitioned() throws Exception {
Schema entriesTableSchema =
TypeUtil.selectNot(
- Spark3Util.loadIcebergTable(spark, tableName +
".entries").schema(),
+ Spark3Util.loadIcebergTable(
+ spark, tableName + "." +
MetadataTableType.ENTRIES.tableName())
+ .schema(),
Set.of(DataFile.FIRST_ROW_ID.fieldId()));
Schema filesTableSchema =
- Spark3Util.loadIcebergTable(spark, tableName +
".all_data_files").schema();
+ Spark3Util.loadIcebergTable(
Review Comment:
Since reading the metadata tables does not require reading any actual files,
the file path would remain unchanged regardless of how many commits are made.
Therefore, this case needs to be handled separately.
##########
core/src/test/java/org/apache/iceberg/hadoop/TestTableSerialization.java:
##########
@@ -194,7 +194,11 @@ public void testSerializableMetadataTablesPlanning(boolean
fromSerialized) throw
Set<CharSequence> newFiles = getFiles(getMetaDataTable(table, type));
// Expect that the new data is changed in the meantime
- assertThat(deserializedFiles).isNotEqualTo(newFiles);
+ if (type == MetadataTableType.METADATA_TABLES) {
Review Comment:
Since reading the metadata tables does not require reading any actual files,
the file path would remain unchanged regardless of how many commits are made.
Therefore, this case needs to be handled separately.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
