szehon-ho commented on code in PR #17957:
URL: https://github.com/apache/iceberg/pull/17957#discussion_r4150909103
##########
spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/source/SparkTable.java:
##########
@@ -94,10 +108,12 @@ public class SparkTable extends BaseSparkTable
TableCapability.OVERWRITE_DYNAMIC);
private final Schema schema; // effective schema (not necessarily current
table schema)
+ private final Set<Integer> mapKeyFieldIds;
private final Snapshot snapshot; // always set unless table is empty
private final String branch; // set if table is loaded for specific branch
private final TimeTravel timeTravel; // set if table is loaded for time
travel
private final Set<TableCapability> capabilities;
+ private final AtomicBoolean acceptAnySchemaWarningLogged = new
AtomicBoolean(false);
Review Comment:
this is a bit hacky and has some issue:
There are two scope limitations:
- Reusing the same wrapper suppresses subsequent warnings throughout its
lifetime.
- Reloading creates a fresh flag. Spark reloads after schema evolution, so
one query could warn again if it checks remaining candidate changes.
But yes its not so clean. Maybe we should just drop this idea, sorry. As
long as we have test coverage that nothing bad happens.
##########
spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/source/SparkTable.java:
##########
@@ -94,10 +108,12 @@ public class SparkTable extends BaseSparkTable
TableCapability.OVERWRITE_DYNAMIC);
private final Schema schema; // effective schema (not necessarily current
table schema)
+ private final Set<Integer> mapKeyFieldIds;
private final Snapshot snapshot; // always set unless table is empty
private final String branch; // set if table is loaded for specific branch
private final TimeTravel timeTravel; // set if table is loaded for time
travel
private final Set<TableCapability> capabilities;
+ private final AtomicBoolean acceptAnySchemaWarningLogged = new
AtomicBoolean(false);
Review Comment:
this is a bit hacky and has some issue:
There are two scope limitations:
- Reusing the same wrapper suppresses subsequent warnings throughout its
lifetime.
- Reloading creates a fresh flag. Spark reloads after schema evolution, so
one query could warn again if it checks remaining candidate changes.
Maybe we should just drop this idea, sorry. As long as we have test
coverage that nothing bad happens.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
