szehon-ho commented on code in PR #8194:
URL: https://github.com/apache/iceberg/pull/8194#discussion_r1289016086
##########
core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java:
##########
@@ -192,6 +195,18 @@ public Table create() {
String baseLocation = location != null ? location :
defaultWarehouseLocation(identifier);
tableProperties.putAll(tableOverrideProperties());
+
+ if (Boolean.parseBoolean(
+
tableProperties.get(TableProperties.LOCATION_CONFLICT_DETECTION_ENABLED))) {
+ boolean conflictLocationDetected =
+ (ops.io() instanceof SupportsPrefixOperations)
Review Comment:
Small Nit: maybe make IO into a temp variable to save a few chars?
##########
core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java:
##########
@@ -192,6 +195,18 @@ public Table create() {
String baseLocation = location != null ? location :
defaultWarehouseLocation(identifier);
tableProperties.putAll(tableOverrideProperties());
+
+ if (Boolean.parseBoolean(
+
tableProperties.get(TableProperties.LOCATION_CONFLICT_DETECTION_ENABLED))) {
+ boolean conflictLocationDetected =
+ (ops.io() instanceof SupportsPrefixOperations)
+ ? !prefixEmpty((SupportsPrefixOperations) ops.io(),
baseLocation)
+ : ops.io().newInputFile(baseLocation).exists();
Review Comment:
I think we should still try to get consistent behavior and try to list in
all cases, even if IO is not SupportsPrefixIO. (ResolvingFileIO does not
implement that, right?) Otherwise it may be surprise to user.
Can we fallback to using File and call listStatus, like we do today in
DeleteOrphanFilesSparkAction, in case FileIO does not support it). We can
probably load this differently depending on catalog type, ie, have HiveCatalog
load File using its own HadoopConf, whereas others will take their default conf.
Not sure anyone has any thoughts, @RussellSpitzer ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
