szehon-ho commented on issue #10454:
URL: https://github.com/apache/iceberg/issues/10454#issuecomment-2154449447

   Yea , I think @RussellSpitzer has a good point.
   
   I think this code 
https://github.com/matthijseikelenboom/iceberg-acid-verification/blob/master/src/main/java/org/example/writer/TransactionWriter.java#L167
  swallows the exception.
   
   Iceberg isolation was designed work mostly by throwing exceptions like 
ValidationException when it detects concurrent changes to the table that would 
break the given isolation level.  The user then needs to retry the query.
   
   Running this test, I saw some instances of IllegalStateException
   ```
   Runtime file filtering is not possible: the table has been concurrently 
modified. "
               + "Row-level operation scan snapshot ID: %s, current table 
snapshot ID: %s. "
               + "If an external process modifies the table, enable table 
caching in the catalog. "
               + "If multiple threads modify the table, use independent Spark 
sessions in each thread.",
   ```
   
   This is related to ValidationException but actually thrown because a certain 
optimization for MERGE INTO (runtime file-filtering) seems to actually requires 
the table to be completely up to date.  This can be turned off if you wish: 
spark.sql.optimizer.runtime.rowLevelOperationGroupFilter.enabled=false 
   
   But for either case, the test should handle the exception and retry.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org

Reply via email to