matthijseikelenboom opened a new issue, #10454: URL: https://github.com/apache/iceberg/issues/10454
### Apache Iceberg version 1.5.2 (latest release) ### Query engine Spark ### Please describe the bug 🐞 ### Problem statement For work we had needed to have a concurrent read/write support for our data lake, which uses Spark. We where noticing some inconsistencies, so we wrote a test that can verify whether something like Iceberg adheres to ACID. We did however find that Iceberg fails this test. Now, it can be that we've wrongly configured Iceberg or that there is some mistake in the test code. My question is if someone of you can take a look at it, and perhaps can explain what is going wrong here. ### To Reproduce How to run the test and it's findings are described in the README of the repository, but here is a short run down Steps to reproduce the behavior: 1. Check out repo: [iceberg-acid-verification](https://github.com/matthijseikelenboom/iceberg-acid-verification) 2. Start Docker if not already running 3. Run the test [TransactionManagerTest.java](https://github.com/matthijseikelenboom/iceberg-acid-verification/blob/master/src/test/java/org/example/writer/TransactionManagerTest.java) 4. Observe that the test fails. ### Expected behavior - I expect that the full amount of transactions are executed and that Iceberg passes the ACID verification test ### Environment Description - Iceberg version : 1.5.2 - Spark version : 3.5.1 - Hive version : 4.0.0-beta-1 - Hadoop version : 3.2.2 - Storage (HDFS/S3/GCS..) : NTFS(Windows), APFS(macOS) & HDFS ### Additional context It's worth noting that other solutions, Hudi and Delta Lake, have also been tested this way. Hudi also didn't pass this test, but it was resolved with a bug ticket on GitHub [issue](https://github.com/apache/hudi/issues/11170). Delta Lake did pass the test. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
