lkindere opened a new issue, #14922:
URL: https://github.com/apache/iceberg/issues/14922
### Feature Request / Improvement
Hello,
I have noticed a problem with our compaction job and specifically we were
passing rewrite-all as string "true" instead of bool true, which seems to cause
different results, string "true" is silently ignored while boolean as expected,
does what it's supposed to.
logger.info("Compacting with extra options: " + extraOptions)
rewrite
...
.options(extraOptions.asJava)
.execute()
Example with string "true":
25/12/23 15:34:03 INFO compaction: Compacting with extra options:
Map(rewrite-all -> "true")
25/12/23 15:34:03 INFO SnapshotScan: Scanning table <table_name> snapshot
7763141941195437971 created at 2025-12-23T15:34:03.148+00:00 with filter
country_code = (hash-7c5a51e9)
bool true:
25/12/23 15:49:09 INFO compaction: Compacting with extra options:
Map(rewrite-all -> true)
**25/12/23 15:49:09 INFO SizeBasedFileRewriter: Configured to rewrite all
provided files in table <table_name>**
25/12/23 15:49:09 INFO SnapshotScan: Scanning table <table_name> snapshot
834249245530340774 created at 2025-12-23T15:47:24.057+00:00 with filter
country_code = (hash-7c5a51e9)
I would expect in this case, either for the string "true" to count as true,
or for it to throw some kind of error stating that the configuration is invalid
or was not taken into account (as it does if we try to pass an option that does
not exist). In this case it seems to get silently ignored, which is not ideal
for debugging.
(Using Iceberg 1.9.2, not sure if any changes related to this happened in
1.10.0)
### Query engine
Spark
### Willingness to contribute
- [ ] I can contribute this improvement/feature independently
- [x] I would be willing to contribute this improvement/feature with
guidance from the Iceberg community
- [ ] I cannot contribute this improvement/feature at this time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
