amogh-jahagirdar commented on code in PR #9400:
URL: https://github.com/apache/iceberg/pull/9400#discussion_r1442227949
##########
spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/RewriteDataFilesSparkAction.java:
##########
@@ -345,7 +345,7 @@ private Result doExecuteWithPartialProgress(
.noRetry()
.onFailure(
(fileGroup, exception) -> {
- LOG.error("Failure during rewrite group {}", fileGroup.info(),
exception);
+ LOG.warn("Failure during rewrite group {}", fileGroup.info(),
exception);
Review Comment:
Hm I think I get the rationale that during partial progress it could be too
noisy if we error logged every failure. But the issue is let's say most of the
commit tasks are failures, and we only see warn logs which are easy to get
masked. The only case we error log is when all tasks fail which seems
insufficient logging in case of failure.
I actually think error is fine here. Are you seeing a lot of noisiness in
this in any systems you may be running?
I also noticed that the non-partial progress case is also warn, but tbh that
also doesn't seem right. What do you think @manuzhang also cc @szehon-ho
@RussellSpitzer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
