yadavay-amzn commented on code in PR #16031:
URL: https://github.com/apache/iceberg/pull/16031#discussion_r3119365563
##########
spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/CommitMetadata.java:
##########
@@ -56,7 +56,7 @@ public static <R, E extends Exception> R withCommitProperties(
ExceptionUtil.castAndThrow(e, exClass);
return null;
} finally {
- COMMIT_PROPERTIES.set(ImmutableMap.of());
+ COMMIT_PROPERTIES.remove();
Review Comment:
@singhpk234 You're right that the practical memory savings are negligible
since `initialValue()` returns `ImmutableMap.of()` anyway. The motivation is
purely best-practice — `remove()` is the idiomatic cleanup pattern for
ThreadLocals, and it's a one-line change across 4 Spark versions.
##########
spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/CommitMetadata.java:
##########
@@ -56,7 +56,7 @@ public static <R, E extends Exception> R withCommitProperties(
ExceptionUtil.castAndThrow(e, exClass);
return null;
} finally {
- COMMIT_PROPERTIES.set(ImmutableMap.of());
+ COMMIT_PROPERTIES.remove();
Review Comment:
@amogh-jahagirdar The description incorrectly referenced executor threads —
this runs on the driver commit path only. Removed the tests and kept just the
source changes per your suggestion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
