steveloughran commented on code in PR #5144:
URL: https://github.com/apache/hadoop/pull/5144#discussion_r1048802203
##########
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java:
##########
@@ -459,6 +458,13 @@ public void initialize(URI name, Configuration
originalConf)
AuditSpan span = null;
try {
LOG.debug("Initializing S3AFileSystem for {}", bucket);
+ if (LOG.isTraceEnabled()) {
+ // log a full trace for deep diagnostics of where an object is created,
+ // for tracking down memory leak issues.
+ LOG.trace("Filesystem for {} created; fs.s3a.impl.disable.cache = {}",
+ name, originalConf.getBoolean("fs.s3a.impl.disable.cache", false),
+ new RuntimeException(super.toString()));
Review Comment:
1. generating that runtime exception creates a stack trace which then gets
logged in the trace call, so we can see who is creating the fs instance
2. calling super.toString() so we get the minimal instance id rather than
the expanded tostring with all the stats and stuff. This is initialize, most of
those values are unset so it would only confuse.
it is that stack trace which is the most critical thing as it lets us track
down where FS instances are being created and then not closed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
