[
https://issues.apache.org/jira/browse/HADOOP-19624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18030908#comment-18030908
]
ASF GitHub Bot commented on HADOOP-19624:
-----------------------------------------
matt-duran-starburst commented on code in PR #7852:
URL: https://github.com/apache/hadoop/pull/7852#discussion_r2442738562
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClientThrottlingAnalyzer.java:
##########
@@ -172,6 +175,22 @@ public boolean suspendIfNecessary() {
return false;
}
+ /**
+ * Closes the throttling analyzer and releases associated resources.
+ * This method cancels the internal timer and cleans up any pending timer
tasks.
+ * It is safe to call this method multiple times.
Review Comment:
Updated!
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClientThrottlingIntercept.java:
##########
@@ -223,4 +224,18 @@ private static long getContentLengthIfKnown(String range) {
}
return contentLength;
}
+
+ /**
+ * Closes the throttling intercept and releases associated resources.
+ * This method closes both the read and write throttling analyzers.
Review Comment:
Updated!
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsNoOpThrottlingIntercept.java:
##########
@@ -40,4 +42,12 @@ public void updateMetrics(final AbfsRestOperationType
operationType,
public void sendingRequest(final AbfsRestOperationType operationType,
final AbfsCounters abfsCounters) {
}
+
+/**
+ * No-op implementation of close method.
Review Comment:
Updated!
> [Bug Report] Thread leak in ABFS AbfsClientThrottlingAnalyzer
> -------------------------------------------------------------
>
> Key: HADOOP-19624
> URL: https://issues.apache.org/jira/browse/HADOOP-19624
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Affects Versions: 3.5.0, 3.4.1
> Reporter: Anuj Modi
> Priority: Major
> Labels: pull-request-available
>
> Bug reported by Matt over common-dev discussion.
> > What seems to be the issue is that the timer tasks are cleaned up but
> > the timer threads themselves are never actually cleaned up. This will
> > eventually lead to an OOM since nothing is collecting these. I was
> > able to reproduce this locally in 3.3.6 and 3.4.1 but I believe that
> > it would affect any version that relies on autothrottling for ABFS.
> >
> > I was also able to make a quick fix as well as confirm a workaround --
> > the long term fix would be to include `timer.cancel()` and
> > `timer.purge()` in a method for AbfsClientThrottlingAnalyzer.java. The
> > short term workaround is to disable autothrottling and rely on Azure
> > to throttle the connections as needed with the below configuration.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
