[
https://issues.apache.org/jira/browse/SOLR-14940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17220941#comment-17220941
]
Mike Drob edited comment on SOLR-14940 at 10/26/20, 7:11 PM:
-------------------------------------------------------------
It looks like we end up registering ReplicationHandler twice -
On core creation,
{noformat}
at
org.apache.solr.handler.ReplicationHandler.registerCloseHook(ReplicationHandler.java:1410)
[main/:?]
at
org.apache.solr.handler.ReplicationHandler.inform(ReplicationHandler.java:1235)
[main/:?]
at
org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:674)
[main/:?]
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1037) [main/:?]
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:929) [main/:?]
at
org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1375)
[main/:?]
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1261)
[main/:?]
{noformat}
And when we register in ZK
{noformat}
at
org.apache.solr.handler.ReplicationHandler.registerCloseHook(ReplicationHandler.java:1410)
[main/:?]
at
org.apache.solr.handler.ReplicationHandler.inform(ReplicationHandler.java:1235)
[main/:?]
at
org.apache.solr.cloud.ReplicateFromLeader.startReplication(ReplicateFromLeader.java:109)
[main/:?]
at
org.apache.solr.cloud.ZkController.startReplicationFromLeader(ZkController.java:1305)
[main/:?]
at org.apache.solr.cloud.ZkController.register(ZkController.java:1198)
[main/:?]
at org.apache.solr.cloud.ZkController.register(ZkController.java:1145)
[main/:?]
at
org.apache.solr.core.ZkContainer.lambda$registerInZk$1(ZkContainer.java:208)
[main/:?]
at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:237)
[main/:?]
at
org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:1195)
[main/:?]
at
org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1385)
[main/:?]
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1261)
[main/:?]
{noformat}
I think this is likely a bug.
was (Author: mdrob):
It looks like we end up registering ReplicationHandler twice -
On core creation,
{noformat}
at
org.apache.solr.handler.ReplicationHandler.registerCloseHook(ReplicationHandler.java:1410)
[main/:?]
at
org.apache.solr.handler.ReplicationHandler.inform(ReplicationHandler.java:1235)
[main/:?]
at
org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:674)
[main/:?]
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1037) [main/:?]
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:929) [main/:?]
at
org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1375)
[main/:?]
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1261)
[main/:?]
{noformat}
And when we register in ZK
{noformat}
at
org.apache.solr.handler.ReplicationHandler.registerCloseHook(ReplicationHandler.java:1410)
[main/:?]
at
org.apache.solr.handler.ReplicationHandler.inform(ReplicationHandler.java:1235)
[main/:?]
at
org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:674)
[main/:?]
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1037) [main/:?]
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:929) [main/:?]
at
org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1375)
[main/:?]
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1261)
[main/:?]
{noformat}
I think this is likely a bug.
> ReplicationHandler memory leak through SolrCore.closeHooks
> ----------------------------------------------------------
>
> Key: SOLR-14940
> URL: https://issues.apache.org/jira/browse/SOLR-14940
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: replication (java)
> Environment: Solr Cloud Cluster on v.8.6.2 configured as 3 TLOG nodes
> with 2 cores in each JVM.
>
> Reporter: Anver Sotnikov
> Priority: Major
> Attachments: Actual references to hooks that in turn hold references
> to ReplicationHandlers.png, Memory Analyzer SolrCore.closeHooks .png
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
> We are experiencing a memory leak in Solr Cloud cluster configured as 3 TLOG
> nodes.
> Leader does not seem to be affected while Followers are.
>
> Looking at memory dump we noticed that SolrCore holds lots of references to
> ReplicationHandler through anonymous inner classes in SolrCore.closeHooks,
> which in turn holds ReplicationHandlers.
> ReplicationHandler registers hooks as anonymous inner classes in
> SolrCore.closeHooks through ReplicationHandler.inform() ->
> ReplicationHandler.registerCloseHook().
>
> Whenever ZkController.stopReplicationFromLeader is called - it would shutdown
> ReplicationHandler (ReplicationHandler.shutdown()), BUT reference to
> ReplicationHandler will stay in SolrCore.closeHooks. Once replication is
> started again on same SolrCore - new ReplicationHandler will be created and
> registered in closeHooks.
>
> It looks like there are few scenarios when replication is stopped and
> restarted on same core and in our TLOG setup it shows up quite often.
>
> Potential solutions:
> # Allow unregistering SolrCore.closeHooks so it can be used from
> ReplicationHandler.shutdown
> # Hack but easier - break the link between ReplicationHandler close hooks
> and full ReplicationHandler object so ReplicationHandler can be GCed even
> when hooks are still registered in SolrCore.closeHooks
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
