[
https://issues.apache.org/jira/browse/PIG-3797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13923054#comment-13923054
]
Cheolsoo Park commented on PIG-3797:
------------------------------------
Why not use
[ThreadLocal.remove()|http://docs.oracle.com/javase/7/docs/api/java/lang/ThreadLocal.html#remove()]
to reinitialize the ThreadLocal variables?
{code}
+ // Reset static variables cleared for avoiding OOM.
+ // TODO: Figure out a cleaner way to do this. ThreadLocals actually
can be avoided all together
+ // for mapreduce/tez mode and just used for Local mode.
+ PhysicalOperator.reporter = new ThreadLocal<PigProgressable>();
+ PigMapReduce.sJobConfInternal = new ThreadLocal<Configuration>();
{code}
{code}
+ // Avoid memory leak. ThreadLocals especially leak a lot of memory.
+ PhysicalOperator.reporter = new ThreadLocal<PigProgressable>();
+ PigMapReduce.sJobConfInternal = new ThreadLocal<Configuration>();
{code}
Otherwise, looks good to me.
> Fix some memory leaks affecting container reuse
> -----------------------------------------------
>
> Key: PIG-3797
> URL: https://issues.apache.org/jira/browse/PIG-3797
> Project: Pig
> Issue Type: Sub-task
> Components: tez
> Reporter: Rohini Palaniswamy
> Assignee: Rohini Palaniswamy
> Fix For: tez-branch
>
> Attachments: PIG-3797-1.patch
>
>
> PigCombiner.sJobContext and PhysicalOperator.reporter hold references to the
> WrappedReducer$Context which in turn holds TezOutputContextImpl which holds
> references to the buffers in DefaultSorter. This was causing OOM after the
> container was reused 2 or 3 times. Debugged this with L17.pig in pigmix.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
