jaisonbi commented on PR #816:
URL: https://github.com/apache/lucene/pull/816#issuecomment-1109236031
This issue has no relationship with "creating and destroying too many
threads", but it's caused by **too many "ThreadLocal" objects created from same
threads**.
Based on the implementation of ThreadLocal:
1. All the "ThreadLocal" objects created from same thread will be stored in
one ThreadLocalMap.
2. Each thread has it's own ThreadLocalMap.
The investigation on the composition of the "ThreadLocalMap":
"**management**" thread: Most entries(Each ThreadLocal object is wrapped
into an Entry) are "CompressingStoredFieldsReader"
```
final CloseableThreadLocal<StoredFieldsReader> fieldsReaderLocal = new
CloseableThreadLocal<StoredFieldsReader>() {
@Override
protected StoredFieldsReader initialValue() {
return fieldsReaderOrig.clone();
}
};
```
"**write**" thread: Most entries are "PerThreadIDVersionAndSeqNoLookup"
```
static final ConcurrentMap<IndexReader.CacheKey,
CloseableThreadLocal<PerThreadIDVersionAndSeqNoLookup[]>> lookupStates =
ConcurrentCollections.newConcurrentMapWithAggressiveConcurrency();
```
Either "CompressingStoredFieldsReader" or "PerThreadIDVersionAndSeqNoLookup"
is created by "CloseableThreadLocal".
ReentrantReadWriteLock will create it's own ThreadLocal object. So the
ReentrantReadWrite lock aquired from "write" thread, it will share the same
ThreadLocalMap with "PerThreadIDVersionAndSeqNoLookup". There're too many
entries stored into ThreadLocalMap, this is why "ThreadLocal#remove" running
with high CPU usage.
So I think the heavy usage of "ThreadLocal" from Lucene's current machanism
is one major reason of this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
