Re: [PR] Performance improvements to use read lock to access LRUQueryCache [lucene]
boicehuang commented on code in PR #13306:
URL: https://github.com/apache/lucene/pull/13306#discussion_r1581809438
##
lucene/core/src/java/org/apache/lucene/search/LRUQueryCache.java:
##
@@ -265,7 +269,6 @@ boolean requiresEviction() {
}
CacheAndCount get(Query key, IndexReader.CacheHelper cacheHelper) {
-assert lock.isHeldByCurrentThread();
assert key instanceof BoostQuery == false;
assert key instanceof ConstantScoreQuery == false;
final IndexReader.CacheKey readerKey = cacheHelper.getKey();
Review Comment:
In my optimization, the writing to `cache` object was still protected via
the write lock, but reading `cache` object concurrently via making `cache`
object to be `ConcurrentHashMap`. This is the key point of my optimization. I
am also trying to read through all the concurrency paths with a picture.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
Re: [PR] Performance improvements to use read lock to access LRUQueryCache [lucene]
boicehuang commented on code in PR #13306:
URL: https://github.com/apache/lucene/pull/13306#discussion_r1581809438
##
lucene/core/src/java/org/apache/lucene/search/LRUQueryCache.java:
##
@@ -265,7 +269,6 @@ boolean requiresEviction() {
}
CacheAndCount get(Query key, IndexReader.CacheHelper cacheHelper) {
-assert lock.isHeldByCurrentThread();
assert key instanceof BoostQuery == false;
assert key instanceof ConstantScoreQuery == false;
final IndexReader.CacheKey readerKey = cacheHelper.getKey();
Review Comment:
In my optimization, the writing to `cache` object was still protected via
the write lock, but reading `cache` object can be a concurrent action via
making `cache` object to be `ConcurrentHashMap`. This is the key point of my
optimization. I
am also trying to read through all the concurrency paths with a picture.
##
lucene/core/src/java/org/apache/lucene/search/LRUQueryCache.java:
##
@@ -265,7 +269,6 @@ boolean requiresEviction() {
}
CacheAndCount get(Query key, IndexReader.CacheHelper cacheHelper) {
-assert lock.isHeldByCurrentThread();
assert key instanceof BoostQuery == false;
assert key instanceof ConstantScoreQuery == false;
final IndexReader.CacheKey readerKey = cacheHelper.getKey();
Review Comment:
In my optimization, the writing to `cache` object was still protected via
the write lock, but reading `cache` object can be a concurrent action via
making `cache` object to be `ConcurrentHashMap`. This is the key point of my
optimization. I am also trying to read through all the concurrency paths with a
picture.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
Re: [PR] Performance improvements to use read lock to access LRUQueryCache [lucene]
boicehuang commented on code in PR #13306:
URL: https://github.com/apache/lucene/pull/13306#discussion_r1581809438
##
lucene/core/src/java/org/apache/lucene/search/LRUQueryCache.java:
##
@@ -265,7 +269,6 @@ boolean requiresEviction() {
}
CacheAndCount get(Query key, IndexReader.CacheHelper cacheHelper) {
-assert lock.isHeldByCurrentThread();
assert key instanceof BoostQuery == false;
assert key instanceof ConstantScoreQuery == false;
final IndexReader.CacheKey readerKey = cacheHelper.getKey();
Review Comment:
In my optimization, the writing to `cache` object was still protected via
the write lock, but reading `cache` object can be a concurrent action via
making `cache` object to be `ConcurrentHashMap`. This is the key point of my
optimization. I am also trying to read through all the concurrency paths.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
Re: [PR] Performance improvements to use read lock to access LRUQueryCache [lucene]
boicehuang commented on code in PR #13306:
URL: https://github.com/apache/lucene/pull/13306#discussion_r1581809438
##
lucene/core/src/java/org/apache/lucene/search/LRUQueryCache.java:
##
@@ -265,7 +269,6 @@ boolean requiresEviction() {
}
CacheAndCount get(Query key, IndexReader.CacheHelper cacheHelper) {
-assert lock.isHeldByCurrentThread();
assert key instanceof BoostQuery == false;
assert key instanceof ConstantScoreQuery == false;
final IndexReader.CacheKey readerKey = cacheHelper.getKey();
Review Comment:
In my optimization, `uniqueQueries` is
`Collections.synchronizedMap(LinkedHashMap)` to achieve LRU expiration, the
writing to `cache` object was protected via the write lock, but reading
`cache` object can be a concurrent action via making `cache` object to be
`ConcurrentHashMap`. This is the key point of my optimization. I am also trying
to read through all the concurrency paths.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
Re: [PR] Performance improvements to use read lock to access LRUQueryCache [lucene]
boicehuang commented on code in PR #13306:
URL: https://github.com/apache/lucene/pull/13306#discussion_r1581809438
##
lucene/core/src/java/org/apache/lucene/search/LRUQueryCache.java:
##
@@ -265,7 +269,6 @@ boolean requiresEviction() {
}
CacheAndCount get(Query key, IndexReader.CacheHelper cacheHelper) {
-assert lock.isHeldByCurrentThread();
assert key instanceof BoostQuery == false;
assert key instanceof ConstantScoreQuery == false;
final IndexReader.CacheKey readerKey = cacheHelper.getKey();
Review Comment:
In my optimization, `uniqueQueries` is Collections.synchronizedMap to
achieve LRU expiration, the writing to `cache` object was protected via the
write lock, but reading `cache` object can be a concurrent action via making
`cache` object to be `ConcurrentHashMap`. This is the key point of my
optimization. I am also trying to read through all the concurrency paths.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
Re: [PR] Performance improvements to use read lock to access LRUQueryCache [lucene]
boicehuang commented on code in PR #13306:
URL: https://github.com/apache/lucene/pull/13306#discussion_r1581809438
##
lucene/core/src/java/org/apache/lucene/search/LRUQueryCache.java:
##
@@ -265,7 +269,6 @@ boolean requiresEviction() {
}
CacheAndCount get(Query key, IndexReader.CacheHelper cacheHelper) {
-assert lock.isHeldByCurrentThread();
assert key instanceof BoostQuery == false;
assert key instanceof ConstantScoreQuery == false;
final IndexReader.CacheKey readerKey = cacheHelper.getKey();
Review Comment:
In my optimization, `uniqueQueries` is
`Collections.synchronizedMap(LinkedHashMap)` to achieve LRU expiration, the
writing to `cache` object is protected via the write lock, but reading
`cache` object can be a concurrent action via making `cache` object to be
`ConcurrentHashMap`. This is the key point of my optimization. I am also trying
to read through all the concurrency paths.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
Re: [I] Decouple within-query concurrency from the index's segment geometry [LUCENE-8675] [lucene]
msokolov commented on issue #9721: URL: https://github.com/apache/lucene/issues/9721#issuecomment-2080592940 I guess we can use the new "parent field" to enforce that intervals must end on a non-child document, and we have to refuse to attempt query execution over sub-leaf interval if there are doc blocks and no parent field, at least if the query is block-sensitive. The documentation says we will be enforcing that in Lucene 10, but it doesn't seem to be the case on main today - do we have an issue open for making the parent field mandatory? At least I have a bunch of tests failing with an exception I added to test for this: > java.lang.IllegalStateException: slicing a segment with blocks and no parent field > Note, this is fully optional in Lucene 9.x while will be mandatory for indices that use document blocks together with index sorting as of 10.0.0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
Re: [I] MIGRATE.md doesn't mention the transition from Collector to CollectorManager [lucene]
mikemccand commented on issue #13218: URL: https://github.com/apache/lucene/issues/13218#issuecomment-2080640729 > is this for 10.0.0 This is for 10.0.0, yes, but is also helpful when upgrading to 9.x since these APIs are deprecated there (and removed in 10.0). @zacharymorn -- I think we can close this now? #13238 resolves this issue? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
Re: [PR] Backport to 9x: Initialize facet counting data structures lazily #12408 [lucene]
github-actions[bot] commented on PR #13300: URL: https://github.com/apache/lucene/pull/13300#issuecomment-2081261294 This PR has not had activity in the past 2 weeks, labeling it as stale. If the PR is waiting for review, notify the d...@lucene.apache.org list. Thank you for your contribution! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
