Re: [PR] Convert BKDConfig to a record [lucene]

2024-08-19 Thread via GitHub
iverase commented on code in PR #13668: URL: https://github.com/apache/lucene/pull/13668#discussion_r1722783860 ## lucene/core/src/java/org/apache/lucene/util/bkd/BKDConfig.java: ## @@ -31,48 +48,23 @@ public final class BKDConfig { /** Maximum number of index dimensions */

Re: [PR] nocommit: demonstrate how a minor change in IndexSearcher can have an inexplicable performance impact [lucene]

2024-08-19 Thread via GitHub
epotyom commented on PR #13657: URL: https://github.com/apache/lucene/pull/13657#issuecomment-2298104290 > @epotyom > > I've noticed by accident that creating two identical lists of collectors, before and after createWeight, makes regression disappear > > Hmm, that is strange. I w

Re: [PR] Leverage doc value skip lists in DocValuesRewriteMethod if indexed [lucene]

2024-08-19 Thread via GitHub
gsmiller commented on PR #13672: URL: https://github.com/apache/lucene/pull/13672#issuecomment-2297951310 It just occurred to me we can’t use DocValuesRangeIterator directly here as it could produce false positive matches. I’ll write a test to capture this and iterate on the PR. -- This

Re: [PR] Leverage doc value skip lists in DocValuesRewriteMethod if indexed [lucene]

2024-08-19 Thread via GitHub
rmuir commented on code in PR #13672: URL: https://github.com/apache/lucene/pull/13672#discussion_r1722557603 ## lucene/core/src/test/org/apache/lucene/search/TestDocValuesRewriteMethod.java: ## @@ -61,14 +61,19 @@ public void setUp() throws Exception { .setMaxB

Re: [PR] Leverage doc value skip lists in DocValuesRewriteMethod if indexed [lucene]

2024-08-19 Thread via GitHub
rmuir commented on code in PR #13672: URL: https://github.com/apache/lucene/pull/13672#discussion_r1722556851 ## lucene/core/src/java/org/apache/lucene/document/DocValuesRangeIterator.java: ## @@ -24,8 +24,10 @@ /** * Wrapper around a {@link TwoPhaseIterator} for a doc-values

[PR] Leverage doc value skip lists in DocValuesRewriteMethod if indexed [lucene]

2024-08-19 Thread via GitHub
gsmiller opened a new pull request, #13672: URL: https://github.com/apache/lucene/pull/13672 ### Description `DocValuesRewriteMethod` (used under-the-hood in `MultiTermQuery`) is another spot where we can leverage the new doc value skip lists when present to potentially skip over doc

Re: [PR] Compute facets while collecting [lucene]

2024-08-19 Thread via GitHub
gsmiller commented on PR #13568: URL: https://github.com/apache/lucene/pull/13568#issuecomment-2297766370 Thanks @javanna for raising this. I agree with trying to figure out how to shrink this API surface area in `IndexSearcher` back down prior to the 10.0 release. I don't have any immediat

[I] Find a way to remove IndexSearcher#search(Query query, CollectorOwner collectorOwner) before 10.0 [lucene]

2024-08-19 Thread via GitHub
gsmiller opened a new issue, #13671: URL: https://github.com/apache/lucene/issues/13671 ### Description GH#13568 introduced a new public method to `IndexSearcher` along with a new concept of a `CollectorOwner`. This was done to allow drill-sideways to use the new sandbox faceting fun

Re: [PR] Compute facets while collecting [lucene]

2024-08-19 Thread via GitHub
javanna commented on PR #13568: URL: https://github.com/apache/lucene/pull/13568#issuecomment-2297566037 Hey all, sorry about my late comment, I noticed the `IndexSearcher` changes, the additional search method that takes a `CollectorOwner` and the new `CollectorOwner` abstraction. It looks

Re: [PR] nocommit: demonstrate how a minor change in IndexSearcher can have an inexplicable performance impact [lucene]

2024-08-19 Thread via GitHub
gsmiller commented on PR #13657: URL: https://github.com/apache/lucene/pull/13657#issuecomment-2297489039 @epotyom -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsub

Re: [PR] Change the switch statement(#13666) [lucene]

2024-08-19 Thread via GitHub
gsmiller merged PR #13669: URL: https://github.com/apache/lucene/pull/13669 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.ap

Re: [PR] Change the switch statement(#13666) [lucene]

2024-08-19 Thread via GitHub
gsmiller commented on PR #13669: URL: https://github.com/apache/lucene/pull/13669#issuecomment-2297481110 Looks good, thanks! I'm OK with merging this, but I want to also take a moment to ask that we're somewhat cautious with racking up lots of small changes like this that cannot be back-po

Re: [PR] [WIP] Multi-Vector support for HNSW search [lucene]

2024-08-19 Thread via GitHub
vigyasharma commented on PR #13525: URL: https://github.com/apache/lucene/pull/13525#issuecomment-2297380564 > This PR has not had activity in the past 2 weeks, labeling it as stale... Just to update on some activity here: I'm working on parent block join benchmarks in `luceneutil`

Re: [PR] Speed up prefix sums when decoding doc IDs. [lucene]

2024-08-19 Thread via GitHub
gsmiller commented on PR #13658: URL: https://github.com/apache/lucene/pull/13658#issuecomment-2297082943 Good question @jpountz. If I'm looking at this properly, it seems to not be getting inlined in my benchmark run. (I grabbed your patch from #13670 and loaded one of the jfr files produc

Re: [PR] Convert BKDConfig to a record [lucene]

2024-08-19 Thread via GitHub
uschindler commented on code in PR #13668: URL: https://github.com/apache/lucene/pull/13668#discussion_r1722023350 ## lucene/core/src/java/org/apache/lucene/util/bkd/BKDConfig.java: ## @@ -31,48 +48,23 @@ public final class BKDConfig { /** Maximum number of index dimensions *

Re: [PR] Add Facets#getBulkSpecificValues method [lucene]

2024-08-19 Thread via GitHub
mikemccand commented on PR #12862: URL: https://github.com/apache/lucene/pull/12862#issuecomment-2296974933 Hmm it looks like this change was nearly ready to be merged? But code has since drifted and the PR now has some rust (conflicts)? -- This is an automated message from the Apache Gi

Re: [PR] Speed up prefix sums when decoding doc IDs. [lucene]

2024-08-19 Thread via GitHub
jpountz commented on PR #13658: URL: https://github.com/apache/lucene/pull/13658#issuecomment-2296948223 Thanks for looking @gsmiller ! Are the splitLongs calls also not inlined for you as well? I submitted a PR to make nightly reports tell us about which methods get inlined / compiled / in

Re: [PR] Convert BKDConfig to a record [lucene]

2024-08-19 Thread via GitHub
uschindler commented on code in PR #13668: URL: https://github.com/apache/lucene/pull/13668#discussion_r1722023350 ## lucene/core/src/java/org/apache/lucene/util/bkd/BKDConfig.java: ## @@ -31,48 +48,23 @@ public final class BKDConfig { /** Maximum number of index dimensions *

Re: [PR] nocommit: demonstrate how a minor change in IndexSearcher can have an inexplicable performance impact [lucene]

2024-08-19 Thread via GitHub
epotyom commented on PR #13657: URL: https://github.com/apache/lucene/pull/13657#issuecomment-2296904888 Just for tracking, repeating this comment https://github.com/apache/lucene/pull/13656#issuecomment-2296813725 here: > The biggest difference in the profiler seems to be that we spe

Re: [PR] Speed up prefix sums when decoding doc IDs. [lucene]

2024-08-19 Thread via GitHub
gsmiller commented on PR #13658: URL: https://github.com/apache/lucene/pull/13658#issuecomment-2296882889 I ran `wikimedium10m` and can (sort of) see the regression on AndHighMed (but only that task). Results below, but I see `2.4% with a 0.058 p-value. This is on an AWS m4.12xlarge x86-bas

Re: [I] TestHnswFloatVectorGraph.testRandomReadWriteAndMerge fails with java.lang.IndexOutOfBoundsException [lucene]

2024-08-19 Thread via GitHub
ChrisHegarty commented on issue #13659: URL: https://github.com/apache/lucene/issues/13659#issuecomment-2296727766 > I mean it would be nice if we didn't generate these degenerate Component in the first place? But this will work I had the same thought, but then backed away quickly! ;

Re: [PR] Convert BKDConfig to a record [lucene]

2024-08-19 Thread via GitHub
iverase commented on code in PR #13668: URL: https://github.com/apache/lucene/pull/13668#discussion_r1721868595 ## lucene/core/src/java/org/apache/lucene/util/bkd/BKDConfig.java: ## @@ -31,48 +48,23 @@ public final class BKDConfig { /** Maximum number of index dimensions */

[PR] Change the switch statement(#13666) [lucene]

2024-08-19 Thread via GitHub
mrhbj opened a new pull request, #13669: URL: https://github.com/apache/lucene/pull/13669 ### Description -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

Re: [PR] Make the code look simpler [lucene]

2024-08-19 Thread via GitHub
mrhbj closed pull request #13666: Make the code look simpler URL: https://github.com/apache/lucene/pull/13666 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-

Re: [PR] Revert changes to IndexSearcher brought in by GH#13568 [lucene]

2024-08-19 Thread via GitHub
mikemccand commented on PR #13656: URL: https://github.com/apache/lucene/pull/13656#issuecomment-2296609036 > Have you followed what @epotyom and I found so far in #13657? Yes! And it's completely insane... it might be time to turn on `-XX:+PrintAssembly` and watch how hotspot is spe

Re: [PR] Only attempt to connect components when entry point is valid [lucene]

2024-08-19 Thread via GitHub
ChrisHegarty commented on PR #13660: URL: https://github.com/apache/lucene/pull/13660#issuecomment-2296590719 Belated LGTM. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Convert BKDConfig to a record [lucene]

2024-08-19 Thread via GitHub
uschindler commented on code in PR #13668: URL: https://github.com/apache/lucene/pull/13668#discussion_r1721755168 ## lucene/core/src/java/org/apache/lucene/util/bkd/BKDConfig.java: ## @@ -31,48 +48,23 @@ public final class BKDConfig { /** Maximum number of index dimensions *

Re: [PR] Make the code look simpler [lucene]

2024-08-19 Thread via GitHub
mrhbj commented on PR #13666: URL: https://github.com/apache/lucene/pull/13666#issuecomment-2296511386 you are right,i will change my commit messave,thanks ---Original--- From: "Michael ***@***.***> Date: Mon, Aug 19, 2024 20:32 PM To: ***@***.***>; Cc: ***@***.

Re: [I] TestHnswFloatVectorGraph.testRandomReadWriteAndMerge fails with java.lang.IndexOutOfBoundsException [lucene]

2024-08-19 Thread via GitHub
benwtrent closed issue #13659: TestHnswFloatVectorGraph.testRandomReadWriteAndMerge fails with java.lang.IndexOutOfBoundsException URL: https://github.com/apache/lucene/issues/13659 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

Re: [PR] Only attempt to connect components when entry point is valid [lucene]

2024-08-19 Thread via GitHub
benwtrent merged PR #13660: URL: https://github.com/apache/lucene/pull/13660 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.a

Re: [PR] Make the code look simpler [lucene]

2024-08-19 Thread via GitHub
msokolov commented on PR #13666: URL: https://github.com/apache/lucene/pull/13666#issuecomment-2296463007 This looks good to me, but perhaps you could make the commit message more descriptive of the actual change. If I read "make the code look simpler" I have misgivings! Maybe mention the s

[I] TestTieredMergePolicy.testSimulateUpdates fails intermittently asserting segment infos [lucene]

2024-08-19 Thread via GitHub
ChrisHegarty opened a new issue, #13667: URL: https://github.com/apache/lucene/issues/13667 ### Description Fails intermittently on _main_, with the following stacktrace: ``` TestTieredMergePolicy > testSimulateUpdates FAILED java.lang.AssertionError: mergeFactor=2 min

Re: [PR] Add reopen method in PerThreadPKLookup [lucene]

2024-08-19 Thread via GitHub
vsop-479 commented on PR #13596: URL: https://github.com/apache/lucene/pull/13596#issuecomment-2296249587 @jpountz Please take a look when you get a chance. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[PR] Make the code look simpler [lucene]

2024-08-19 Thread via GitHub
mrhbj opened a new pull request, #13666: URL: https://github.com/apache/lucene/pull/13666 ### Description -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

Re: [PR] Make the code look simpler [lucene]

2024-08-19 Thread via GitHub
mrhbj closed pull request #13665: Make the code look simpler URL: https://github.com/apache/lucene/pull/13665 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-

[PR] Make the code look simpler [lucene]

2024-08-19 Thread via GitHub
mrhbj opened a new pull request, #13665: URL: https://github.com/apache/lucene/pull/13665 ### Description -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns