Re: [PR] Binary vector format for flat and hnsw vectors [lucene]

2025-02-25 Thread via GitHub
lpld commented on PR #14078: URL: https://github.com/apache/lucene/pull/14078#issuecomment-2681113395 @benwtrent Thanks for your reply! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] Do not enable security manager on JDK 24+ [lucene]

2025-02-25 Thread via GitHub
uschindler commented on PR #14179: URL: https://github.com/apache/lucene/pull/14179#issuecomment-2681224605 Works well with RC of Java 24: https://jenkins.thetaphi.de/job/Lucene-main-Linux/53317/consoleFull Also Java 25 EA is fine: https://jenkins.thetaphi.de/job/Lucene-main-Lin

Re: [PR] Do not enable security manager on JDK 24+ [lucene]

2025-02-25 Thread via GitHub
uschindler commented on PR #14179: URL: https://github.com/apache/lucene/pull/14179#issuecomment-2681228674 In case we may need another Lucene 9 release and want to enable Jenkins we may need to backport. -- This is an automated message from the Apache Git Service. To respond to the messa

Re: [PR] Adds github action to verify changelog entry and set milestone to PRs [lucene]

2025-02-25 Thread via GitHub
stefanvodita commented on code in PR #14279: URL: https://github.com/apache/lucene/pull/14279#discussion_r1969509673 ## .github/workflows/verify-changelog-and-set-milestone.yml: ## @@ -0,0 +1,100 @@ +name: "Change Log Entry Verifier and Milestone Setter" +run-name: Change log en

Re: [PR] Adds github action to verify changelog entry and set milestone to PRs [lucene]

2025-02-25 Thread via GitHub
stefanvodita commented on code in PR #14279: URL: https://github.com/apache/lucene/pull/14279#discussion_r1969511868 ## .github/workflows/verify-changelog-and-set-milestone.yml: ## @@ -0,0 +1,100 @@ +name: "Change Log Entry Verifier and Milestone Setter" +run-name: Change log en

Re: [PR] Adds github action to verify changelog entry and set milestone to PRs [lucene]

2025-02-25 Thread via GitHub
stefanvodita commented on PR #14279: URL: https://github.com/apache/lucene/pull/14279#issuecomment-2681537099 Thank you for making the changes @pseudo-nymous! I think once we make the action use the new label I created we're ready to test this out. -- This is an automated message from th

Re: [PR] Adds github action to verify changelog entry and set milestone to PRs [lucene]

2025-02-25 Thread via GitHub
stefanvodita commented on code in PR #14279: URL: https://github.com/apache/lucene/pull/14279#discussion_r1969506749 ## .github/workflows/verify-changelog-and-set-milestone.yml: ## @@ -0,0 +1,100 @@ +name: "Change Log Entry Verifier and Milestone Setter" +run-name: Change log en

Re: [PR] Adds github action to verify changelog entry and set milestone to PRs [lucene]

2025-02-25 Thread via GitHub
pseudo-nymous commented on PR #14279: URL: https://github.com/apache/lucene/pull/14279#issuecomment-2681569542 I have updated the script to use new label. We are ready to test the changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] Adds github action to verify changelog entry and set milestone to PRs [lucene]

2025-02-25 Thread via GitHub
pseudo-nymous commented on code in PR #14279: URL: https://github.com/apache/lucene/pull/14279#discussion_r1969536626 ## .github/workflows/verify-changelog-and-set-milestone.yml: ## @@ -0,0 +1,100 @@ +name: "Change Log Entry Verifier and Milestone Setter" +run-name: Change log e

[PR] Remove bits method from DocIdSet [lucene]

2025-02-25 Thread via GitHub
javanna opened a new pull request, #14290: URL: https://github.com/apache/lucene/pull/14290 The bits method is now redundant in the DocIdSet base class. This commit removes it, while leaving a single leftover usage in BitDocIdSet as a new method that is specific to that implementation. --

Re: [PR] Utility classes to make it easier to use sandbox facet API for most common cases [lucene]

2025-02-25 Thread via GitHub
stefanvodita commented on PR #14237: URL: https://github.com/apache/lucene/pull/14237#issuecomment-2681470186 @Shradha26, if you don't have more feedback, I'll merge this soon. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

Re: [I] Refactor QueryCache to improve concurrency and performance [lucene]

2025-02-25 Thread via GitHub
stefanvodita commented on issue #14222: URL: https://github.com/apache/lucene/issues/14222#issuecomment-2681480284 > Do you have any recommendations, such as any existing benchmarks I could use for this use case? Have you looked at [luceneutil](https://github.com/mikemccand/luceneutil

[PR] Deprecated DocIdSet#all [lucene]

2025-02-25 Thread via GitHub
javanna opened a new pull request, #14289: URL: https://github.com/apache/lucene/pull/14289 DocIdSet#all is no longer needed since the merge of Query and Filter. It is unused. This commits deprecates it. It will be removed in the next major version. -- This is an automated message fr

[PR] Fix optimization to help inline calls to live docs. [lucene]

2025-02-25 Thread via GitHub
jpountz opened a new pull request, #14294: URL: https://github.com/apache/lucene/pull/14294 While doing benchmarks on indexes with deletions, I found a bug in `ScorerUtil`, which optimizes live docs for the wrong class: `FixedBitSet` instead of `FixedBit`. Another performance bug is that `F

Re: [PR] Fix optimization to help inline calls to live docs. [lucene]

2025-02-25 Thread via GitHub
gf2121 commented on code in PR #14294: URL: https://github.com/apache/lucene/pull/14294#discussion_r1971084748 ## lucene/core/src/java/org/apache/lucene/search/ScorerUtil.java: ## @@ -113,22 +116,21 @@ static Scorable likelyTermScorer(Scorable scorable) { /** * Optimize

[I] Improve documentation for org.apache.lucene.search Sort class [lucene]

2025-02-25 Thread via GitHub
massimobrivio opened a new issue, #14295: URL: https://github.com/apache/lucene/issues/14295 ### Description **Problem** The documentation for the RELEVANCE sort criteria states: *Represents sorting by computed relevance. Using this sort criteria returns the same results as ca

Re: [PR] Use DenseConjunctionBulkScorer for single queries sometimes. [lucene]

2025-02-25 Thread via GitHub
gf2121 commented on code in PR #14293: URL: https://github.com/apache/lucene/pull/14293#discussion_r1971058705 ## lucene/core/src/java/org/apache/lucene/search/PointRangeQuery.java: ## @@ -341,11 +341,10 @@ public ScorerSupplier scorerSupplier(LeafReaderContext context) throws

Re: [PR] Use DenseConjunctionBulkScorer for single queries sometimes. [lucene]

2025-02-25 Thread via GitHub
gf2121 commented on code in PR #14293: URL: https://github.com/apache/lucene/pull/14293#discussion_r1971065012 ## lucene/core/src/java/org/apache/lucene/document/SortedSetDocValuesRangeQuery.java: ## @@ -158,16 +157,15 @@ public Scorer get(long leadCost) throws IOException {

Re: [PR] Use DenseConjunctionBulkScorer for single queries sometimes. [lucene]

2025-02-25 Thread via GitHub
gf2121 commented on code in PR #14293: URL: https://github.com/apache/lucene/pull/14293#discussion_r1971065012 ## lucene/core/src/java/org/apache/lucene/document/SortedSetDocValuesRangeQuery.java: ## @@ -158,16 +157,15 @@ public Scorer get(long leadCost) throws IOException {

Re: [PR] Use DenseConjunctionBulkScorer for single queries sometimes. [lucene]

2025-02-25 Thread via GitHub
jpountz commented on PR #14293: URL: https://github.com/apache/lucene/pull/14293#issuecomment-2683464574 Same dataset but with 5% docs deleted this time and #14294 applied on both the baseline and the candidate: ``` TaskQPS baseline StdDevQPS my_mo

[PR] Use DenseConjunctionBulkScorer for single queries sometimes. [lucene]

2025-02-25 Thread via GitHub
jpountz opened a new pull request, #14293: URL: https://github.com/apache/lucene/pull/14293 Even though there is a single clause, it often needs to be intersected, either with live docs or with the collector's competitive iterator. This uses `DenseConjunctionBulkScorer` for: - `Ma

Re: [PR] Add a HNSW collector that exits early when nearest neighbor queue saturates [lucene]

2025-02-25 Thread via GitHub
tteofili commented on PR #14094: URL: https://github.com/apache/lucene/pull/14094#issuecomment-2681635026 updated `lucene_util` benchmarks, with different parameters. ### maxconn=32 baseline ``` recall latency (ms)nDoc topK fanout maxConn beamWidth quantized vi

[PR] Remove IOException from ScorerSupplier#setTopLevelScoringClause signature [lucene]

2025-02-25 Thread via GitHub
javanna opened a new pull request, #14291: URL: https://github.com/apache/lucene/pull/14291 There should be no reason to throw IOException in any implementation of setTopLevelScoringClause . This commit removes IOException from its signature. -- This is an automated message from the A

Re: [PR] Remove bits method from DocIdSet [lucene]

2025-02-25 Thread via GitHub
jpountz commented on PR #14290: URL: https://github.com/apache/lucene/pull/14290#issuecomment-2683084817 > leaving a single leftover usage in BitDocIdSet Is it because of `QueryBitSetProducer`? If so it looks like this class could be refactored to not use `BitDocIdSet`. All it needs i

Re: [PR] improve checkJavadocLinks.py to detect "invalid reference" [lucene]

2025-02-25 Thread via GitHub
rmuir merged PR #14287: URL: https://github.com/apache/lucene/pull/14287 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apach

Re: [PR] OptimisticKnnVectorQuery [lucene]

2025-02-25 Thread via GitHub
msokolov commented on PR #14226: URL: https://github.com/apache/lucene/pull/14226#issuecomment-2682565353 oh darn, in all my rebasing and force-pushing I seem to have ended up with the wrong version, grr. I will check reflog and recover... -- This is an automated message from the Apache G

Re: [PR] Remove bits method from DocIdSet [lucene]

2025-02-25 Thread via GitHub
javanna commented on PR #14290: URL: https://github.com/apache/lucene/pull/14290#issuecomment-2683098818 Yes, the only usage is in QueryBitSetProducer, that was already previously casting anyways. I will try to apply your suggesting as a follow-up once I merged this change. Also, that one m

[PR] Deprecate the redundant DocIdSet#bits method [lucene]

2025-02-25 Thread via GitHub
javanna opened a new pull request, #14292: URL: https://github.com/apache/lucene/pull/14292 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m

Re: [PR] Utility classes to make it easier to use sandbox facet API for most common cases [lucene]

2025-02-25 Thread via GitHub
epotyom commented on PR #14237: URL: https://github.com/apache/lucene/pull/14237#issuecomment-2683168623 @stefanvodita , I've pushed a commit https://github.com/apache/lucene/pull/14237/commits/83ddb8d48ef2cb050ae2bc8baaff684c1d28fb8c based on @Shradha26 comments + offline discussion, also

Re: [PR] Use DenseConjunctionBulkScorer for single queries sometimes. [lucene]

2025-02-25 Thread via GitHub
jpountz commented on PR #14293: URL: https://github.com/apache/lucene/pull/14293#issuecomment-2683288839 luceneutil on wikibigall: ``` TaskQPS baseline StdDevQPS my_modified_version StdDevPct diff p-value T

Re: [PR] Remove unused DocIdSet#all method [lucene]

2025-02-25 Thread via GitHub
javanna merged PR #14288: URL: https://github.com/apache/lucene/pull/14288 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apa

Re: [PR] Deprecated DocIdSet#all [lucene]

2025-02-25 Thread via GitHub
javanna merged PR #14289: URL: https://github.com/apache/lucene/pull/14289 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apa

Re: [I] Stop duplicating per-segment work across segment partitions [lucene]

2025-02-25 Thread via GitHub
javanna commented on issue #13745: URL: https://github.com/apache/lucene/issues/13745#issuecomment-2682478759 This is a heads up that I started working on this. My focus is currently on `PointRangeQuery`. The overall goal is to share the bitset computation across scorer suppliers for the sa

Re: [PR] Add a HNSW collector that exits early when nearest neighbor queue saturates [lucene]

2025-02-25 Thread via GitHub
tteofili commented on PR #14094: URL: https://github.com/apache/lucene/pull/14094#issuecomment-2681626815 I've updated this and moved the early termination logic not to kick in by default but to be based on a (wrapping) `PatienceKnnVectorQuery`. -- This is an automated message from the Ap

Re: [PR] Remove IOException from DocIdSet#iterator signature [lucene]

2025-02-25 Thread via GitHub
javanna merged PR #14284: URL: https://github.com/apache/lucene/pull/14284 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apa