Re: [PR] Better vectorize score computations. [lucene]

2025-06-10 Thread via GitHub
jpountz commented on PR #14704: URL: https://github.com/apache/lucene/pull/14704#issuecomment-2960446053 @mikemccand FYI if a nightly run started with this change, it may fail due to this thread safety issue. The commit is now reverted, so restarting should be enough to fix the problem. -

Re: [PR] Better vectorize score computations. [lucene]

2025-06-10 Thread via GitHub
jpountz commented on PR #14704: URL: https://github.com/apache/lucene/pull/14704#issuecomment-2960390626 I had to revert this PR, it's not thread-safe: I thought `SimScorer`s were created per `Scorer`, but they are created per `Weight` and may be used across multiple threads. -- This is

Re: [PR] Better vectorize score computations. [lucene]

2025-06-10 Thread via GitHub
jpountz merged PR #14704: URL: https://github.com/apache/lucene/pull/14704 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apa

Re: [PR] Better vectorize score computations. [lucene]

2025-05-26 Thread via GitHub
jpountz commented on PR #14704: URL: https://github.com/apache/lucene/pull/14704#issuecomment-2909588910 I was looking at it too as I was worried it may have been caused by recent optimizations, but it looks like an old and rare failure. I opened a tentative test fix at https://github.com/a

Re: [PR] Better vectorize score computations. [lucene]

2025-05-23 Thread via GitHub
rmuir commented on PR #14704: URL: https://github.com/apache/lucene/pull/14704#issuecomment-2905412399 before adding more complexity to scores can we resolve the current problem with scoring in jenkins? https://jenkins.thetaphi.de/job/Lucene-MMAPv2-Linux/3326/ ``` java.lang.AssertionE

Re: [PR] Better vectorize score computations. [lucene]

2025-05-23 Thread via GitHub
jpountz commented on code in PR #14704: URL: https://github.com/apache/lucene/pull/14704#discussion_r2104638649 ## lucene/core/src/java/org/apache/lucene/search/similarities/BM25Similarity.java: ## @@ -229,7 +249,39 @@ public float score(float freq, long encodedNorm) { //

Re: [PR] Better vectorize score computations. [lucene]

2025-05-23 Thread via GitHub
rmuir commented on code in PR #14704: URL: https://github.com/apache/lucene/pull/14704#discussion_r2104654751 ## lucene/core/src/java/org/apache/lucene/search/similarities/BM25Similarity.java: ## @@ -217,6 +221,19 @@ private static class BM25Scorer extends SimScorer { thi

Re: [PR] Better vectorize score computations. [lucene]

2025-05-23 Thread via GitHub
jpountz commented on PR #14704: URL: https://github.com/apache/lucene/pull/14704#issuecomment-2904496600 Great feedback once again @gf2121 ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

Re: [PR] Better vectorize score computations. [lucene]

2025-05-23 Thread via GitHub
jpountz commented on PR #14704: URL: https://github.com/apache/lucene/pull/14704#issuecomment-2904498062 Here is what luceneutil has to say for exhaustive evaluation (totalHitsThreshold=Integer.MAX_VALUE): ``` TaskQPS baseline StdDevQPS my_modified

Re: [PR] Better vectorize score computations. [lucene]

2025-05-23 Thread via GitHub
gf2121 commented on code in PR #14704: URL: https://github.com/apache/lucene/pull/14704#discussion_r2104631515 ## lucene/core/src/java/org/apache/lucene/search/similarities/BM25Similarity.java: ## @@ -229,7 +249,39 @@ public float score(float freq, long encodedNorm) { //

Re: [PR] Better vectorize score computations. [lucene]

2025-05-23 Thread via GitHub
github-actions[bot] commented on PR #14704: URL: https://github.com/apache/lucene/pull/14704#issuecomment-2904443217 This PR does not have an entry in lucene/CHANGES.txt. Consider adding one. If the PR doesn't need a changelog entry, then add the skip-changelog-check label to it and you wil

[PR] Better vectorize score computations. [lucene]

2025-05-23 Thread via GitHub
jpountz opened a new pull request, #14704: URL: https://github.com/apache/lucene/pull/14704 Existing vectorization of scores is a bit fragile since it relies on `SimScorer#score` being inlined in the for loops where it is called. This is currently the case in nightly benchmarks, but may not