Re: [PR] Add Bulk Scorer For ToParentBlockJoinQuery [lucene]

2024-08-28 Thread via GitHub
Mikep86 commented on code in PR #13697: URL: https://github.com/apache/lucene/pull/13697#discussion_r1735379356 ## lucene/join/src/java/org/apache/lucene/search/join/ToParentBlockJoinQuery.java: ## @@ -440,6 +500,83 @@ private String formatScoreExplanation(int matches, int star

Re: [PR] Add Bulk Scorer For ToParentBlockJoinQuery [lucene]

2024-08-28 Thread via GitHub
Mikep86 commented on code in PR #13697: URL: https://github.com/apache/lucene/pull/13697#discussion_r1735376518 ## lucene/join/src/test/org/apache/lucene/search/join/TestBlockJoinBulkScorer.java: ## Review Comment: I plan on adding another test to check that the min competi

Re: [PR] Add Bulk Scorer For ToParentBlockJoinQuery [lucene]

2024-08-28 Thread via GitHub
Mikep86 commented on code in PR #13697: URL: https://github.com/apache/lucene/pull/13697#discussion_r1735374017 ## lucene/join/src/test/org/apache/lucene/search/join/TestBlockJoinBulkScorer.java: ## @@ -0,0 +1,264 @@ +package org.apache.lucene.search.join; + +import com.carrotse

Re: [PR] Add Bulk Scorer For ToParentBlockJoinQuery [lucene]

2024-08-28 Thread via GitHub
Mikep86 commented on code in PR #13697: URL: https://github.com/apache/lucene/pull/13697#discussion_r1735373453 ## lucene/join/src/test/org/apache/lucene/search/join/TestBlockJoinBulkScorer.java: ## @@ -0,0 +1,264 @@ +package org.apache.lucene.search.join; + +import com.carrotse

Re: [PR] Add Bulk Scorer For ToParentBlockJoinQuery [lucene]

2024-08-28 Thread via GitHub
Mikep86 commented on code in PR #13697: URL: https://github.com/apache/lucene/pull/13697#discussion_r1735372582 ## lucene/join/src/test/org/apache/lucene/search/join/TestBlockJoinBulkScorer.java: ## @@ -0,0 +1,264 @@ +package org.apache.lucene.search.join; + +import com.carrotse

[PR] Add Bulk Scorer For ToParentBlockJoinQuery [lucene]

2024-08-28 Thread via GitHub
Mikep86 opened a new pull request, #13697: URL: https://github.com/apache/lucene/pull/13697 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m

Re: [I] Gradle build sometimes gives spurious "unreferenced license file" warnings [lucene]

2024-08-28 Thread via GitHub
uschindler commented on issue #13695: URL: https://github.com/apache/lucene/issues/13695#issuecomment-2316262279 > > after having a shower in my German bathroom > > I'm sure all of us can't stop picturing this German bathroom, Uwe! :) https://x.com/UweSays/status/915396168506449

Re: [I] Gradle build sometimes gives spurious "unreferenced license file" warnings [lucene]

2024-08-28 Thread via GitHub
dweiss commented on issue #13695: URL: https://github.com/apache/lucene/issues/13695#issuecomment-2316189280 This is a regression from adding incremental builds. These tasks have side effects and should always execute. -- This is an automated message from the Apache Git Service. To respo

Re: [I] Gradle build sometimes gives spurious "unreferenced license file" warnings [lucene]

2024-08-28 Thread via GitHub
dweiss commented on issue #13695: URL: https://github.com/apache/lucene/issues/13695#issuecomment-2316191913 > after having a shower in my German bathroom I'm sure all of us can't stop picturing this German bathroom, Uwe! :) -- This is an automated message from the Apache Git Servic

Re: [PR] Fix Gradle build sometimes gives spurious "unreferenced license file" warnings [lucene]

2024-08-28 Thread via GitHub
uschindler merged PR #13696: URL: https://github.com/apache/lucene/pull/13696 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.

Re: [I] Gradle build sometimes gives spurious "unreferenced license file" warnings [lucene]

2024-08-28 Thread via GitHub
uschindler closed issue #13695: Gradle build sometimes gives spurious "unreferenced license file" warnings URL: https://github.com/apache/lucene/issues/13695 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

Re: [I] Gradle build sometimes gives spurious "unreferenced license file" warnings [lucene]

2024-08-28 Thread via GitHub
uschindler commented on issue #13695: URL: https://github.com/apache/lucene/issues/13695#issuecomment-2315998879 @mikemccand: Beer is ready and after having a shower in my German bathroom the hot day can end! -- This is an automated message from the Apache Git Service. To respond to the m

Re: [I] Gradle build sometimes gives spurious "unreferenced license file" warnings [lucene]

2024-08-28 Thread via GitHub
uschindler commented on issue #13695: URL: https://github.com/apache/lucene/issues/13695#issuecomment-2315983388 Here is the PR: #13696 I added a comment to the top of this file and a TODO to explain that all tasks in this file must always run and therefore cannot define any outputs.

[PR] Fix Gradle build sometimes gives spurious "unreferenced license file" warnings [lucene]

2024-08-28 Thread via GitHub
uschindler opened a new pull request, #13696: URL: https://github.com/apache/lucene/pull/13696 Revert changes by #12150 in jar-checks.gradle, because tasks in this file shares internal state between tasks without using files. Because of this all tasks here must always execute together, so t

Re: [I] Gradle build sometimes gives spurious "unreferenced license file" warnings [lucene]

2024-08-28 Thread via GitHub
uschindler commented on issue #13695: URL: https://github.com/apache/lucene/issues/13695#issuecomment-2315959316 It's fast. I see no delay. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [I] Gradle build sometimes gives spurious "unreferenced license file" warnings [lucene]

2024-08-28 Thread via GitHub
mikemccand commented on issue #13695: URL: https://github.com/apache/lucene/issues/13695#issuecomment-2315958533 +1 to revert that part of the optimization (no incremental building of license checking) for now? How slow is the full check? -- This is an automated message from the Ap

Re: [PR] Gradle optimizations [lucene]

2024-08-28 Thread via GitHub
uschindler commented on PR #12150: URL: https://github.com/apache/lucene/pull/12150#issuecomment-2315957699 Unfortunately the changes applied here for the file `jar-checks.gradle` (the new build outputs) cause random build warnings, see https://github.com/apache/lucene/issues/13695.

Re: [I] Gradle build sometimes gives spurious "unreferenced license file" warnings [lucene]

2024-08-28 Thread via GitHub
mikemccand commented on issue #13695: URL: https://github.com/apache/lucene/issues/13695#issuecomment-2315956877 Wow, thank you for digging so deep @uschindler! Maybe your fridge has beers again! -- This is an automated message from the Apache Git Service. To respond to the message, plea

Re: [PR] Introduce new encoding of BPV 21 for DocIdsWriter used in BKD Tree [lucene]

2024-08-28 Thread via GitHub
mikemccand commented on PR #13521: URL: https://github.com/apache/lucene/pull/13521#issuecomment-2315947436 > @mikemccand -- did you mean to close this PR? Ugh, no I did not! Sorry, I'll reopen!! -- This is an automated message from the Apache Git Service. To respond to the message

[PR] Introduce new encoding of BPV 21 for DocIdsWriter used in BKD Tree [lucene]

2024-08-28 Thread via GitHub
expani opened a new pull request, #13521: URL: https://github.com/apache/lucene/pull/13521 ### Background Lucene uses 3 different ways of storing the docIds in KDD file of a BKD Tree based index if the docIds in a leaf block are not sorted : - If the difference b/w min and max

Re: [PR] Introduce new encoding of BPV 21 for DocIdsWriter used in BKD Tree [lucene]

2024-08-28 Thread via GitHub
msfroh commented on PR #13521: URL: https://github.com/apache/lucene/pull/13521#issuecomment-2315946796 @mikemccand -- did you mean to close this PR? If @expani adds the unrolled version (`Bit21With3StepsEncoder`) to `DocIdsWriter`, the change should be ready to go, IMO. Even o

Re: [I] Gradle build sometimes gives spurious "unreferenced license file" warnings [lucene]

2024-08-28 Thread via GitHub
uschindler commented on issue #13695: URL: https://github.com/apache/lucene/issues/13695#issuecomment-2315945766 If I comment out those 2 lines, it works: https://github.com/apache/lucene/blob/d55b92bae01be46656c4d373d8cb1d081b280469/gradle/validation/jar-checks.gradle#L201-L202 So

Re: [PR] Introduce new encoding of BPV 21 for DocIdsWriter used in BKD Tree [lucene]

2024-08-28 Thread via GitHub
mikemccand commented on PR #13521: URL: https://github.com/apache/lucene/pull/13521#issuecomment-2315942081 > We can replace it with Bit21With2StepsEncoder in future when the performance is comparable to x86. I wonder what mechanism we could use to remind ourselves when performance of

Re: [PR] Introduce new encoding of BPV 21 for DocIdsWriter used in BKD Tree [lucene]

2024-08-28 Thread via GitHub
mikemccand commented on PR #13521: URL: https://github.com/apache/lucene/pull/13521#issuecomment-2315935834 > @jpountz IMO We should use `Bit21With3StepsEncoder` in DocIdsWriter as using `Bit21With2StepsEncoder` might lead to performance regression for workloads in aarch64 platforms.

Re: [I] Gradle build sometimes gives spurious "unreferenced license file" warnings [lucene]

2024-08-28 Thread via GitHub
uschindler commented on issue #13695: URL: https://github.com/apache/lucene/issues/13695#issuecomment-2315935860 OK, I found the bug. The comment explains it: https://github.com/apache/lucene/blob/d55b92bae01be46656c4d373d8cb1d081b280469/gradle/validation/jar-checks.gradle#L138-L140

Re: [PR] Introduce new encoding of BPV 21 for DocIdsWriter used in BKD Tree [lucene]

2024-08-28 Thread via GitHub
mikemccand closed pull request #13521: Introduce new encoding of BPV 21 for DocIdsWriter used in BKD Tree URL: https://github.com/apache/lucene/pull/13521 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [PR] Introduce new encoding of BPV 21 for DocIdsWriter used in BKD Tree [lucene]

2024-08-28 Thread via GitHub
msfroh commented on PR #13521: URL: https://github.com/apache/lucene/pull/13521#issuecomment-2315907724 Okay -- I was able to speed up the SIMD implementation a fair bit. Honestly, my main stupid mistake was that I hadn't declared `LONG_SPECIES` as `static final`, which probably prevented s

Re: [I] Gradle build sometimes gives spurious "unreferenced license file" warnings [lucene]

2024-08-28 Thread via GitHub
uschindler commented on issue #13695: URL: https://github.com/apache/lucene/issues/13695#issuecomment-2315903858 OK, I debugged a bit: ``` >gradlew :lucene:checkDanglingLicenseFiles Starting a Gradle Daemon (subsequent builds will be faster) > Task :errorProneSkipped WA

Re: [I] Gradle build sometimes gives spurious "unreferenced license file" warnings [lucene]

2024-08-28 Thread via GitHub
risdenk commented on issue #13695: URL: https://github.com/apache/lucene/issues/13695#issuecomment-2315812110 Solr has the same issue and it was reported here https://issues.apache.org/jira/browse/SOLR-17142 - there was a mailing list post where I found the culprit but I haven't had a chanc

Re: [PR] Speed up advancing within a block. [lucene]

2024-08-28 Thread via GitHub
jpountz merged PR #13692: URL: https://github.com/apache/lucene/pull/13692 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apa

Re: [I] Gradle build sometimes gives spurious "unreferenced license file" warnings [lucene]

2024-08-28 Thread via GitHub
uschindler commented on issue #13695: URL: https://github.com/apache/lucene/issues/13695#issuecomment-2315672592 P.S.: I did not open an issue because the last time I was confronted with it (about 1 year ago), I was at the mentag stage of "damn fcking gradle - I need more beers", but my fri

Re: [I] Gradle build sometimes gives spurious "unreferenced license file" warnings [lucene]

2024-08-28 Thread via GitHub
uschindler commented on issue #13695: URL: https://github.com/apache/lucene/issues/13695#issuecomment-2315667317 Maybe it can be solved by making the "ext" property a build output and let all tasks depend on it. -- This is an automated message from the Apache Git Service. To respond to th

Re: [I] Gradle build sometimes gives spurious "unreferenced license file" warnings [lucene]

2024-08-28 Thread via GitHub
uschindler commented on issue #13695: URL: https://github.com/apache/lucene/issues/13695#issuecomment-2315658433 Let's ask @dweiss, I gave up on this, as my Gradle-Foo is too limited, -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

Re: [I] Gradle build sometimes gives spurious "unreferenced license file" warnings [lucene]

2024-08-28 Thread via GitHub
uschindler commented on issue #13695: URL: https://github.com/apache/lucene/issues/13695#issuecomment-2315656656 I think to fix this, the `collectJarInfos` must write that information to a monitored file in the build outputs. All pother tasks using that information must read the outpu

Re: [I] Gradle build sometimes gives spurious "unreferenced license file" warnings [lucene]

2024-08-28 Thread via GitHub
uschindler commented on issue #13695: URL: https://github.com/apache/lucene/issues/13695#issuecomment-2315647367 > Just curious - what is the purpose of the check? I guess it sounds self-explanatory: if there is a license file thatisn't referenced (where?) we would presumably want to clean

Re: [I] Gradle build sometimes gives spurious "unreferenced license file" warnings [lucene]

2024-08-28 Thread via GitHub
msokolov commented on issue #13695: URL: https://github.com/apache/lucene/issues/13695#issuecomment-2315623970 Just curious - what is the purpose of the check? I guess it sounds self-explanatory: if there is a license file thatisn't referenced (where?) we would presumably want to clean it u

Re: [I] Gradle build sometimes gives spurious "unreferenced license file" warnings [lucene]

2024-08-28 Thread via GitHub
uschindler commented on issue #13695: URL: https://github.com/apache/lucene/issues/13695#issuecomment-2315617744 I alos noticed this several times. I also reported an issue to Dawid, but the cause for this is very strage. It depends on the order of execution. I used to know the cause

[I] Gradle build sometimes gives spurious "unreferenced license file" warnings [lucene]

2024-08-28 Thread via GitHub
mikemccand opened a new issue, #13695: URL: https://github.com/apache/lucene/issues/13695 ### Description On `main` today, with Java 21 on a recent Arch Linux, if I run `./gradlew clean` then `./gradlew check`, all is good. It passes. But if I run `./gradlew check` again, I so

Re: [PR] Add a few toString methods for HNSW scoring classes [lucene]

2024-08-28 Thread via GitHub
mikemccand merged PR #13694: URL: https://github.com/apache/lucene/pull/13694 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.

Re: [PR] Add a few toString methods for HNSW scoring classes [lucene]

2024-08-28 Thread via GitHub
mikemccand commented on code in PR #13694: URL: https://github.com/apache/lucene/pull/13694#discussion_r1734770769 ## build-tools/build-infra/src/main/java/org/apache/lucene/gradle/ProfileResults.java: ## @@ -156,7 +156,12 @@ private static String pad(String input) { /** Pr

Re: [I] Find a way to remove IndexSearcher#search(Query query, CollectorOwner collectorOwner) before 10.0 [lucene]

2024-08-28 Thread via GitHub
gsmiller commented on issue #13671: URL: https://github.com/apache/lucene/issues/13671#issuecomment-2315408209 @epotyom right makes sense. I should have been a little more specific. When I said "new method" I was thinking along the same lines of changing the recently-added method to not nee

Re: [PR] Speed up advancing within a block. [lucene]

2024-08-28 Thread via GitHub
gsmiller commented on code in PR #13692: URL: https://github.com/apache/lucene/pull/13692#discussion_r1734731228 ## lucene/core/src/java/org/apache/lucene/codecs/lucene912/Lucene912PostingsReader.java: ## @@ -212,13 +213,74 @@ static void prefixSum(long[] buffer, int count, long

Re: [PR] Speed up advancing within a block. [lucene]

2024-08-28 Thread via GitHub
jpountz commented on code in PR #13692: URL: https://github.com/apache/lucene/pull/13692#discussion_r1734653387 ## lucene/core/src/java/org/apache/lucene/codecs/lucene912/Lucene912PostingsReader.java: ## @@ -212,13 +213,74 @@ static void prefixSum(long[] buffer, int count, long

Re: [PR] Add a few toString methods for HNSW scoring classes [lucene]

2024-08-28 Thread via GitHub
msokolov commented on code in PR #13694: URL: https://github.com/apache/lucene/pull/13694#discussion_r1734623994 ## build-tools/build-infra/src/main/java/org/apache/lucene/gradle/ProfileResults.java: ## @@ -156,7 +156,12 @@ private static String pad(String input) { /** Proc

[PR] Add a few toString methods for HNSW scoring classes [lucene]

2024-08-28 Thread via GitHub
mikemccand opened a new pull request, #13694: URL: https://github.com/apache/lucene/pull/13694 I was trying to debug whether the distance metric I had requested using `luceneutil`'s `knnPerfTest.py` was properly working its way all the way down that stack, and found a few `toString` methods

Re: [I] Find a way to remove IndexSearcher#search(Query query, CollectorOwner collectorOwner) before 10.0 [lucene]

2024-08-28 Thread via GitHub
mikemccand commented on issue #13671: URL: https://github.com/apache/lucene/issues/13671#issuecomment-2315164006 I added `blocker` label to this and removed `type:enhancement` (it seemed super weird to have an enhancement blocker!). -- This is an automated message from the Apache Git Serv

Re: [I] Significant drop in recall for 8 bit Scalar Quantizer [lucene]

2024-08-28 Thread via GitHub
mikemccand commented on issue #13519: URL: https://github.com/apache/lucene/issues/13519#issuecomment-2315154624 I just tested KNN recall using [`knnPerfTest.py` from `luceneutil`](https://github.com/mikemccand/luceneutil/blob/main/src/python/knnPerfTest.py) on 4, 7, 8 bit quantization, and

Re: [PR] Remove mention of SolrNamedThreadFactory [lucene]

2024-08-28 Thread via GitHub
stefanvodita merged PR #13690: URL: https://github.com/apache/lucene/pull/13690 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucen

Re: [I] Find a way to remove IndexSearcher#search(Query query, CollectorOwner collectorOwner) before 10.0 [lucene]

2024-08-28 Thread via GitHub
epotyom commented on issue #13671: URL: https://github.com/apache/lucene/issues/13671#issuecomment-2314681028 @gsmiller , there is new method ```java public void DrillSideways#search( DrillDownQuery query, CollectorOwner drillDownCollectorOwner, List> drillSidewaysCol