Re: [PR] Introduce new encoding of BPV 21 for DocIdsWriter used in BKD Tree [lucene]

2025-03-18 Thread via GitHub
gf2121 merged PR #14361: URL: https://github.com/apache/lucene/pull/14361 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apac

Re: [PR] Introduce new encoding of BPV 21 for DocIdsWriter used in BKD Tree [lucene]

2025-03-18 Thread via GitHub
gf2121 closed pull request #13521: Introduce new encoding of BPV 21 for DocIdsWriter used in BKD Tree URL: https://github.com/apache/lucene/pull/13521 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Introduce new encoding of BPV 21 for DocIdsWriter used in BKD Tree [lucene]

2025-03-18 Thread via GitHub
gf2121 commented on PR #13521: URL: https://github.com/apache/lucene/pull/13521#issuecomment-2732037641 Closing this in favor of https://github.com/apache/lucene/pull/14361. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] Introduce new encoding of BPV 21 for DocIdsWriter used in BKD Tree [lucene]

2025-03-17 Thread via GitHub
gf2121 commented on PR #14361: URL: https://github.com/apache/lucene/pull/14361#issuecomment-2729280384 OK i get expected results that multiple of 16 faster than multiple of 8 when i force `-XX:UseAVX=3`, it can be seen AVX3 is slower on this chip, that may be why java disabled it by defaul

Re: [PR] Introduce new encoding of BPV 21 for DocIdsWriter used in BKD Tree [lucene]

2025-03-17 Thread via GitHub
gf2121 commented on PR #14361: URL: https://github.com/apache/lucene/pull/14361#issuecomment-2729250628 Thanks for feedback, > Should we floor to a multiple of 16 instead of 8 so that we have a perfect second loop with AVX-512 as well? That is what i thought initially. But my A

Re: [PR] Introduce new encoding of BPV 21 for DocIdsWriter used in BKD Tree [lucene]

2025-03-17 Thread via GitHub
jpountz commented on PR #14361: URL: https://github.com/apache/lucene/pull/14361#issuecomment-2729147376 Should we floor to a multiple of 16 instead of 8 so that we have a perfect second loop with AVX-512 as well? (By the way, which of your machine produced the above benchmark results?) Oth

Re: [PR] Introduce new encoding of BPV 21 for DocIdsWriter used in BKD Tree [lucene]

2025-03-17 Thread via GitHub
gf2121 commented on PR #14361: URL: https://github.com/apache/lucene/pull/14361#issuecomment-2729094239 Results on `wikimediumall`: ``` TaskQPS baseline StdDevQPS my_modified_version StdDevPct diff p-value

Re: [PR] Introduce new encoding of BPV 21 for DocIdsWriter used in BKD Tree [lucene]

2024-12-17 Thread via GitHub
github-actions[bot] commented on PR #13521: URL: https://github.com/apache/lucene/pull/13521#issuecomment-2549977673 This PR has not had activity in the past 2 weeks, labeling it as stale. If the PR is waiting for review, notify the d...@lucene.apache.org list. Thank you for your contributi

Re: [PR] Introduce new encoding of BPV 21 for DocIdsWriter used in BKD Tree [lucene]

2024-11-09 Thread via GitHub
github-actions[bot] commented on PR #13521: URL: https://github.com/apache/lucene/pull/13521#issuecomment-2466522392 This PR has not had activity in the past 2 weeks, labeling it as stale. If the PR is waiting for review, notify the d...@lucene.apache.org list. Thank you for your contributi

Re: [PR] Introduce new encoding of BPV 21 for DocIdsWriter used in BKD Tree [lucene]

2024-10-15 Thread via GitHub
expani commented on code in PR #13521: URL: https://github.com/apache/lucene/pull/13521#discussion_r1801081957 ## lucene/benchmark-jmh/src/java/org/apache/lucene/benchmark/jmh/DocIdEncodingBenchmark.java: ## @@ -0,0 +1,404 @@ +/* + * Licensed to the Apache Software Foundation (A

Re: [PR] Introduce new encoding of BPV 21 for DocIdsWriter used in BKD Tree [lucene]

2024-10-15 Thread via GitHub
expani commented on code in PR #13521: URL: https://github.com/apache/lucene/pull/13521#discussion_r1801053720 ## lucene/benchmark-jmh/src/java/org/apache/lucene/benchmark/jmh/DocIdEncodingBenchmark.java: ## @@ -0,0 +1,404 @@ +/* + * Licensed to the Apache Software Foundation (A

Re: [PR] Introduce new encoding of BPV 21 for DocIdsWriter used in BKD Tree [lucene]

2024-10-15 Thread via GitHub
expani commented on code in PR #13521: URL: https://github.com/apache/lucene/pull/13521#discussion_r1801052715 ## lucene/benchmark-jmh/src/java/org/apache/lucene/benchmark/jmh/DocIdEncodingBenchmark.java: ## @@ -0,0 +1,404 @@ +/* + * Licensed to the Apache Software Foundation (A

Re: [PR] Introduce new encoding of BPV 21 for DocIdsWriter used in BKD Tree [lucene]

2024-10-15 Thread via GitHub
expani commented on code in PR #13521: URL: https://github.com/apache/lucene/pull/13521#discussion_r1801052715 ## lucene/benchmark-jmh/src/java/org/apache/lucene/benchmark/jmh/DocIdEncodingBenchmark.java: ## @@ -0,0 +1,404 @@ +/* + * Licensed to the Apache Software Foundation (A

Re: [PR] Introduce new encoding of BPV 21 for DocIdsWriter used in BKD Tree [lucene]

2024-09-28 Thread via GitHub
github-actions[bot] commented on PR #13521: URL: https://github.com/apache/lucene/pull/13521#issuecomment-2381035791 This PR has not had activity in the past 2 weeks, labeling it as stale. If the PR is waiting for review, notify the d...@lucene.apache.org list. Thank you for your contributi

Re: [PR] Introduce new encoding of BPV 21 for DocIdsWriter used in BKD Tree [lucene]

2024-09-14 Thread via GitHub
gf2121 commented on code in PR #13521: URL: https://github.com/apache/lucene/pull/13521#discussion_r1759694410 ## lucene/benchmark-jmh/src/java/org/apache/lucene/benchmark/jmh/DocIdEncodingBenchmark.java: ## @@ -0,0 +1,404 @@ +/* + * Licensed to the Apache Software Foundation (A

Re: [PR] Introduce new encoding of BPV 21 for DocIdsWriter used in BKD Tree [lucene]

2024-08-29 Thread via GitHub
expani commented on PR #13521: URL: https://github.com/apache/lucene/pull/13521#issuecomment-2317985648 Thanks for taking the time to review @msfroh and @mikemccand I had enabled JVM Options to check what is going on with JIT C2 Compiler for the different encoder variations.

Re: [PR] Introduce new encoding of BPV 21 for DocIdsWriter used in BKD Tree [lucene]

2024-08-28 Thread via GitHub
mikemccand commented on PR #13521: URL: https://github.com/apache/lucene/pull/13521#issuecomment-2315947436 > @mikemccand -- did you mean to close this PR? Ugh, no I did not! Sorry, I'll reopen!! -- This is an automated message from the Apache Git Service. To respond to the message

Re: [PR] Introduce new encoding of BPV 21 for DocIdsWriter used in BKD Tree [lucene]

2024-08-28 Thread via GitHub
msfroh commented on PR #13521: URL: https://github.com/apache/lucene/pull/13521#issuecomment-2315946796 @mikemccand -- did you mean to close this PR? If @expani adds the unrolled version (`Bit21With3StepsEncoder`) to `DocIdsWriter`, the change should be ready to go, IMO. Even o

Re: [PR] Introduce new encoding of BPV 21 for DocIdsWriter used in BKD Tree [lucene]

2024-08-28 Thread via GitHub
mikemccand commented on PR #13521: URL: https://github.com/apache/lucene/pull/13521#issuecomment-2315942081 > We can replace it with Bit21With2StepsEncoder in future when the performance is comparable to x86. I wonder what mechanism we could use to remind ourselves when performance of

Re: [PR] Introduce new encoding of BPV 21 for DocIdsWriter used in BKD Tree [lucene]

2024-08-28 Thread via GitHub
mikemccand commented on PR #13521: URL: https://github.com/apache/lucene/pull/13521#issuecomment-2315935834 > @jpountz IMO We should use `Bit21With3StepsEncoder` in DocIdsWriter as using `Bit21With2StepsEncoder` might lead to performance regression for workloads in aarch64 platforms.

Re: [PR] Introduce new encoding of BPV 21 for DocIdsWriter used in BKD Tree [lucene]

2024-08-28 Thread via GitHub
mikemccand closed pull request #13521: Introduce new encoding of BPV 21 for DocIdsWriter used in BKD Tree URL: https://github.com/apache/lucene/pull/13521 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [PR] Introduce new encoding of BPV 21 for DocIdsWriter used in BKD Tree [lucene]

2024-08-28 Thread via GitHub
msfroh commented on PR #13521: URL: https://github.com/apache/lucene/pull/13521#issuecomment-2315907724 Okay -- I was able to speed up the SIMD implementation a fair bit. Honestly, my main stupid mistake was that I hadn't declared `LONG_SPECIES` as `static final`, which probably prevented s

Re: [PR] Introduce new encoding of BPV 21 for DocIdsWriter used in BKD Tree [lucene]

2024-08-27 Thread via GitHub
msfroh commented on PR #13521: URL: https://github.com/apache/lucene/pull/13521#issuecomment-2313906920 I tried modifying the loop to process 4 longs per iteration and noticed no difference on my Xeon host, which is unsurprising since there was no difference between 1 and 3. I also t

Re: [PR] Introduce new encoding of BPV 21 for DocIdsWriter used in BKD Tree [lucene]

2024-08-27 Thread via GitHub
msfroh commented on PR #13521: URL: https://github.com/apache/lucene/pull/13521#issuecomment-2313628836 The approach is pretty neat. I'm wondering if `Bit21With3StepsEncoder` does better on aarch64 because of the explicitly unrolled loop? If so, I'm wondering if unrolling to a multip

Re: [PR] Introduce new encoding of BPV 21 for DocIdsWriter used in BKD Tree [lucene]

2024-08-20 Thread via GitHub
expani commented on PR #13521: URL: https://github.com/apache/lucene/pull/13521#issuecomment-2298718372 While trying out 2 different ways ( one a subset of other ) I found that `Bit21With3StepsEncoder` is better than `Bit21With2StepsEncoder` in aarch64 platforms with OpenJDK 21/22 whereas t

Re: [PR] Introduce new encoding of BPV 21 for DocIdsWriter used in BKD Tree [lucene]

2024-07-11 Thread via GitHub
github-actions[bot] commented on PR #13521: URL: https://github.com/apache/lucene/pull/13521#issuecomment-2224171037 This PR has not had activity in the past 2 weeks, labeling it as stale. If the PR is waiting for review, notify the d...@lucene.apache.org list. Thank you for your contributi