Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-15 Thread via GitHub
jpountz commented on PR #13636: URL: https://github.com/apache/lucene/pull/13636#issuecomment-2291136659 A few queries got a small-ish speedup, e.g. - [AndMedOrHighHigh](https://home.apache.org/~mikemccand/lucenebench/AndMedOrHighHigh.html) +2.5% - [AndHighHigh](https://home.apache

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-14 Thread via GitHub
ChrisHegarty commented on PR #13636: URL: https://github.com/apache/lucene/pull/13636#issuecomment-2288426886 Apologies for not interacting on this PR before, but I was away. Thanks for fixing the type of the length parameter in MemorySegmentIndexInput. I'll spend some time studying the cod

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-13 Thread via GitHub
jpountz merged PR #13636: URL: https://github.com/apache/lucene/pull/13636 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apa

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-12 Thread via GitHub
gsmiller commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1714509772 ## lucene/core/src/java/org/apache/lucene/codecs/lucene912/ForUtil.java: ## @@ -300,13 +301,14 @@ int numBytes(int bitsPerValue) { return bitsPerValue << (BLOCK_

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-12 Thread via GitHub
jpountz commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1714276855 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java: ## @@ -0,0 +1,109 @@ +/* + * Licensed to the Apache Software

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-12 Thread via GitHub
jpountz commented on PR #13636: URL: https://github.com/apache/lucene/pull/13636#issuecomment-2284765651 @uschindler @rmuir Any objections to merging this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-11 Thread via GitHub
uschindler commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1713051966 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java: ## @@ -0,0 +1,109 @@ +/* + * Licensed to the Apache Softwa

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-11 Thread via GitHub
jpountz commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1713050918 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java: ## @@ -0,0 +1,109 @@ +/* + * Licensed to the Apache Software

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-10 Thread via GitHub
uschindler commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1712624281 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java: ## @@ -0,0 +1,109 @@ +/* + * Licensed to the Apache Softwa

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-10 Thread via GitHub
uschindler commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1712624281 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java: ## @@ -0,0 +1,109 @@ +/* + * Licensed to the Apache Softwa

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-10 Thread via GitHub
uschindler commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1712624281 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java: ## @@ -0,0 +1,109 @@ +/* + * Licensed to the Apache Softwa

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-09 Thread via GitHub
jpountz commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1712554332 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java: ## @@ -0,0 +1,109 @@ +/* + * Licensed to the Apache Software

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-09 Thread via GitHub
gsmiller commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1711646318 ## lucene/core/src/java/org/apache/lucene/internal/vectorization/DefaultPostingDecodingUtil.java: ## @@ -0,0 +1,41 @@ +/* + * Licensed to the Apache Software Foundati

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-09 Thread via GitHub
uschindler commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1711262142 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java: ## @@ -0,0 +1,109 @@ +/* + * Licensed to the Apache Softwa

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-09 Thread via GitHub
uschindler commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1711262142 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java: ## @@ -0,0 +1,109 @@ +/* + * Licensed to the Apache Softwa

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-09 Thread via GitHub
jpountz commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1711141270 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java: ## @@ -0,0 +1,109 @@ +/* + * Licensed to the Apache Software

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-09 Thread via GitHub
uschindler commented on PR #13636: URL: https://github.com/apache/lucene/pull/13636#issuecomment-2277393866 I cleaned up the constants a bit more, there are still some duplicated in VectorUtilSupport, but I'd change this in the followup PR. -- This is an automated message from the Apache

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
rmuir commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1710505009 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java: ## @@ -0,0 +1,109 @@ +/* + * Licensed to the Apache Software Fo

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
rmuir commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1710505391 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Fou

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
rmuir commented on PR #13636: URL: https://github.com/apache/lucene/pull/13636#issuecomment-2276947096 > This class in the Panama codebase is huge and hard to read, so splitting into byte/float implementations looks like a good cleanup. No public API is involved. Cool idea, thanks fo

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
uschindler commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1709965505 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java: ## @@ -37,15 +44,23 @@ private static T doPrivileged(Privilege

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
uschindler commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1709961906 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java: ## @@ -16,20 +16,56 @@ */ package org.apache.lucene.internal

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
uschindler commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1709899394 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java: ## @@ -16,20 +16,56 @@ */ package org.apache.lucene.internal

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
uschindler commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1709866139 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java: ## @@ -16,20 +16,56 @@ */ package org.apache.lucene.internal

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
jpountz commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1709880508 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java: ## @@ -16,20 +16,56 @@ */ package org.apache.lucene.internal.ve

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
jpountz commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1709749808 ## lucene/core/src/java/org/apache/lucene/codecs/lucene912/ForUtil.java: ## @@ -630,55 +637,55 @@ private static void decode1(DataInput in, long[] tmp, long[] longs)

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
jpountz commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1709705545 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java: ## @@ -16,20 +16,56 @@ */ package org.apache.lucene.internal.ve

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
uschindler commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1709520659 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java: ## @@ -16,20 +16,56 @@ */ package org.apache.lucene.internal

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
jpountz commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1709403728 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java: ## @@ -16,20 +16,56 @@ */ package org.apache.lucene.internal.ve

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
jpountz commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1709378126 ## lucene/core/src/java/org/apache/lucene/internal/vectorization/DefaultPostingDecodingUtil.java: ## @@ -0,0 +1,41 @@ +/* + * Licensed to the Apache Software Foundatio

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
jpountz commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1709362066 ## lucene/core/src/java/org/apache/lucene/internal/vectorization/DefaultPostingDecodingUtil.java: ## @@ -0,0 +1,41 @@ +/* + * Licensed to the Apache Software Foundatio

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
uschindler commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1709359720 ## lucene/core/src/java/org/apache/lucene/codecs/lucene912/PostingIndexInput.java: ## @@ -0,0 +1,55 @@ +/* + * Licensed to the Apache Software Foundation (ASF) unde

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
uschindler commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1709358369 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java: ## @@ -16,20 +16,56 @@ */ package org.apache.lucene.internal

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
jpountz commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1709340084 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java: ## @@ -16,20 +16,56 @@ */ package org.apache.lucene.internal.ve

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
jpountz commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1709339018 ## lucene/core/src/java/org/apache/lucene/codecs/lucene912/PostingIndexInput.java: ## @@ -0,0 +1,55 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under o

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
jpountz commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1709336210 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java: ## @@ -79,4 +113,15 @@ public VectorUtilSupport getVectorUtilSuppo

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
uschindler commented on PR #13636: URL: https://github.com/apache/lucene/pull/13636#issuecomment-2275507708 > To help with @rmuir's fast integer vectors issues: Once this is merged, I plan to split the VectorUtilSupport into ByteVectorUtilSupport and FloatVectorUtilSupport and use 2 getters

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
uschindler commented on PR #13636: URL: https://github.com/apache/lucene/pull/13636#issuecomment-2275500767 As discussed before: I'd move all static constants to a package-private class, so it is initialized a bit delayed and not in class loading of VectorizationProvider. To help wit

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
uschindler commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1709095157 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java: ## @@ -16,20 +16,56 @@ */ package org.apache.lucene.internal

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
jpountz commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1708960612 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java: ## @@ -79,4 +102,18 @@ public VectorUtilSupport getVectorUtilSuppo

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
jpountz commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1708959969 ## lucene/core/src/java/org/apache/lucene/internal/vectorization/DefaultPostingDecodingUtil.java: ## @@ -0,0 +1,41 @@ +/* + * Licensed to the Apache Software Foundatio

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
jpountz commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1708943154 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software F

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
jpountz commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1708936620 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software F

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-07 Thread via GitHub
rmuir commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1707498886 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Fou

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-07 Thread via GitHub
uschindler commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1707463821 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java: ## @@ -79,4 +102,18 @@ public VectorUtilSupport getVectorUtilSu

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-07 Thread via GitHub
gsmiller commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1707451406 ## lucene/core/src/java/org/apache/lucene/internal/vectorization/DefaultPostingDecodingUtil.java: ## @@ -0,0 +1,41 @@ +/* + * Licensed to the Apache Software Foundati

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-07 Thread via GitHub
gsmiller commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1707451406 ## lucene/core/src/java/org/apache/lucene/internal/vectorization/DefaultPostingDecodingUtil.java: ## @@ -0,0 +1,41 @@ +/* + * Licensed to the Apache Software Foundati

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-07 Thread via GitHub
uschindler commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1707448409 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Softwar

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-07 Thread via GitHub
uschindler commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1707445042 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Softwar

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-07 Thread via GitHub
rmuir commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1707377294 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Fou

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-07 Thread via GitHub
rmuir commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1707373761 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Fou

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-07 Thread via GitHub
rmuir commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1707370205 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Fou

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-07 Thread via GitHub
jpountz commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1707341860 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software F

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-07 Thread via GitHub
jpountz commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1707332904 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software F

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-07 Thread via GitHub
jpountz commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1707325377 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software F

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-06 Thread via GitHub
rmuir commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1706278927 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Fou

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-06 Thread via GitHub
rmuir commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1706272955 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Fou

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-06 Thread via GitHub
rmuir commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1706268616 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Fou

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-06 Thread via GitHub
rmuir commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1706267079 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Fou

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-06 Thread via GitHub
jpountz commented on PR #13636: URL: https://github.com/apache/lucene/pull/13636#issuecomment-2272150091 For what it's worth, this PR is quite different from https://github.com/apache/lucene/pull/12412 in that it does not rewrite `ForUtil.java` completely, only the bits where we read some l

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-06 Thread via GitHub
jpountz commented on PR #13636: URL: https://github.com/apache/lucene/pull/13636#issuecomment-2272145780 Thanks @uschindler, it was helpful. I refactored the PR a bit based on your recommendation. It's now ready for review. -- This is an automated message from the Apache Git Service. To r

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-06 Thread via GitHub
uschindler commented on PR #13636: URL: https://github.com/apache/lucene/pull/13636#issuecomment-2271580963 > how to make it properly work out of the box (without enabling the vector module and preview features) The vector module must always be enabled, but that's also the case for `

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-06 Thread via GitHub
jpountz commented on PR #13636: URL: https://github.com/apache/lucene/pull/13636#issuecomment-2271546834 Oh, I had forgotten about this other PR! Thanks Uwe, I'll look into it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-06 Thread via GitHub
uschindler commented on PR #13636: URL: https://github.com/apache/lucene/pull/13636#issuecomment-2271539092 Hi, your setup needs to be a bit different: - Don't invent new providers, use the existing one, so add a new factory method to the generic VectorizationProvider. Here you return

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-06 Thread via GitHub
jpountz commented on PR #13636: URL: https://github.com/apache/lucene/pull/13636#issuecomment-2271488324 @uschindler @ChrisHegarty I could use a bit of help with this change regarding code organization and how to make it properly work out of the box (without enabling the vector module and p

[PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-06 Thread via GitHub
jpountz opened a new pull request, #13636: URL: https://github.com/apache/lucene/pull/13636 Our postings use a layout that helps take advantage of Java's auto-vectorization to be reasonably fast to decode. But we can make it a bit faster by using explicit vectorization on MemorySegment: