[GitHub] [lucene] rmuir commented on a change in pull request #69: LUCENE-9850: Use PFOR encoding for doc IDs (instead of FOR)

[GitBox]

rmuir commented on a change in pull request #69:
URL: https://github.com/apache/lucene/pull/69#discussion_r608759137



##########
File path: lucene/core/src/java/org/apache/lucene/codecs/lucene90/PForUtil.java
##########
@@ -121,4 +167,146 @@ void skip(DataInput in) throws IOException {
       in.skipBytes(forUtil.numBytes(bitsPerValue) + (numExceptions << 1));
     }
   }
+
+  /**
+   * Fill {@code longs} with the final values for the case of all deltas being 
1. Note this assumes
+   * there are no exceptions to apply.
+   */
+  private static void prefixSumOfOnes(long[] longs, long base) {
+    System.arraycopy(IDENTITY_PLUS_ONE, 0, longs, 0, ForUtil.BLOCK_SIZE);
+    // This loop gets auto-vectorized
+    for (int i = 0; i < ForUtil.BLOCK_SIZE; ++i) {
+      longs[i] += base;
+    }
+  }
+
+  /**
+   * Fill {@code longs} with the final values for the case of all deltas being 
{@code val}. Note
+   * this assumes there are no exceptions to apply.
+   */
+  private static void prefixSumOf(long[] longs, long base, long val) {
+    for (int i = 0; i < ForUtil.BLOCK_SIZE; i++) {
+      longs[i] = (i + 1) * val + base;

Review comment:
       here's my notes for building the shared library from last time i 
recently did this (may be incomplete, sorry):
   
   ```
   $ git clone g...@github.com:openjdk/jdk.git
   $ cd jdk/src/utils/hsdis/build
   $ curl -fLO https://ftp.gnu.org/gnu/binutils/binutils-2.36.tar.gz
   $ gzip -dc binutils-2.36.tar.gz | tar -xvf -
   $ mv binutils-2.36 binutils
   $ cd ..
   $ make ARCH=amd64
   ```
   
   This will create build/linux-amd64/hsdis-amd64.so
   For me it initially fails because of binutils enabling `-Werror` (i have a 
newer gcc), so i had to hack the Makefile for it to really work:
   ```
   --- a/src/utils/hsdis/Makefile
   +++ b/src/utils/hsdis/Makefile
   @@ -192,7 +192,7 @@ $(LIBRARIES): $(TARGET_DIR) $(TARGET_DIR)/Makefile
           if [ ! -f $@ ]; then cd $(TARGET_DIR); make all-opcodes; fi
   
    $(TARGET_DIR)/Makefile:
   -       (cd $(TARGET_DIR); CC=$(CC) CFLAGS="$(CFLAGS)" AR="$(AR)" 
$(BINUTILSDIR)/configure --disable-nls $(CONFIGURE_ARGS))
   +       (cd $(TARGET_DIR); CC=$(CC) CFLAGS="$(CFLAGS)" AR="$(AR)" 
$(BINUTILSDIR)/configure --disable-werror --disable-nls $(CONFIGURE_ARGS))
   ```
   
   Now you can print assembly generated from the JIT, but you need to ensure 
the code is really fully C2-compiled so that you are looking at the correct 
stuff, etc.... essentially target the code in question and heat it up. So I use 
JMH for that, they have instructions in the repo: https://github.com/openjdk/jmh




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org