iprithv opened a new pull request, #15832:
URL: https://github.com/apache/lucene/pull/15832

   ## Summary
   This change adds SIMD acceleration for popcount-based `FixedBitSet` hot 
paths (e.g. `cardinality()`, `intersectionCount`, `unionCount`, `andNotCount`) 
using the Java Vector API on Java 25+.
   
   The Vector API implementation lives in Lucene’s Java 25 multi-release 
sources so base classes do not link against incubator Vector API types unless 
the runtime supports them. If `jdk.incubator.vector` is not resolved (or on 
unsupported JVM configurations), the code falls back to the existing scalar 
implementations.
   
   ## Implementation notes
   - New internal dispatcher: `org.apache.lucene.util.VectorizedBitSetOps`
   - Java 25+ Vector API helper (MRJAR): 
`lucene/core/src/java25/.../VectorizedBitSetOpsHelper.java`
   - Runtime gating:
     - HotSpot only, no JVMCI, no Client VM
     - requires `--add-modules jdk.incubator.vector` for the vector module to 
be resolved
     - otherwise scalar fallback (no functional change)
   - Thresholding (small bitset fallback):
     - Vectorization is enabled only for lengths >= 64 64-bit words (4096 
bits). Smaller bitsets use the scalar path. This avoids regressions on small 
filters where scalar `Long.bitCount` intrinsics are highly efficient and Vector 
API setup/tail overhead can dominate.
   
   ## Test plan
   - ./gradlew :lucene:core:test --tests org.apache.lucene.util.TestFixedBitSet
   
   ## Benchmarks
   
   ### JMH (microbench)
   Build:
     ./gradlew :lucene:benchmark-jmh:assemble
   
   Run (baseline and patch, Vector API enabled):
     java --add-modules jdk.incubator.vector -jar 
lucene/benchmark-jmh/build/benchmarks/lucene-benchmark-jmh-11.0.0-SNAPSHOT.jar \
       
"FixedBitSetBenchmark\.(cardinality|intersectionCount|unionCount|andNotCount)$" 
\
       -p numBits=512,1024,4096,65536,1048576 -p density=0.10 -wi 3 -i 5 -f 1
   
   Results (ops/us; higher is better). Threshold is 4096 bits, so <4096-bit 
cases remain scalar.
   
   Size (bits) | Operation         | Baseline | Patch  | Gain
   -----------|-------------------|----------|--------|------
   4096       | cardinality       | 31.392   | 51.012 | ~1.63×
   4096       | intersectionCount | 8.547    | 47.470 | ~5.55×
   4096       | unionCount        | 8.579    | 49.107 | ~5.72×
   4096       | andNotCount       | 8.607    | 46.408 | ~5.39×
   65536      | cardinality       | 1.776    | 3.518  | ~1.98×
   65536      | intersectionCount | 0.508    | 3.449  | ~6.79×
   65536      | unionCount        | 0.518    | 3.424  | ~6.61×
   65536      | andNotCount       | 0.533    | 3.439  | ~6.45×
   1048576    | cardinality       | 0.109    | 0.223  | ~2.05×
   1048576    | intersectionCount | 0.033    | 0.220  | ~6.67×
   1048576    | unionCount        | 0.033    | 0.221  | ~6.70×
   1048576    | andNotCount       | 0.033    | 0.221  | ~6.70×
   
   ### luceneutil (end-to-end) — wikimedium1m, searchConcurrency=1
   Command:
     cd /Users/prithvi/work/util
     python src/python/localrun.py -source wikimedium1m -searchConcurrency 1 \
       -b /Users/prithvi/work/lucene_baseline \
       -c /Users/prithvi/work/lucene \
       -iterations 10 -warmups 10
   
   Observed: mostly small deltas, but a few tasks regressed in this run, e.g.
   - HighTermDayOfYearSort: about -12.5% (p≈0.047)
   - MedPhrase: about -4.3% (p≈0.039)
   
   The change is gated with a conservative small-bitset threshold to keep small 
bitsets on the scalar path; further tuning may be needed for this workload.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to