iverase commented on pull request #541:
URL: https://github.com/apache/lucene/pull/541#issuecomment-1014284136
This is looking much better. I tried some of my benchmarks and for points it
looks like:
```
|Approach||Index time (sec)||Force merge time (sec)||Index size (GB)||Reader
heap (MB)||
||Dev||Base||Diff ||Dev ||Base ||diff
||Dev||Base||Diff||Dev||Base||Diff ||
|points|104.2s|104.2s|-0%|0.0s|0.0s| 0%|0.50|0.50|-0%|0.00|0.00| 0%|
|geo3d|135.0s|135.1s|-0%|0.0s|0.0s| 0%|0.70|0.70|-0%|0.00|0.00| 0%|
|shapes|168.6s|167.6s| 1%|0.0s|0.0s| 0%|1.26|1.26|-0%|0.00|0.00| 0%|
()
||Approach||Shape||M hits/sec ||QPS ||Hit count ||
||Dev||Base ||Diff||Dev||Base||Diff||Dev||Base||Diff||
|points|polyRussia|16.15|16.00| 1%|4.60|4.56| 1%|3508846|3508846| 0%|
|points|polyMedium|9.33|9.33|-0%|114.28|114.33|-0%|2693559|2693559| 0%|
|points|poly 10|87.86|86.64| 1%|55.56|54.79| 1%|355809475|355809475| 0%|
|points|box|91.51|91.54|-0%|93.11|93.15|-0%|221118844|221118844| 0%|
|points|distance|86.66|85.71| 1%|50.91|50.36| 1%|382961957|382961957| 0%|
|points|nearest 10|0.03|0.03|-1%|2752.04|2783.10|-1%|60844404|60844404| 0%|
|geo3d|polyRussia|0.71|0.69| 3%|0.20|0.20| 3%|3508671|3508671| 0%|
|geo3d|polyMedium|0.74|0.74|-0%|9.10|9.10|-0%|2693541|2693541| 0%|
|geo3d|poly 10|57.57|56.69| 2%|36.40|35.84| 2%|355855227|355855227| 0%|
|geo3d|box|62.51|60.90| 3%|63.60|61.97| 3%|221118844|221118844| 0%|
|geo3d|distance|76.84|76.16| 1%|45.10|44.70| 1%|383371904|383371904| 0%|
|shapes|polyRussia|10.08|10.49|-4%|2.87|2.99|-4%|3508846|3508846| 0%|
|shapes|polyMedium|3.85|3.80| 1%|47.18|46.52| 1%|2693559|2693559| 0%|
|shapes|poly 10|44.75|41.90| 7%|28.30|26.49| 7%|355809475|355809475| 0%|
|shapes|box|47.84|44.50| 8%|48.68|45.29| 8%|221118844|221118844| 0%|
|shapes|distance|49.17|46.19| 6%|28.89|27.14| 6%|382961957|382961957| 0%|
```
There is a nice improvement for high dimensional cases. This is similar to
what I was seeing except in my approach I noticed a bigger slow down for the
2-D case. I think this type of change speeds up the case when we visit the leaf
node but it can slow down the case where we visit only doc_Ids. Your approach
seems to limit the second case.
On the other hand, I tried another benchmark with LatLonShape and I am
seeing an error. I haven't been able to dug it properly but I think we need to
understand the error before moving forward:
```
0.4s: intersects, polyRussia: Exception in thread "main"
java.lang.ArrayIndexOutOfBoundsException: Index 153844 out of bounds for length
92548
0.4s: intersects, polyRussia: at
org.apache.lucene.util.FixedBitSet.get(FixedBitSet.java:186)
0.4s: intersects, polyRussia: at
org.apache.lucene.document.SpatialQuery$4.visit(SpatialQuery.java:479)
0.4s: intersects, polyRussia: at
org.apache.lucene.util.bkd.BKDReader$BKDPointTree.visitCompressedDocValues(BKDReader.java:870)
0.4s: intersects, polyRussia: at
org.apache.lucene.util.bkd.BKDReader$BKDPointTree.visitDocValuesWithCardinality(BKDReader.java:784)
0.4s: intersects, polyRussia: at
org.apache.lucene.util.bkd.BKDReader$BKDPointTree.visitDocValues(BKDReader.java:554)
0.4s: intersects, polyRussia: at
org.apache.lucene.util.bkd.BKDReader$BKDPointTree.visitLeavesOneByOne(BKDReader.java:539)
0.4s: intersects, polyRussia: at
org.apache.lucene.util.bkd.BKDReader$BKDPointTree.visitDocValues(BKDReader.java:533)
0.4s: intersects, polyRussia: at
org.apache.lucene.index.PointValues.intersect(PointValues.java:364)
0.4s: intersects, polyRussia: at
org.apache.lucene.index.PointValues.intersect(PointValues.java:357)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
