mikemccand commented on PR #12624:
URL: https://github.com/apache/lucene/pull/12624#issuecomment-1829613228
Thanks @dungba88 -- I will catch up with the latest iterations soon. I
tested just how much slower the `ByteBuffer` based store is than the FST's
`BytesStore`:
9.x:
```
1> TEST: now verify [fst size=5368709408; nodeCount=2252341463;
arcCount=2264078568]
1> 144.4s 0...
1> 150.7s 1000000...
1> 157.1s 2000000...
1> 163.5s 3000000...
1> 169.9s 4000000...
1> 176.2s 5000000...
1> 182.6s 6000000...
1> 188.9s 7000000...
1> 195.3s 8000000...
1> 201.6s 9000000...
1> 208.0s 10000000...
1> 214.3s 11000000...
```
This PR:
```
1> TEST: now verify [fst size=4622366544; nodeCount=2252341629;
arcCount=2264078736]
1> 0.0s 0...
1> 599.5s 100000...
1> 1201.7s 200000...
1> 1797.6s 300000...
1> 2392.0s 400000...
1> 2967.8s 500000...
1> 3532.1s 600000...
1> 4110.5s 700000...
1> 4678.4s 800000...
1> 5247.9s 900000...
1> 5815.1s 1000000...
1> 6382.3s 1100000...
1> 6950.0s 1200000...
1> 7514.7s 1300000...
1> 8087.2s 1400000...
1> 8657.6s 1500000...
1> 9227.2s 1600000...
1> 9795.8s 1700000...
1> 10358.1s 1800000...
1> 10923.6s 1900000...
1> 11491.5s 2000000...
1> 12057.4s 2100000...
1> 12649.5s 2200000...
1> 13231.7s 2300000...
1> 13797.8s 2400000...
1> 14382.8s 2500000...
1> 15017.1s 2600000...
1> 15633.8s 2700000...
1> 16200.9s 2800000...
1> 16762.8s 2900000...
1> 17329.6s 3000000...
1> 17891.7s 3100000...
1> 18462.2s 3200000...
1> 19025.3s 3300000...
1> 19609.4s 3400000...
1> 20171.3s 3500000...
1> 20738.6s 3600000...
1> 21306.2s 3700000...
1> 21872.2s 3800000...
1> 22439.8s 3900000...
1> 22998.3s 4000000...
```
More than two orders-of-magnitude (base 10) slower!
+1 to `freeze` and do the output -> input conversion only once. Thanks
@dungba88.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
