xande commented on PR #15903:
URL: https://github.com/apache/lucene/pull/15903#issuecomment-4165138163
> ```
> recall latency(ms) netCPU avgCpuCount nDoc topK fanout maxConn
beamWidth quantized index(s) index_docs/s force_merge(s) num_segments
index_size(MB) vec_disk(MB) vec_RAM(MB) indexType
> 0.875 0.858 0.855 0.996 1000000 10 100 32
250 -4 bits 203.69 4909.49 168.11 1
3349.44 3311.157 381.470 HNSW
> ```
>
> Thats the last run @mccullocht did for Lucene's OSQ technique (1M vectors,
would need to do the exact same data set for Apples to apples).
>
> I realize performance apples to apples will take way more work (panama
vector APIs, etc.). I am more concerned about recall, and I am not sure TQ will
provide any significant recall improvement itself.
>
> The main thing I think that OSQ might be missing is some random rotation
for non-guassian component vectors (which are an anomaly for the modern
models). But, adding that to the existing OSQ for Lucene would be a snap
(though careful thought would be needed as that could be a significant
performance burden with very little benefit for many users).
>
> It would be good to just do a "flat" index to remove any HNSW noise.
Interestingly enough, at Amazon we were putting a lot of bets on OSQ, though
on internal datasets we did not see meaningful recall improvement vs non-
OSQ - low enough for us not to use it. I am planning to run more benchmarks
to compare and see how TQ compares.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
