stefanvodita commented on PR #12844: URL: https://github.com/apache/lucene/pull/12844#issuecomment-1832732598
Thanks for the suggestions @zhaih! I have to think about option 2 a bit more. If we change `ramBytesUsed` back, performance recovers (Mostly? I'm not sure how noisy this benchmark is). Benchmark config: ``` dim = 100 doc_vectors = '%s/data/enwiki-20120502-lines-1k-100d.vec' % constants.BASE_DIR query_vectors = '%s/util/tasks/vector-task-minilm.vec' % constants.BASE_DIR ``` main: ``` recall latency nDoc fanout maxConn beamWidth visited index ms 0.720 0.16 10000 0 64 250 100 2551 1.00 post-filter 0.542 0.23 100000 0 64 250 100 43620 1.00 post-filter 0.512 0.27 200000 0 64 250 100 108235 1.00 post-filter ``` This PR: ``` recall latency nDoc fanout maxConn beamWidth visited index ms 0.720 0.16 10000 0 64 250 100 5028 1.00 post-filter 0.542 0.23 100000 0 64 250 100 315369 1.00 post-filter 0.512 0.28 200000 0 64 250 100 1578461 1.00 post-filter ``` This PR with the old memory estimation: ``` recall latency nDoc fanout maxConn beamWidth visited index ms 0.720 0.16 10000 0 64 250 100 2497 1.00 post-filter 0.542 0.23 100000 0 64 250 100 43886 1.00 post-filter 0.512 0.28 200000 0 64 250 100 117152 1.00 post-filter ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
