zhaih commented on PR #12371: URL: https://github.com/apache/lucene/pull/12371#issuecomment-1595003976
Thank you @Jackyrie2 for working on it, I think @benwtrent 's concern about memory makes sense but seems to me we should be able to reduce the memory usage later on (as this is still a draft) Maybe we should first run some benchmarks? As Benjamin mentioned we have some test tools in luceneutil. I would suggest you take a look at [this method](https://github.com/mikemccand/luceneutil/blob/master/src/main/KnnGraphTester.java#L690) and maybe try to create an extreme case index (like 2 segments, one super big one super small) and try to merge the segments to measure the best possible gain of this change first. Then maybe run a normal benchmark with [knnGraphTester](https://github.com/mikemccand/luceneutil/blob/master/src/python/knnPerfTest.py) to see whether we can observe some gain during normal index build. (I think @msokolov has added high dimensional vector support, maybe we should use higher dimensional vectors to better see the gain) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
