lizhiyuell commented on PR #16526: URL: https://github.com/apache/lucene/pull/16526#issuecomment-5599312406
@msokolov Thanks for your interest in PathSeer! Over the past few weeks, we have continued evolving PathSeer’s search logic and improving its performance across a broader range of workloads. Based on the evaluations and discussions during our [Weaviate integration](https://redirect.github.com/weaviate/weaviate/pull/12137), we made two improvements to PathSeer’s search logic. With the improvements, PathSeer has now been merged into Weaviate’s main branch: 1. We changed the phase-switching condition between PathSeer’s two search stages to depend on whether the result heap has reached `ef`. This allows the transition point to adapt naturally to the selectivity of the workload, and has proven effective in improving search performance across different workload settings. 2. We added a guard to the phase-switching condition to handle cases where the number of vectors satisfying the filter is smaller than `ef`, preventing the search from degenerating in such cases. We have incorporated these improvements into this Lucene PR as well. We would be very happy to discuss any suggestions or feedback you may have on our approach, and please let us know if there is anything else we can improve or adjust. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
