costin commented on PR #16381: URL: https://github.com/apache/lucene/pull/16381#issuecomment-4929292184
Thanks for the feedback! I've moved the optimization inside CharArrayMap (and Set) so all consumers benefit from it, including StopFilter which is untouched. The cache threshold defaults to 8 characters and is configurable via constructor. The main motivation behind the PR is improving QPS, especially as LLM-driven agents are generating more search traffic than ever. Updated the micro-benchmark to test CharArrayMap with/without the feature (same machine type and sample size): | Text type | baseline (ops/µs) | packed (ops/µs) | Speedup | |-----------|-------------------|-----------------|---------| | english | 0.024 ± 0.002 | 0.068 ± 0.001 | 2.8x | | technical | 0.024 ± 0.003 | 0.065 ± 0.002 | 2.7x | Let me know what you think. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
