mikemccand commented on PR #12517: URL: https://github.com/apache/lucene/pull/12517#issuecomment-2025238979
> The built kuromoji jar with unidic-cwj-3.1.1-full eventually becomes 442M. Besides the size, I think we should consider performance. I'm worried that there can be a significant impact on analysis/indexing speed. Do you have any benchmark result on that? luceneutil has a `runAnalyzerPerf.py` benchmark, and the nightly script runs it and [posts the results here](https://home.apache.org/~mikemccand/lucenebench/analyzers.html), including Kuromoji. We could maybe test tokens/sec throughput using that benchy for this PR running on UniDic? I'm not sure how we would conclude whether it's fast enough since current Kuromoji (`main`) can't build UniDic. Maybe we could defer performance testing / optimizing on UniDic to a later issue... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
