markrmiller commented on pull request #177: URL: https://github.com/apache/lucene/pull/177#issuecomment-935338177
Just stumbled on this. I've also recently thought a bit about Lucene JMH benchmarks that are semi equivalent to lucene-util benchmarks. I've been spending some free time here and there trying to bring some JMH to lucene-util, mainly perfasm output and async profiler, for which I also have a rough patch to use in the tests waiting for some clean up. I have some rough working stuff, but there tends to be a lot of resulting noise in the outputs that I have to look at filtering out or targeting the capture more with a more invasive integration. More than once, I've thought, it would be nicer to just offer the same suite of benchmarks within JMH and capture the other things it offers as well. As Robert says, it's no replacement for realistic high level benchmarks in most case, especially for Solr, I've used the same unit/integration test analogy, but it's fantastic for fast, reliable feedback with superb introspection tools built in. Also, those lambdas *are* damn slow until hotspot takes care of them :) I saw that slowness way back and chased what the deal was for a while, I was ready to toss them out of my toolbox, and yeah, they just need time. Then they are fine, probably not great for places that don't run a lot. JMH will actually run the profilers during warmup runs to warm them up, but then it dumps the data, so it's nice that even profiling just captures the warm iterations if you'd like. I don't like that it also includes setup, as it says, to be sure and catch the edges. I could do without that when you are constructing indexes or something to be queried, but if you setup larger stuff like lucene-util does to have at hand already, that circumvents that. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
