uschindler edited a comment on pull request #177: URL: https://github.com/apache/lucene/pull/177#issuecomment-866054411
Hi @mikemccand, I will open a few more issues in luceneutil. The way how it invokes the JVM is not like it is done in production: - `-Xbatch` as default doesn't help for reproducibility and makes the results problematic for production use, as unrealistic. - `-XX:-TieredCompilation` as default is only useful to run tests (so we don't spend too much time for recompile), but as it is default in modern JDKS since Java 8, it should really be swritched on for benchmarks. Benchmarks run long time and they get real speed boosts by this. While testing this pull request, disabling tiered compilation made results 20% worse because it prevented escape analysis from kicking in early enough. With modern JVMs, doing batch compilation brings nothing, as a lot of stuff is optimized now also on the Java side, not just in Hotspot. E.g., when a lambda or functional method reference is used with stream() API, they are also optimized at runtime outside hotspot by rewriting it to temporary classes and optimized byte code (lambda trasformations, invokedynamic). So in general the benchmark should mimic real-live. You are repeating the runs several times anyways, so changing the optimizations to be non-default is not a good idea, because your settings do not have less fluctuation don't help, they slow down! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
