[
https://issues.apache.org/jira/browse/LUCENE-7882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17191381#comment-17191381
]
Uwe Schindler edited comment on LUCENE-7882 at 9/6/20, 9:57 PM:
----------------------------------------------------------------
Hi Mike,
Java 15 now has the "solutions" for the problems you are seeing. We can now use
"hidden classes" ([JEP-371|https://openjdk.java.net/jeps/371]) for this.
Performance should be much better and pressure on GC lower.
Here is a first PR: https://github.com/apache/lucene-solr/pull/1837
Backside: We don't see the expression in stack frames anymore. I have no idea
how to solve this, we may need to make the source code available in a different
way on errors (see my question on openjdk mailing lists:
https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-September/068542.html).
The attached PR does not pass all tests because of missing stack frames, but
the expressions module works.
Could you test the attached PR with JDK 15 and your test code that had the
problems?
was (Author: thetaphi):
Hi Mike,
Java 15 now has the "solutions" for the problems you are seeing. We can now use
"hidden classes" ([JEP-371|https://openjdk.java.net/jeps/371]) for this.
Performance should be much better and pressure on GC lower.
Here is a first PR: https://github.com/apache/lucene-solr/pull/1837
Backside: We don't see the expression in stack frames anymore. I have no idea
how to solve this, we may need to make the source code available in a different
way on errors. The attached PR does not pass all tests because of missing stack
frames, but the expressions module works.
Could you test the attached PR with JDK 15 and your test code that had the
problems?
> Maybe expression compiler should cache recently compiled expressions?
> ---------------------------------------------------------------------
>
> Key: LUCENE-7882
> URL: https://issues.apache.org/jira/browse/LUCENE-7882
> Project: Lucene - Core
> Issue Type: Improvement
> Components: modules/expressions
> Reporter: Michael McCandless
> Assignee: Uwe Schindler
> Priority: Major
> Attachments: demo.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> I've been running search performance tests using a simple expression
> ({{_score + ln(1000+unit_sales)}}) for sorting and hit this odd bottleneck:
> {noformat}
> "pool-1-thread-30" #70 prio=5 os_prio=0 tid=0x00007eea7000a000 nid=0x1ea8a
> waiting for monitor entry [0x00007eea867dd000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.lucene.expressions.js.JavascriptCompiler$CompiledExpression.evaluate(_score
> + ln(1000+unit_sales))
> at
> org.apache.lucene.expressions.ExpressionFunctionValues.doubleValue(ExpressionFunctionValues.java:49)
> at
> com.amazon.lucene.OrderedVELeafCollector.collectInternal(OrderedVELeafCollector.java:123)
> at
> com.amazon.lucene.OrderedVELeafCollector.collect(OrderedVELeafCollector.java:108)
> at
> org.apache.lucene.search.MultiCollectorManager$Collectors$LeafCollectors.collect(MultiCollectorManager.java:102)
> at
> org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:241)
> at
> org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:184)
> at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:658)
> at org.apache.lucene.search.IndexSearcher$5.call(IndexSearcher.java:600)
> at org.apache.lucene.search.IndexSearcher$5.call(IndexSearcher.java:597)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> I couldn't see any {{synchronized}} in the sources here, so I'm not sure
> which object monitor it's blocked on.
> I was accidentally compiling a new expression for every query, and that
> bottleneck would cause overall QPS to slow down drastically (~4X slower after
> ~1 hour of redline tests), as if the JVM is getting slower and slower to
> evaluate each expression the more expressions I had compiled.
> I tested JDK 9-ea and it also kept slowing down over time as the performance
> test ran.
> Maybe we should put a small cache in front of the expressions compiler to
> make it less trappy? Or maybe we can get to the root cause of why the JVM
> slows down more and more, the more expressions you compile?
> I won't have time to work on this in the near future so if anyone else feels
> the itch, please scratch it!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
