benwtrent commented on code in PR #14154:
URL: https://github.com/apache/lucene/pull/14154#discussion_r1929163572
##########
lucene/analysis/common/src/java/org/apache/lucene/analysis/query/QueryAutoStopWordAnalyzer.java:
##########
@@ -126,7 +126,7 @@ public QueryAutoStopWordAnalyzer(
public QueryAutoStopWordAnalyzer(
Analyzer delegate, IndexReader indexReader, Collection<String> fields,
int maxDocFreq)
throws IOException {
- super(delegate.getReuseStrategy());
+ super(PER_FIELD_REUSE_STRATEGY);
Review Comment:
Why did this change?
##########
lucene/core/src/java/org/apache/lucene/analysis/AnalyzerWrapper.java:
##########
@@ -151,4 +160,46 @@ protected final Reader initReaderForNormalization(String
fieldName, Reader reade
protected final AttributeFactory attributeFactory(String fieldName) {
return getWrappedAnalyzer(fieldName).attributeFactory(fieldName);
}
+
+ public TokenStreamComponents getWrappedComponents(String fieldName) {
+ return wrappedComponentsPerField.get(fieldName);
+ }
+
+ /**
+ * A {@link org.apache.lucene.analysis.Analyzer.ReuseStrategy} that checks
the wrapped analyzer's
+ * strategy for reusability. If the wrapped analyzer's strategy returns
null, components need to
+ * be re-created.
+ */
+ public static final class UnwrappingReuseStrategy extends ReuseStrategy {
+ private final ReuseStrategy reuseStrategy;
+
+ public UnwrappingReuseStrategy(ReuseStrategy reuseStrategy) {
+ this.reuseStrategy = reuseStrategy;
+ }
+
+ @Override
+ public TokenStreamComponents getReusableComponents(Analyzer analyzer,
String fieldName) {
+ if (analyzer instanceof AnalyzerWrapper wrapper) {
+ final Analyzer wrappedAnalyzer = wrapper.getWrappedAnalyzer(fieldName);
+ if
(wrappedAnalyzer.getReuseStrategy().getReusableComponents(wrappedAnalyzer,
fieldName)
+ == null) {
+ return null;
+ }
+ }
+ return reuseStrategy.getReusableComponents(analyzer, fieldName);
+ }
+
+ @Override
+ public void setReusableComponents(
+ Analyzer analyzer, String fieldName, TokenStreamComponents components)
{
+ reuseStrategy.setReusableComponents(analyzer, fieldName, components);
+ if (analyzer instanceof AnalyzerWrapper wrapper) {
+ final Analyzer wrappedAnalyzer = wrapper.getWrappedAnalyzer(fieldName);
+ wrappedAnalyzer
+ .getReuseStrategy()
+ .setReusableComponents(
+ wrappedAnalyzer, fieldName,
wrapper.getWrappedComponents(fieldName));
+ }
+ }
Review Comment:
I am not gonna lie, I have a very hard time understanding all this reusable
stuff. It seems like we are storing things twice?
The analyzer that is wrapped has a `storedValue` thread local field. That
`storedValue` is conditionally globally or to a hashmap that is per field. The
stored value can be anything really.
So, here we store the components of the analyzer. And, if the analyzer is a
wrapper, we will un-wrap it, look at a hashmap that isn't thread safe to get
the wrapped components.
But, aren't the components added to `setReusableComponents` already the
wrapped ones?
Are we trying to store for reuse both the wrapped components and the
unwrapped ones?
This is very confusing to me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
