Copilot commented on code in PR #15830:
URL: https://github.com/apache/pinot/pull/15830#discussion_r2094393158
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/text/CaseAwareStandardAnalyzer.java:
##########
@@ -19,43 +19,60 @@
package org.apache.pinot.segment.local.segment.index.text;
import org.apache.lucene.analysis.CharArraySet;
+import org.apache.lucene.analysis.LowerCaseFilter;
import org.apache.lucene.analysis.StopFilter;
import org.apache.lucene.analysis.StopwordAnalyzerBase;
import org.apache.lucene.analysis.TokenStream;
import org.apache.lucene.analysis.standard.StandardTokenizer;
/**
- * A {@link org.apache.lucene.analysis.Analyzer} for case-sensitive text.
+ * A {@link org.apache.lucene.analysis.Analyzer} for standard text.
Review Comment:
[nitpick] Update the class Javadoc to explicitly describe this as a
"case-aware" analyzer, clarifying that it supports both case-sensitive and
case-insensitive modes.
```suggestion
* A {@link org.apache.lucene.analysis.Analyzer} for standard text that is
case-aware.
* This analyzer supports both case-sensitive and case-insensitive modes,
making it
* suitable for use cases where case sensitivity is configurable.
* <p>
```
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/store/TextIndexUtils.java:
##########
@@ -280,14 +284,12 @@ public static Analyzer
getStandardAnalyzerWithCustomizedStopWords(@Nullable List
if (stopWordsExclude != null) {
stopWordsExclude.forEach(stopWordSet::remove);
}
- if (isCaseSensitive) {
- return new CaseSensitiveAnalyzer(new CharArraySet(stopWordSet, false));
- }
- return new StandardAnalyzer(new CharArraySet(stopWordSet, true));
+ return new CaseAwareStandardAnalyzer(new CharArraySet(stopWordSet,
!isCaseSensitive), isCaseSensitive);
Review Comment:
[nitpick] Consider renaming `getStandardAnalyzerWithCustomizedStopWords` to
reflect that it now returns a `CaseAwareStandardAnalyzer`, improving API
clarity for callers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
