zhaih commented on code in PR #12885:
URL: https://github.com/apache/lucene/pull/12885#discussion_r1421011978
##########
lucene/analysis/kuromoji/src/java/org/apache/lucene/analysis/ja/JapaneseReadingFormFilter.java:
##########
@@ -43,10 +43,30 @@ public JapaneseReadingFormFilter(TokenStream input) {
this(input, false);
}
+ private boolean isAllHiragana(CharSequence s) {
+ int len = s.length();
+ for (int i = 0; i < len; i++) {
+ char ch = s.charAt(i);
+ if (ch < 0x3040 || ch > 0x3097) {
+ return false;
+ }
+ }
+ return true;
+ }
+
@Override
public boolean incrementToken() throws IOException {
if (input.incrementToken()) {
String reading = readingAttr.getReading();
+ if (reading == null & isAllHiragana(termAttr)) {
Review Comment:
What if we have hiragana and katagana mixed text? I feel like we still need
to convert the hiragana part to katagana?
##########
lucene/analysis/kuromoji/src/java/org/apache/lucene/analysis/ja/JapaneseReadingFormFilter.java:
##########
@@ -43,10 +43,30 @@ public JapaneseReadingFormFilter(TokenStream input) {
this(input, false);
}
+ private boolean isAllHiragana(CharSequence s) {
+ int len = s.length();
+ for (int i = 0; i < len; i++) {
+ char ch = s.charAt(i);
+ if (ch < 0x3040 || ch > 0x3097) {
+ return false;
+ }
+ }
+ return true;
+ }
+
@Override
public boolean incrementToken() throws IOException {
if (input.incrementToken()) {
String reading = readingAttr.getReading();
+ if (reading == null & isAllHiragana(termAttr)) {
Review Comment:
Maybe use `&&` instead?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
