kamthorn commented on PR #16722:
URL: https://github.com/apache/lucene/pull/16722#issuecomment-5872895223
Hi @rmuir and everyone — thank you for merging the Thai analysis
improvements (#16717, #16718, #16720, #16722, #16727). Much appreciated.
I would like to ask about a possible **backport to `branch_10x`**.
These changes currently only exist on `main` (11.0.0, which requires Java
25). `branch_10x` still contains just the original 4 Thai files
(`ThaiAnalyzer`, `ThaiTokenizer`, `ThaiTokenizerFactory`, `package-info`), so
downstream consumers that cannot move to Java 25 yet get none of the new Thai
functionality.
The concrete case is **OpenSearch**: it builds against Lucene 10.5.1 and its
minimum JDK is still 21 (opensearch-project/OpenSearch#21174, the JDK 25 bump,
is open and facing a Bouncy Castle FIPS objection). So OpenSearch cannot
consume Lucene 11 yet, and consequently cannot use
`ThaiCharFilter`/`ThaiNormalizer`, the curated stopwords, `ThaiRepeatFilter`,
the `ThaiTokenizer` user dictionary, or the configurable buffer size. Thai
search quality there is stuck on the 10.x Thai code.
I noticed that new features *are* backported to `branch_10x` when
appropriate — e.g. #16680 ("Add type to compound word", a 10.6.0 New Feature)
was backported on 2026-09-28 (`0157353e`). So I wanted to ask whether the Thai
changes could be considered for a 10.x backport too.
I understand that backporting new features is a judgement call, so to keep
the ask narrow and reviewable, the highest-value item is probably **#16722**
(`user_dictionary` on `ThaiTokenizer`/`ThaiTokenizerFactory`): self-contained,
purely additive, no file-format or breaking public-API change. The rest are
similarly additive and analysis-only:
- #16717 — new `ThaiCharFilter` / `ThaiNormalizationFilter` /
`ThaiNormalizer` + `ThaiAnalyzer` wiring
- #16718 — curated Thai stopwords (data only)
- #16720 — new `ThaiRepeatFilter` / `ThaiRepeatFilterFactory`
- #16727 — configurable buffer size in `SegmentingTokenizerBase` +
whitespace safe boundary in `ThaiTokenizer` (this one touches a shared base
class, so possibly less suitable for a maintenance branch)
None of them change the index format, so they should be BWC-safe on 10.x.
I am happy to prepare the backport PR(s) against `branch_10x` myself if that
helps — just say the word, or let me know if you would rather handle it as
committers. And if the answer is simply "new features don't get backported",
that is completely fine; I mainly want to know so I can advise OpenSearch users
accurately.
Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
