[
https://issues.apache.org/jira/browse/LUCENE-9929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17321102#comment-17321102
]
Jan Høydahl commented on LUCENE-9929:
-------------------------------------
We could, but it would be a breaking change. The language rules are similar
enough to keep them together, and the rest of the normalizations all make
sense, i.e äæ->æ. I think some also index mixed Norwegian/Danish/Swedish
content into the same index and just want hits across various spellings without
further language analysis. That would be hard if we remove the Scandinavian
filter and add three new. Finally, Danish and Norwegian have exactly the same
rules (I believe) so they would anyway use exactly the same code.
So this is not one class with three distinct methods, it's the same flow, but
some of the foldings will now be conditional. I'll throw up a PR for
illustration.
[~cm] any insight?
> Make ScandinavianNormalizationFilter configurable wrt foldings
> --------------------------------------------------------------
>
> Key: LUCENE-9929
> URL: https://issues.apache.org/jira/browse/LUCENE-9929
> Project: Lucene - Core
> Issue Type: Improvement
> Components: modules/analysis
> Reporter: Jan Høydahl
> Assignee: Jan Høydahl
> Priority: Major
>
> The ScandinavianNormalizationFilter applies foldings for aa, ao, ae, oe and
> oo. But all those five do not make sense for both Norwegian, Swedish and
> Danish. Implement an optional configuration option where users can select
> which of them to apply. I.e. for Norwegian, a user would then configure (in
> Solr):
> {code:java}
> <filter class="solr.ScandinavianNormalizationFilterFactory
> foldings="ae,oe,aa"/>
> {code}
> This would activate foldings for ae->æ, oe->ø, aa->å, but not oo->o and ao->a.
> The default will be to activate all five as before, so it will be backward
> compatible.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
