David and I had a discussion about moving ascii-folding to come before stemming on English Wikipedia. It seemed like a good idea, but we decided we should run some tests before implementing it, just to be sure.
Turns out it is a good idea! Much more detail: https://www.mediawiki.org/wiki/User:TJones_(WMF)/Notes/Re-Ordering_Stemming_and_Ascii-Folding_on_English_Wikipedia We won't deploy it until we deploy BM25 later in the year, since it requires a full re-index of English Wikipedia, as does BM25. That's something we should only do once. —Trey Trey Jones Software Engineer, Discovery Wikimedia Foundation
_______________________________________________ discovery mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/discovery
