aronnopanen-veeva opened a new pull request, #16913: URL: https://github.com/apache/lucene/pull/16913
### Problem Description When using: * hl.method=unified * hl.weightMatches=true * qs>0 * `PhraseQuery` A query where: * Query slop is required for the match to succeed * The query phrase contains a particular term twice * The document being matched contains that same term a _third_ time, beyond the match boundary ...can result in the associated highlight snippet extending beyond the proper end point. E.g.: * Document text: "The badger dug a burrow quietly near the badger that lived by the creek before sunrise. Weeks later, hikers spotted a different badger crossing the same field at dusk." * Search phrase: "The badger dug a burrow quietly near the badger that lived by the creek" * Returned highlight snippet (highlight in <strong>bold</strong>): "The <strong>badger dug a burrow quietly near the badger that lived by the creek before sunrise. Weeks later, hikers spotted a different badger</strong> crossing the same field at dusk." ### Root Cause Analysis When a phrase query's search text contains a word that repeats within the phrase itself (e.g. "a b a"), SloppyPhraseMatcher creates a separate `PhrasePositions` per occurrence, each independently walking that term's postings. When slop forces the matcher to search ahead for a valid combination, `advanceRpts()` can resolve a position collision between two such repeat `PhrasePositions` by permanently advancing one of them - even though it isn't the one driving minimization - past the document occurrence that was actually part of the match, into a later occurrence of the same term elsewhere in the document (e.g. a 3rd occurrence of the word, located after the matched text). `startOffset()`/`endOffset()`/`startPosition()`/`endPosition()` read that sibling's live (now-corrupted) position instead of a value frozen at match-detection time, so the reported match - and anything built from it, such as `UnifiedHighlighter` snippets - extends past the text that actually matched. ### Fix Description Each `PhrasePositions` now remembers its position/offsets from just before its most recent advance. A `bumped` flag, set when `advanceRpts()` forcibly advances a repeat-group sibling and cleared at the top of each `nextMatch()` and on every deliberate main-loop advance, tells the offset/ position accessors to use that prior (correct) value instead of the live (corrupted) one for that specific `PhrasePositions`. All of the new bookkeeping is gated behind `captureLeadMatch`, which is only true on the Matches/highlighting path (never during plain scoring), so this has no effect on search relevance scoring; verified scoring is unchanged on `TestSloppyPhraseQuery`'s fixtures before/after this change. Updates `TestMatchesIterator#testSloppyPhraseQueryWithRepeats`, whose expected offsets for the second reported match were (silently) encoding this same bug, and adds a focused regression test to `TestSloppyPhraseQuery` reproducing the originally-reported case. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
