Traktormaster commented on a change in pull request #1123: LUCENE-9093: Unified
highlighter with word separator never gives context to the left
URL: https://github.com/apache/lucene-solr/pull/1123#discussion_r361663103
##########
File path:
lucene/highlighter/src/java/org/apache/lucene/search/uhighlight/LengthGoalBreakIterator.java
##########
@@ -174,7 +175,48 @@ private int moveToBreak(int idx) { // precondition: idx
is a known break
// called at start of new Passage given first word start offset
@Override
public int preceding(int offset) {
- return baseIter.preceding(offset); // no change needed
+ final int fragmentStart = Math.max(baseIter.preceding(offset), 0); //
convert DONE to 0
+ fragmentEndFromPreceding = baseIter.following(fragmentStart);
+ if (fragmentEndFromPreceding == DONE) {
+ fragmentEndFromPreceding = baseIter.last();
+ }
+ final int centerLength = fragmentEndFromPreceding - fragmentStart;
+ final int extraPrecedingLengthGoal = (int)((lengthGoal - centerLength) *
fragmentAlignment);
Review comment:
The change is the same in WORD mode too, but the match and the fragment of
the match is mostly the same. It's just not that apparent compared to the
SENTENCE mode.
I'll try and clarify all the expressions:
- Source text: `I'm looking for a beach ball for my vacation. I've lost the
old one. I liked that.`
- Query text: `vacation`
- Match term is `Match{start=36, end=44, text=vacation}` thus `matchsize` is
`8`
- Match-fragment is the fragment that contains the match according to the
underlying BI:
- WORD: `vacation` because it breaks on space and period.
- SENTENCE: `I'm looking for a beach ball for my vacation.` because it
breaks on period.
- `hl.bs.type=SENTENCE&hl.fragsize=10&hl.fragalign=0.0`:
- Match fragment `I'm looking for a beach ball for my vacation.` has 1
extra char on the right, but 10 is requested. Will try to append next fragments
to reach `match.end+((1-fragalign)*fragsize)` (which is `54`).
- Next break after `54` is the end of the second sentence so the result
snippet will be: `I'm looking for a beach ball for my vacation. I've lost the
old one.`
- `hl.bs.type=SENTENCE&hl.fragsize=10&hl.fragalign=0.5`:
- Match fragment `I'm looking for a beach ball for my vacation.` has 36
extra chars on the left, while 5 is requested. Also 5 are requested on the
right so the extension will happen just like in the previous example. The
result will be `I'm looking for a beach ball for my vacation. I've lost the old
one.`.
- `hl.bs.type=SENTENCE&hl.fragsize=10&hl.fragalign=1.0`:
- Match fragment `I'm looking for a beach ball for my vacation.` has 36
extra chars on the left, while 10 is requested. No extra chars are requested on
the right so the match fragment is the result.
As you can see the `fragsize` is **not** about the final size of the
snippet, but about the text around the match.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
