Traktormaster commented on a change in pull request #1123: LUCENE-9093: Unified
highlighter with word separator never gives context to the left
URL: https://github.com/apache/lucene-solr/pull/1123#discussion_r361840489
##########
File path:
lucene/highlighter/src/test/org/apache/lucene/search/uhighlight/LengthGoalBreakIteratorTest.java
##########
@@ -39,65 +41,142 @@
// 0 1
// 01234567890123456789
static final String CONTENT = "Aa bb. Cc dd. Ee ff";
+ static final String CONTENT2 = "Aa bb Cc dd X Ee ff Gg hh.";
+
+ public void testFragmentAlignmentConstructor() throws IOException {
+ BreakIterator baseBI = new CustomSeparatorBreakIterator('.');
+ // test fragmentAlignment validation
+ float[] valid_aligns = {0.f, 0.3333f, 0.5f, 0.99f, 1.f};
+ for (float alignment : valid_aligns) {
+ LengthGoalBreakIterator.createClosestToLength(baseBI, 50, alignment);
+ }
+ float[] invalid_aligns = {-0.01f, -1.f, 1.5f, Float.NaN,
Float.NEGATIVE_INFINITY, Float.POSITIVE_INFINITY};
+ for (float alignment : invalid_aligns) {
+ expectThrows(IllegalArgumentException.class, () -> {
+ LengthGoalBreakIterator.createClosestToLength(baseBI, 50, alignment);
+ });
+ }
+ // test backwards compatibility constructors
+ String backwardCompString =
LengthGoalBreakIterator.createClosestToLength(baseBI, 50).toString();
+ assertTrue(backwardCompString,
backwardCompString.contains("fragAlign=0.0"));
+ backwardCompString = LengthGoalBreakIterator.createMinLength(baseBI,
50).toString();
+ assertTrue(backwardCompString,
backwardCompString.contains("fragAlign=0.0"));
+ }
public void testTargetLen() throws IOException {
// "goal" means target length goal to find closest break
// at first word:
Query query = query("aa");
- assertEquals("almost two sent",
- "<b>Aa</b> bb.", highlightClosestToLen(CONTENT, query, 9));
Review comment:
I've already thought about trying that, but there's one case I think could
break and I didn't see it worthwhile to investigate further.
Since the index tokenizer and the highlight separator may work on completely
different characters, the separator could break inside the match if I allow it
to process the chars of the match itself. In that case with very low `fragsize`
or very uneven `fragAlignRatio` a break inside the match could be accepted and
the [first
assertion](https://github.com/apache/lucene-solr/blob/3ae1a0b3bad84cdfaa3941b87a1a7fcad63a66d4/lucene/highlighter/src/java/org/apache/lucene/search/uhighlight/Passage.java#L45)
in `Passage.addMatch` would fail.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
