zhaih commented on a change in pull request #157:
URL: https://github.com/apache/lucene/pull/157#discussion_r644172488
##########
File path:
lucene/analysis/common/src/java/org/apache/lucene/analysis/core/FlattenGraphFilter.java
##########
@@ -255,6 +260,32 @@ private boolean releaseBufferedToken() {
return false;
}
+ /**
+ * Free output nodes before the given outputs. Free inputs nodes before the
minimum input node for
+ * this output.
+ *
+ * @param output target output node
+ */
+ private void freeBefore(OutputNode output) {
+ // We've released all of the tokens that end at the current output,
+ // so free all output nodes before this. Input nodes are more complex.
+ // The second shingled tokens with alternate paths can appear later in the
output graph than
+ // than some of their alternate path tokens.
+ // Because of this case we can only free from the minimum because the
minimum node will have
+ // come from before the second shingled token.
+ // This means we have to hold onto input nodes who's tokens get stacked on
previous nodes until
+ // we've completely passed those inputs.
+ // Related tests testShingledGap, testShingledGapWithHoles
+ outputFrom++;
+ int freeBefore = Collections.min(output.inputNodes);
+ // This will catch a node being freed early if it's input to the next
output.
+ // Could a freed early node be input to a later output?
+ assert outputNodes.get(outputFrom).inputNodes.stream().filter(n ->
freeBefore < n).count() > 0
+ : "FreeBefore " + output.inputNodes.get(0) + " will free in use nodes";
Review comment:
Isn't this still the old assertion that need to be changed?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
