I'm seeing a problem with the LengthFilter. It appears to work fine until I increase the max value above 254. At the point it stops removing the very large token from the stream. As a result I get the error: java.lang.IllegalArgumentException: Document contains at least one immense term...... UTF8 encoding is longer than the max length 32766
I'm certain I'm doing this wrong. Can someone please show me the light. :) <fieldType name="text_std" class="solr.TextField" positionIncrementGap="100"> <analyzer type="index"> <tokenizer class="solr.WhitespaceTokenizerFactory"/> <filter class="solr.LengthFilterFactory" min="1" max="254" /> </analyzer> </fieldType> Solr Version - 4.8.1 -Charles
