StandardTokenizer is .... tricky.
it does a lot of kooky things that probably made sense when it was
written, you'll not in your output that the "term type" is getting set to
"HOST" Standard Tokenizer has decided that L.I.C looks like a hostname, so
it's not splitting on the periods.
: analyser output. Sohould I use WhiteSpaceTokenizerFactory with the
: StandardFilterFactory ?
your milage may vary ... but based on what little i know about your use
case WhiteSpaceTokenizerFactory with a WordDelimiterFilterFactory would
probably work.
: org.apache.solr.analysis.StandardTokenizerFactory {}
: term position 1
: term text L.I.C
: term type <HOST>
: source start,end 0,5
: payload
-Hoss
