Re: KeywordTokenizerFactory and stopwords

2011-06-08 Thread Matt Mitchell
Hi Erik. Yes something like what you describe would do the trick. I did find this: http://lucene.472066.n3.nabble.com/Concatenate-multiple-tokens-into-one-td1879611.html I might try the pattern replace filter with stopwords, even though that feels kinda clunky. Matt On Wed, Jun 8, 2011 at 11:04

Re: KeywordTokenizerFactory and stopwords

2011-06-08 Thread Erik Hatcher
This seems like it deserves some kind of "collecting" TokenFilter(Factory) that will slurp up all incoming tokens and glue them together with a space (and allow separator to be configurable). Hmmm surprised one of those doesn't already exist. With something like that you could have a stan

KeywordTokenizerFactory and stopwords

2011-06-08 Thread Matt Mitchell
Hi, I have an "autocomplete" fieldType that works really well, but because the KeywordTokenizerFactory (if I understand correctly) is emitting a single token, the stopword filter will not detect any stopwords. Anyone know of a way to strip out stopwords when using KeywordTokenizerFactory? I did tr