Again thanks. I'll take a stab at that are you aware of any resources/examples of how to do this? I figured I'd start with WhiteSpaceTokenizer but wasn't sure if there was a simpler place to start.
On Thu, Feb 9, 2012 at 8:44 PM, Robert Muir <rcm...@gmail.com> wrote: > On Thu, Feb 9, 2012 at 8:28 PM, Jamie Johnson <jej2...@gmail.com> wrote: >> Thanks Robert, I'll take a look there. Does it sound like I'm on the >> right the right track with what I'm implementing, in other words is a >> TokenFilter appropriate or is there something else that would be a >> better fit for what I've described? > > I can't say for sure to be honest... because its a bit too > abstract...I don't know the reasoning behind trying to convert > "abcdefghijk" to 1 2 3 4, and I'm not sure I really understand what > that means either. > > But in general: if you are taking the whole content of a field and > making it into tokens, then its best implemented as a tokenizer. > > -- > lucidimagination.com
