Re: How to effectively search inside fields that should be indexed with changing them.

2007-12-14 Thread Brian Carmalt
Hello Otis, The example I provided was a simplified one. The real usecase is that will have to dynamically adapt to field values, from which we have no idea what form they will have.. So unfortunately, a custom tokenizer will not work. I changed the n-gram values to min=max= 2 and I can match s

Re: How to effectively search inside fields that should be indexed with changing them.

2007-12-11 Thread Otis Gospodnetic
Brian, This is not really a job for n-grams. It sounds like you'll want to write a custom Tokenizer that has knowledge about this particular pattern, knows how to split input like the one in your example, and produce multiple tokens out of it. For the natural language part you can probably ge