Re: WordDelimiterFilter looses position increments of tokens

2006-07-05 Thread Yonik Seeley
On 7/5/06, Eric Jain <[EMAIL PROTECTED]> wrote: Yonik Seeley wrote: > Something like this I suppose: Yes, just what I meant :-) Thanks a lot! The fix is committed (and new tests confirm it works ;-) -Yonik

Re: WordDelimiterFilter looses position increments of tokens

2006-07-05 Thread Eric Jain
Yonik Seeley wrote: Something like this I suppose: Yes, just what I meant :-) Thanks a lot!

Re: WordDelimiterFilter looses position increments of tokens

2006-07-05 Thread Yonik Seeley
On 7/5/06, Yonik Seeley <[EMAIL PROTECTED]> wrote: So fixing the first token at the end of next() and also at the other exit point (line 276) is probably the easiest fix. Something like this I suppose: Index: src/java/org/apache/solr/analysis/WordDelimiterFilter.java ==

Re: WordDelimiterFilter looses position increments of tokens

2006-07-05 Thread Yonik Seeley
On 7/5/06, Eric Jain <[EMAIL PROTECTED]> wrote: Yonik Seeley wrote: > Your'e right, it does look possible that position info can be lost. > The fix probably isn't as simple as copying the position increment in > newTok()... the original increment should only be copied for the first > token genera

Re: WordDelimiterFilter looses position increments of tokens

2006-07-04 Thread Eric Jain
Yonik Seeley wrote: Your'e right, it does look possible that position info can be lost. The fix probably isn't as simple as copying the position increment in newTok()... the original increment should only be copied for the first token generated. If something like for (int i = 1, max = queue.

Re: WordDelimiterFilter looses position increments of tokens

2006-07-04 Thread Yonik Seeley
On 7/4/06, Eric Jain <[EMAIL PROTECTED]> wrote: Just noticed that in the newTok method in the WordDelimiterFilter, the "position increment" of the token doesn't seem to be copied? Your'e right, it does look possible that position info can be lost. The fix probably isn't as simple as copying the

WordDelimiterFilter looses position increments of tokens

2006-07-04 Thread Eric Jain
Just noticed that in the newTok method in the WordDelimiterFilter, the "position increment" of the token doesn't seem to be copied?