Re: using CJKTokenizerFactory for Japanese language

2010-11-11 Thread Koji Sekiguchi
(10/11/12 1:49), Kumar Pandey wrote: I am exploring support for Japanese language in solr. Solr seems to provide CJKTokenizerFactory. How useful is this module? Has anyone been using this in production for Japanese language? CJKTokenizer is used in a lot of places in Japan. One shortfall it s

using CJKTokenizerFactory for Japanese language

2010-11-11 Thread Kumar Pandey
I am exploring support for Japanese language in solr. Solr seems to provide CJKTokenizerFactory. How useful is this module? Has anyone been using this in production for Japanese language? One shortfall it seems to have from what I have been able to read up on is that it can generate lot of false m