Re: Split words with period in between into separate tokens

2016-10-12 Thread Derek Poh
lkarni On Wed, Oct 12, 2016 at 6:24 AM, Derek Poh wrote: Hi How can I split words with period in between into separate tokens. Eg. "Co.Ltd" => "Co" "Ltd" . I am using StandardTokenizerFactory and it does notreplace periods (dots) that are not followed

Re: Split words with period in between into separate tokens

2016-10-12 Thread Georg Sorst
is not applicable. > Thank you. > > > On 10/12/2016 4:22 PM, Dheerendra Kulkarni wrote: > > You can use LetterTokenizerFactory instead. > > > > Regards, > > Dheerendra Kulkarni > > > > On Wed, Oct 12, 2016 at 6:24 AM, Derek Poh > wrote: > >

Re: Split words with period in between into separate tokens

2016-10-12 Thread Derek Poh
, Oct 12, 2016 at 6:24 AM, Derek Poh wrote: Hi How can I split words with period in between into separate tokens. Eg. "Co.Ltd" => "Co" "Ltd" . I am using StandardTokenizerFactory and it does notreplace periods (dots) that are not followed by whitespace are kep

Re: Split words with period in between into separate tokens

2016-10-12 Thread Dheerendra Kulkarni
You can use LetterTokenizerFactory instead. Regards, Dheerendra Kulkarni On Wed, Oct 12, 2016 at 6:24 AM, Derek Poh wrote: > Hi > > How can I split words with period in between into separate tokens. > Eg. "Co.Ltd" => "Co" "Ltd" . > > I am us

Split words with period in between into separate tokens

2016-10-11 Thread Derek Poh
Hi How can I split words with period in between into separate tokens. Eg. "Co.Ltd" => "Co" "Ltd" . I am using StandardTokenizerFactory and it does notreplace periods (dots) that are not followed by whitespace are kept as part of the token, including Internet