Re: FW: Difference Between Tokenizer and filter

2016-03-03 Thread Jack Krupansky
; From: G, Rajesh > Sent: Thursday, March 3, 2016 6:12 PM > To: 'solr-user@lucene.apache.org' > Subject: RE: FW: Difference Between Tokenizer and filter > > Thanks Shawn. This helps > > -Original Message----- > From: Shawn Heisey [mailto:apa...@elyograg.

RE: FW: Difference Between Tokenizer and filter

2016-03-03 Thread Vanlerberghe, Luc
pre-defined 'canonical' synonym and save both index and query time) Luc -Original Message- From: G, Rajesh [mailto:r...@cebglobal.com] Sent: donderdag 3 maart 2016 14:51 To: solr-user@lucene.apache.org Subject: RE: FW: Difference Between Tokenizer and filter Hi Shawn, One last q

RE: FW: Difference Between Tokenizer and filter

2016-03-03 Thread G, Rajesh
16 6:12 PM To: 'solr-user@lucene.apache.org' Subject: RE: FW: Difference Between Tokenizer and filter Thanks Shawn. This helps -Original Message- From: Shawn Heisey [mailto:apa...@elyograg.org] Sent: Wednesday, March 2, 2016 11:04 PM To: solr-user@lucene.apache.org Subject: Re: FW

RE: FW: Difference Between Tokenizer and filter

2016-03-03 Thread G, Rajesh
...@elyograg.org] Sent: Wednesday, March 2, 2016 11:04 PM To: solr-user@lucene.apache.org Subject: Re: FW: Difference Between Tokenizer and filter On 3/2/2016 9:55 AM, G, Rajesh wrote: > Thanks for your email Koji. Can you please explain what is the role of > tokenizer and filter so I can understand

Re: FW: Difference Between Tokenizer and filter

2016-03-02 Thread Shawn Heisey
On 3/2/2016 9:55 AM, G, Rajesh wrote: > Thanks for your email Koji. Can you please explain what is the role of > tokenizer and filter so I can understand why I should not have two tokenizer > in index and I should have at least one tokenizer in query? You can't have two tokenizers. It's not all

RE: FW: Difference Between Tokenizer and filter

2016-03-02 Thread G, Rajesh
Sekiguchi [mailto:koji.sekigu...@rondhuit.com] Sent: Wednesday, March 2, 2016 8:10 PM To: solr-user@lucene.apache.org Subject: Re: FW: Difference Between Tokenizer and filter Hi, ... must have one and only one and it can have zero or more s. From the point of view of the rules, your ... is not

Re: FW: Difference Between Tokenizer and filter

2016-03-02 Thread Koji Sekiguchi
Hi, ... must have one and only one and it can have zero or more s. From the point of view of the rules, your ... is not correct because it has more than one and ... is not correct as well because it has no . Koji On 2016/03/02 20:25, G, Rajesh wrote: Hi Team, Can you please clarify the bel

FW: Difference Between Tokenizer and filter

2016-03-02 Thread G, Rajesh
Hi Team, Can you please clarify the below. My understanding is tokenizer is used to say how the content should be indexed physically in file system. Filters are used to query result. The blow lines are from my setup. But I have seen eg that include filters inside and tokenizer in that confuse

Re: FW: Difference Between Tokenizer and filter

2016-03-02 Thread Emir Arnautovic
Hi Rajesh, Processing flow is same for both indexing and querying. What is compared at the end are resulting tokens. In general flow is: text -> char filter -> filtered text -> tokenizer -> tokens -> filter1 -> tokens ... -> filterN -> tokens. You can read more about analysis chain in Solr wi

FW: Difference Between Tokenizer and filter

2016-03-02 Thread G, Rajesh
Hi Team, Can you please clarify the below. My understanding is tokenizer is used to say how the content should be indexed physically in file system. Filters are used to query result. The blow lines are from my setup. But I have seen eg that include filters inside and tokenizer in that confuse