autosuggest via solr.EdgeNGramFilterFactory (was: Re: wildcards in stopword list)

2010-02-03 Thread Lukas Kahwe Smith
Hi Ahmet, Well after some more testing I am now convinced that you rock :) I like the solution because its obviously way less hacky and more importantly I expect this to be a lot faster and less memory intensive, since instead of a facet prefix or terms search, I am doing an "equality" compariso

Re: wildcards in stopword list

2010-02-03 Thread Lukas Kahwe Smith
On 03.02.2010, at 14:34, Ahmet Arslan wrote: > >> Actually I plan to write a bigger blog post about the >> approach. In order to match the different fields I actually >> have a separate core with an index dedicated to auto suggest >> alone where I merge all fields together via some javascript >>

Re: wildcards in stopword list

2010-02-03 Thread Lukas Kahwe Smith
On 03.02.2010, at 15:19, Ahmet Arslan wrote: >>> With this field type, the query "ding" or "din" or >> "di" would return "Foo Bar Ding Dong". >> >> hmm wouldnt it return "foo bar ding dong" ? > > No, it will return original string. In this method you are not using faceting > anymore. You are

Re: wildcards in stopword list

2010-02-03 Thread Ahmet Arslan
> > With this field type, the query "ding" or "din" or > "di" would return "Foo Bar Ding Dong". > > hmm wouldnt it return "foo bar ding dong" ? No, it will return original string. In this method you are not using faceting anymore. You are just querying and requesting a field.

Re: wildcards in stopword list

2010-02-03 Thread Lukas Kahwe Smith
On 03.02.2010, at 14:34, Ahmet Arslan wrote: > >> Actually I plan to write a bigger blog post about the >> approach. In order to match the different fields I actually >> have a separate core with an index dedicated to auto suggest >> alone where I merge all fields together via some javascript >>

Re: wildcards in stopword list

2010-02-03 Thread Ahmet Arslan
> Actually I plan to write a bigger blog post about the > approach. In order to match the different fields I actually > have a separate core with an index dedicated to auto suggest > alone where I merge all fields together via some javascript > code: > > This way I can then use terms for a single

Re: wildcards in stopword list

2010-02-03 Thread Lukas Kahwe Smith
On 03.02.2010, at 13:54, Lukas Kahwe Smith wrote: > The issue is that I have multiple fields of data (names, address etc) that > should all be relevant for the auto suggest. Furthermore a "phrase" entered > can either match on one field or any combination of fields. Phrase in this > context me

Re: wildcards in stopword list

2010-02-03 Thread Lukas Kahwe Smith
On 03.02.2010, at 13:41, Ahmet Arslan wrote: >> this way i can do a prefix facet search for the term "foo" >> or "bar" and in both cases i can show the user "Foo Bar" >> with a bit of frontend logic to split off the "payload" aka >> original data. > > So you have a list of phrases (pre-extracted

Re: wildcards in stopword list

2010-02-03 Thread Ahmet Arslan
> this way i can do a prefix facet search for the term "foo" > or "bar" and in both cases i can show the user "Foo Bar" > with a bit of frontend logic to split off the "payload" aka > original data. So you have a list of phrases (pre-extracted) to be used for auto-suggest? Or you are using bi-gra

Re: wildcards in stopword list

2010-02-03 Thread Lukas Kahwe Smith
On 03.02.2010, at 13:07, Ahmet Arslan wrote: >> i am doing some funky hackery inside DIH via javascript to >> make my autosuggest work. i basically split phrases and >> store them together with the full phrase: >> >> the phrase: >> "Foo Bar" >> >> becomes: >> >> Foo Bar >> foo bar >> {foo}Foo_

Re: wildcards in stopword list

2010-02-03 Thread Ahmet Arslan
> I am wondering if there is some way to maintain a stopword > list with widcards: > > ignoring anything that starts with "foo": > foo* A custom TokenFilterFactory derived from StopFilterFactory can remove a token if it matches a java.util.regex.Pattern. List of patterns can be loaded from a fi