Hello Erick, Thank you so much for your help. That makes perfect sense. I will do the changes you suggest and let you know how it goes.
Thanks! On Mon, Sep 22, 2014 at 4:12 PM, Erick Erickson [via Lucene] < ml-node+s472066n4160547...@n3.nabble.com> wrote: > You have your index and query time analysis chains defined much > differently. Omitting the WordDelimiterFilterFactory from the > query-time analysis chain will lead to endless problems. > > With the definition you have, here are the terms in the index and > their term positions as below. This is available from the > admin/analysis page if you click the "verbose" checkbox, although I > admit it's kind of hard to read: > 1 2 3 4 > fatty acid-binding binding protein > acid > > But at query time, this is how they're being analyzed > 1 2 3 > fatty acid-binding protein > > So searching for "fatty acid-binding protein" requires that the tokens > "fatty" "acid-binding" and "protein" appear in term positions 1, 2, 3 > rather than where they actually are (1, 2, 4). Searching for "fatty > acid-binding protein"~1 would actually find this, the "~1" means allow > one gap in there. > > HOWEVER, that's the least of your problems. WordDelimiterFilterFactory > will _also_ "split on intra-word delimiters (all non alpha-numeric > characters)". While that doesn't really say so explicitly, that will > have the effect of removing puncutation. So searching for "fatty > acid-binding protein."~1 (note the period) will fail since the token > will include the period. > > I'd _really_ advise you to use the stock WordDelimiterFilterFactory > settings in both analysis and query times included in the stock Solr > release for, say, text_en_splitting or even a single analyzer like > text_en_splitting_tight. > > Best, > Erick > > On Mon, Sep 22, 2014 at 6:33 AM, aaguilar <[hidden email] > <http://user/SendEmail.jtp?type=node&node=4160547&i=0>> wrote: > > > Hello Erick. > > > > Below is the information you requested. Thanks for your help! > > > > <fieldType name="text_ws_finer" class="solr.TextField" > positionIncrementGap= > > "100"> <analyzer type="index"> <tokenizer class= > > "solr.WhitespaceTokenizerFactory"/> <filter class= > > "solr.WordDelimiterFilterFactory" splitOnNumerics="0" > splitOnCaseChange="0" > > generateWordParts="1" generateNumberParts="0" catenateWords="0" > > catenateNumbers="0" catenateAll="0" preserveOriginal="1"/> <filter > class= > > "solr.StopFilterFactory"/> <filter class="solr.LowerCaseFilterFactory"/> > </ > > analyzer> <analyzer type="query"> <tokenizer class= > > "solr.WhitespaceTokenizerFactory"/> <filter class= > > "solr.LowerCaseFilterFactory"/> </analyzer> </fieldType> > > > > > > <field name="description" type="text_ws_finer" indexed="true" > stored="true" > > /> > > > > On Fri, Sep 19, 2014 at 7:36 PM, Erick Erickson [via Lucene] < > > [hidden email] <http://user/SendEmail.jtp?type=node&node=4160547&i=1>> > wrote: > > > >> Hmmm, I'd have to see the schema definition for your description > >> field. For this, the admin/analysis page is very helpful. Here's my > >> guess: > >> > >> Your analysis chain doesn't break the incoming tokens up quite like > >> you think it is. Thus you have the tokens in your index like > >> 'protein,' (notice the comma) and 'protein-like' rather than just > >> 'protein'. However, I can't quite reconcile this with your statement: > >> "Another weird thing is that if I used description:"fatty > >> acid-binding" AND description:"protein" > >> > >> so I'm at something of a loss. If you paste in your schema definition > >> for the 'description' field _and_ the corresponding <fieldType> > >> definition I can give it a quick whirl. > >> > >> Best, > >> Erick > >> > >> On Fri, Sep 19, 2014 at 11:53 AM, aaguilar <[hidden email] > >> <http://user/SendEmail.jtp?type=node&node=4160122&i=0>> wrote: > >> > >> > Hello Erick, > >> > > >> > Thanks for the response. I tried adding the debug=True to the query, > >> but I > >> > do not know exactly what I am looking for in the output. Would it be > >> > possible for you to look at the results? I would really appreciate > it. > >> I > >> > attached two files, one of them is with the filter query > >> description:"fatty > >> > acid-binding" and the other is with the filter query > description:"fatty > >> > acid-binding protein". If you see the file that has the results for > >> > description:"fatty acid-binding" , you can see that the hits do have > >> "fatty > >> > acid-binding protein" and nothing in between. I really appreciate > any > >> help > >> > you can provide. > >> > > >> > Thanks you > >> > > >> > On Fri, Sep 19, 2014 at 2:03 PM, Erick Erickson [via Lucene] < > >> > [hidden email] <http://user/SendEmail.jtp?type=node&node=4160122&i=1>> > > >> wrote: > >> > > >> >> Your very best friend here is attaching &debug=query to the URL and > >> >> looking at the parsed query results. Upon occasion there's some > >> >> > >> >> One possible explanation is that description field has something > like > >> >> "fatty acid-binding some words protein" in which case your query > >> >> "fatty acid-binding protein" would fail, but "fatty acid-binding > >> >> protein"~4 would succeed. > >> >> > >> >> The other possibility is that your query parsing isn't quite doing > >> >> what you think, but adding &debug=query should help there. > >> >> > >> >> Best, > >> >> Erick > >> >> > >> >> On Fri, Sep 19, 2014 at 8:10 AM, aaguilar <[hidden email] > >> >> <http://user/SendEmail.jtp?type=node&node=4160036&i=0>> wrote: > >> >> > >> >> > Hello All, > >> >> > > >> >> > I recently came across a problem when I tried using > >> description:"fatty > >> >> > acid-binding protein" as a filter query when doing a query through > >> the > >> >> query > >> >> > interface for Solr in the Tomcat server. Using that filter query > did > >> >> not > >> >> > give me any results at all, however if I used description:"fatty > >> >> > acid-binding" as the filter query, it would give me the results I > >> >> wanted. > >> >> > > >> >> > The thing is that some of the results I got back from Solr, did > have > >> the > >> >> > words "fatty acid-binding protein" in the description field. So I > >> >> really do > >> >> > not know what might be causing the issue of Solr not being able to > >> find > >> >> > those hits. > >> >> > > >> >> > Another weird thing is that if I used description:"fatty > >> acid-binding" > >> >> AND > >> >> > description:"protein" as the filter query when doing a query, it > gave > >> me > >> >> the > >> >> > results I anticipated (with some extra results that did not have > the > >> >> exact > >> >> > phrase "fatty acid-binding protein"). Does anyone have an idea as > to > >> >> what > >> >> > might be happening? Just in case this is helpful, the version of > >> Solr > >> >> we > >> >> > are using is 4.0.0.2012.10.06.03.04.33. I appreciate any help > anyone > >> >> can > >> >> > provide. > >> >> > > >> >> > Thanks! > >> >> > > >> >> > > >> >> > > >> >> > -- > >> >> > View this message in context: > >> >> > >> > http://lucene.472066.n3.nabble.com/Issue-Adding-Filter-Query-tp4159990.html > >> >> > Sent from the Solr - User mailing list archive at Nabble.com. > >> >> > >> >> > >> >> ------------------------------ > >> >> If you reply to this email, your message will be added to the > >> discussion > >> >> below: > >> >> > >> >> > >> > http://lucene.472066.n3.nabble.com/Issue-Adding-Filter-Query-tp4159990p4160036.html > >> >> To unsubscribe from Issue Adding Filter Query, click here > >> >> < > >> >> . > >> >> NAML > >> >> < > >> > http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > > >> > >> >> > >> > > >> > > >> > fatty_acid-binding_protein.xml (1K) < > >> > http://lucene.472066.n3.nabble.com/attachment/4160048/0/fatty_acid-binding_protein.xml> > > >> > >> > fatty_acid-binding.xml (63K) < > >> > http://lucene.472066.n3.nabble.com/attachment/4160048/1/fatty_acid-binding.xml> > > >> > >> > > >> > > >> > > >> > > >> > -- > >> > View this message in context: > >> > http://lucene.472066.n3.nabble.com/Issue-Adding-Filter-Query-tp4159990p4160048.html > >> > Sent from the Solr - User mailing list archive at Nabble.com. > >> > >> > >> ------------------------------ > >> If you reply to this email, your message will be added to the > discussion > >> below: > >> > >> > http://lucene.472066.n3.nabble.com/Issue-Adding-Filter-Query-tp4159990p4160122.html > >> To unsubscribe from Issue Adding Filter Query, click here > >> < > >> . > >> NAML > >> < > http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > > >> > > > > > > > > > > -- > > View this message in context: > http://lucene.472066.n3.nabble.com/Issue-Adding-Filter-Query-tp4159990p4160423.html > > Sent from the Solr - User mailing list archive at Nabble.com. > > > ------------------------------ > If you reply to this email, your message will be added to the discussion > below: > > http://lucene.472066.n3.nabble.com/Issue-Adding-Filter-Query-tp4159990p4160547.html > To unsubscribe from Issue Adding Filter Query, click here > <http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4159990&code=QW50ZWxtby5BZ3VpbGFyLjE3QG5kLmVkdXw0MTU5OTkwfC0xMDkyNTg2ODY3> > . > NAML > <http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > -- View this message in context: http://lucene.472066.n3.nabble.com/Issue-Adding-Filter-Query-tp4159990p4160576.html Sent from the Solr - User mailing list archive at Nabble.com.
