Solr Multiword Synonym Problem
Hi all,
I am trying to solve the solr multiword synonym issue at our installation, I
am currently using SOLR-4.9.x version. I used the
"com.lucidworks.analysis.AutoPhrasingTokenFilterFactory" from Lucidworks git
repo and used this in my schema.xml and also used their
"com.lucidworks.analysis.AutoPhrasingQParserPlugin" in the solrconfig.xml.
To make testing easier for the solr community, i used the autophrases.txt as
below.
big apple
new york city
city of new york
new york new york
new york ny
ny city
ny ny
new york
When i run a query for "big+apple" my parsedQuery converts perfectly.
"parsedquery":"(+DisjunctionMaxQuery((searchField:big_apple)))/no_coord",
"parsedquery_toString":"+(searchField:big_apple)",
..
but when i search for new+york+city, it converts to
"parsedquery":"(+(DisjunctionMaxQuery((searchField:new_york_city))
DisjunctionMaxQuery((searchField:city/no_coord",
"parsedquery_toString":"+((searchField:new_york_city)
(searchField:city))",
"explain":{},
Why is it trying to parse the word "city" separately. I thought when it
finds an exact match "new york city" in the auto phrases.txt it should just
replace the white space with underscore ( which is what i choose) in my
solrconfig.
But if i comment out the following in my autophrases.txt
#city of new york
it works, fine, it doesn't perform a DisjunctionMaxQuery on "city".
Same with "New york Ny", since there is an entry in auto phrases.txt
beginning with Ny , its searching for NY as well.
Its like an overlap causing this problem.
Did anybody face this problem, if so could you please throw some light on
how you solved this? I used the branch from git for lucid works, that was
10 months old.
Any help is highly appreciated.
this is my solrconfig.xml
--
autophrases.txt
_
edismax
explicit
10
text
autophrasingParser
--
This is my setting from schema.xml
--
--
thanks
SolrUser
--
View this message in context:
http://lucene.472066.n3.nabble.com/Solr-Multiword-Synonym-Problem-tp4204979.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Geodist
hi Eric, thank you for the response. We use this product , which comes with solr, http://www.lucidimagination.com/. They have a UI on top of solr, its easy to add fields or edit their properties using their UI. The queries I write are using the solr which comes with lucid imagination , it says its SOLR version as below. Solr Specification Version: 4.0.0.2011.03.21.13.41.34. Solr Implementation Version: 4.0-SNAPSHOT exported - markrmiller - 2011-03-21 13:41:34 Lucene Specification Version: 4.0-SNAPSHOT Lucene Implementation Version: 4.0-SNAPSHOT exported - 2011-03-21 13:42:35 I am assuming this is SOLR versin 4.0. I haven't downloaded just solr, and run my queries, i was using Lucidimagination's solr. Also is it possible to return the distance, in the below scenario, we dont want to perform a lat long search. Suppose the user types in "Jacksonville", "TN", can we run the query and return all the search results, and show the distance from Jacksonville TN on all search results. I mean we know the lat and long , but we dont want to use that and perform a radial search. We just want to perform a city, state search and return distance. I tried passing in fl=geodist(lat, long,coordinates_0_coordinate,coordinates_1_coordinate),score,name,street...etc, where the coordinates_0 and coordinates_1 are the lat and long of all the records in the solr index. This is not returning the distance. Can you please let me know if you tried this option? thanks SN -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Geodist-tp3287005p3293551.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Geodist
Eric, thanks for the update, I thought solr 4.0 should have the pseudo columns and i am using the right version. So did you ever worked on a query to return distance, where there is no long, lat are used in the where clause. I mean not in a radial search, but a city search, but displayed the distance. So my thought was to pass in the long and lat to the geodist and also the coordinates(long and lat) of every record, and let geodist compute the distance. Can you please let me know if this worked for you? thanks SN -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Geodist-tp3287005p3293779.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Geodist
hi Eric, thank you for the tip, i will try that option. Where can i find a document that shows details of geodist arguments, when i google, i did not find one. so this is what my query is like. I want the distance to be returned. i dont now exactly what all to pass to geodist, as i couldnt find a proper document. http://localhost:/solr/apex_dev/select/?q=city:Quincy&fl=city,state,coordinates,score,geodist(39.9435,-120.9226). So i want to pass in the long and lat of Quincy and then i want all the records which are tagged with Quincy should be returned ( as i am doing a q=city:Qunicy search) and also distance to be displayed. Can you please let me know what i should pass into goedist(), in this scenario. thanks SN -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Geodist-tp3287005p3295606.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Geodist
Eric, thank you for the quick update, so in the below query you sent to me, i can also add any conditions right? i mean city:Boston and state:MA...etc , can i also use "dismax" query syntax? The confusion from the beginning seems to be the version of solr i was trying and the one you are trying. Looks like the latest trunc of solr has the geodist and the one i am using is not returning geodist. &q=*:*&sfield=store&pt=45.15,-93.85&fl=name,store,geodist() thanks SN -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Geodist-tp3287005p3295868.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Geodist
Eric, can you please let me know the solr build, that you are using. I went to this below site, but i want to use the same build, you are using, so i can make sure the queries work. http://wiki.apache.org/solr/FrontPage#solr_development thanks SN -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Geodist-tp3287005p3296210.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Geodist
I think i found the link to the nightly build, i am going to try this flavor of solr and run the query and check what happens. The link i am using is https://builds.apache.org/job/Solr-trunk/lastSuccessfulBuild/artifact/artifacts/ thanks SN -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Geodist-tp3287005p3296316.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Geodist
hi Erik, today i had the distance working. Since the solr version under LucidImagination is not returning geodist(), I downloaded Solr 4.0 from the nightly build. On lucid we had the full schema defined. So i copied that schema to the "example" directory of solr-4 and removed all references to Lucid and started the index. I wanted to try our schema under solr-4. Then i had the data indexed ( we have a rake written in ruby to index the contents) and ran the geodist queries and they all run like a charm. I do get distance as a pseudo column. Is there any documentation that gives me all the arguments of geodist(), i couldnt find it online. Erick, thanks for your help in going through my examples. NOw they all work on my solr installation. thanks SN -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Geodist-tp3287005p3297088.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Geodist
hi Lance, thanks for the link, i went to their site, lucidimagination forum, when i searched on geodist, i see my own posts. Is this forum part of lucidimagination? Just curious. thanks SN -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Geodist-tp3287005p3297262.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Geodist
ok, thank you Erick, i will check this forum as well. thanks SN -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Geodist-tp3287005p3300236.html Sent from the Solr - User mailing list archive at Nabble.com.
Solr Geodist
Hi, i am trying to return distance in the solr query, by passing in the
"fl=geodist()" and i dont see distance being returned.
We have a field called coordinates which is configured as latlong and when i
perform the following search, i do see results
q=*:*&fq={!geofilt}&sfield=coordinates&pt=31.2225,-85.3931&fl=id,name =>
this works fine
q=*:*&fq={!geofilt}&sfield=coordinates&pt=31.2225,-85.3931&fl=id,name,geodist()
=> this doesnt return distance
But when i add geodist() to the "fl parameter", i dont see the pseudo column
being returned. It returns the "id" and "name" only, but not the distance.
WE are using Solr 4.0. Can anybody please let me know what is wrong with the
query? I also tried passing in the lat , long as arguments to the geodist,
but doesnt help.
I am totally lost in the solr system, can anybody please help?
thanks
SolrNovice
--
View this message in context:
http://lucene.472066.n3.nabble.com/Solr-Geodist-tp3287005p3287005.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Geodist
Eric, thanks for the quick response. I left out the "d" value, yes, when you
perform a spatial query, we should have a distance of d>0, sorry about that.
What is the setting of your "store" value, i mean in the schema, was it
marked at LatLong. For some reason i dont see the geodist() being returned
in the result set. my coordinates is setup as "type=location", below is the
snapshot from my schema.xml.
We are using LucidImagination, so i guess it comes with Solr 4.0, please let
me know if i am wrong. That may be the reason for geodist() not being
returned.I checked the solr version by going to solr admin and checked the
version. it shows 4.0.
For now i found a work around, this works for me. the distance is returned
in the form of "score".
http://127.0.0.1:/solr/apex_dev/select/?q=*:*+_val_:%22geodist%28%29%22&rows=100&fq={!geofilt}&sfield=coordinates&pt=31.2225,-85.3931&d=50&sort=geodist%28%29%20asc&fl=*,score
I read in a different post that , earlier versions of solr ( prior to 4.0),
we have to use the score option.
thanks for taking time to try the query.
SN
--
View this message in context:
http://lucene.472066.n3.nabble.com/Solr-Geodist-tp3287005p3287806.html
Sent from the Solr - User mailing list archive at Nabble.com.
