Multiple groups of boolean queries in a single query.
Hello friends, I have a schema which contains various types of records of three different categories for ease of management and for making a single query to fetch all the data. The fields are grouped into three different types of records. For example: fields type 1: fields type 2: fields type 3: common partition field which identifies the category of the data record What should I do to fetch all these records in the form: (+x_date:[2011-01-01T00:00:00Z TO *] +x_type:(1 OR 2 OR 3 OR 4) +xyz_category:X) OR (+y_date:[2012-06-01T00:00:00Z TO *] +y_name:sam~ +xyz_category:Y) OR (+z_date:[2013-03-01T00:00:00Z TO *] +xyz_category:Z) Can we construct a query like this? Or is it even possible? Sam -- View this message in context: http://lucene.472066.n3.nabble.com/Multiple-groups-of-boolean-queries-in-a-single-query-tp4074294.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Multiple groups of boolean queries in a single query.
My entire concern is to be able to make a single query to fetch all the types of records. If I had to create three different cores for this different types of data, I would have to make 3 calls to solr to fetch the entire set of data. And I will be having approx 15 such types in real. Also, at any given record, either the section 1 fields are filled up or section 2's or section 3's. At no point, will we have all these fields populated in a single record. Only field that will have data for all records is xyz_category to allow us to partition the data set. Any suggestions in writing a single query to fetch all the data we need will be highly appreciated. Thanks. -- View this message in context: http://lucene.472066.n3.nabble.com/Multiple-groups-of-boolean-queries-in-a-single-query-tp4074294p4074296.html Sent from the Solr - User mailing list archive at Nabble.com.
Is cache useful for my scenario?
Hi, I am new in Solr and wish to use version 4.2.x for my app in production. I want to show hundreds and thousands of markers on a map with contents coming from Solr. As the user moves around the map and pans, the browser will fetch data/markers using a BBOX filter (based on the maps' viewport boundary). There will be a lot of data that will be indexed in Solr. My question is, does caching help in my case? As the filter queries will vary for almost all users ( because the viewport latitude/longitude would vary), in what ways can I use Caching to increase performance. Should I completely turn off caching? If you can suggest by your experience, it would be really nice. Thanks Sam -- View this message in context: http://lucene.472066.n3.nabble.com/Is-cache-useful-for-my-scenario-tp4056250.html Sent from the Solr - User mailing list archive at Nabble.com.
Best way to design a "story and comments" schema.
Hi, I wish to know how to best design a schema to store comments in stories / articles posted. I have a set of fields: // Users can post their comments on a post and I should be able to retrieve these comments and show it along side the original post. I only need to show the last 3 comments and show a facet of the remaining comments which user can click and see the rest of the comments ( something like facebook does ). One alternative, I could think of, was adding a dynamic field for all comments : / / So, to store each comments, I would send a text to solr of the form -> For Field Name: /comment_n/ Value:/[Commenter Name]:[Commenter ID]:[Actual Comment Text]/ And to keep the count of those comments, I could use another field like so :/ / With this approach, I will have to do some calculation when a comment is deleted by the user but I still can manage to show the comments right. My idea is to find the best solution for this scenario which will be fast and also be simple. Kindly suggest. -- View this message in context: http://lucene.472066.n3.nabble.com/Best-way-to-design-a-story-and-comments-schema-tp4062867.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Best way to design a "story and comments" schema.
Thanks for your reply. I generally get confused by a collection and a core. But just FYI, I do have two cores at the moment - one for the users and another for the Stories. Initially I thought of adding an extra core for the Comments too but realized that it would mean multiple HTTP calls to fetch both the story and the comments. Also, when a story is deleted, so should be its comments. Having that spread across two cores might cause issues with transaction when I delete the story and try to delete the respective comments? Or when I delete the User and all hos stories and comments? I really wish to understand how that works. Sam -- View this message in context: http://lucene.472066.n3.nabble.com/Best-way-to-design-a-story-and-comments-schema-tp4062867p4062913.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Best way to design a "story and comments" schema.
I think I got your point. So, what I will create are three cores (or collections) - one for the users, one for the stories and the last one for comments. When I need to find all the stories posted by a single user, I first need to search the stories core with a unique userid in the filter and then run another query to fetch the collection of comments. Correct? Also, I have no such requirement to search through the comments and its mostly a storage filed for me. So, do you think I should shift that into a DB from where I may query the comments? Or will it be too costly for Solr to just plain store that data in a core? Which would be the best option here? Also, the idea of custom search component sounds great. But as you said, I will first try this out with a simple possible setup and then go from there. -- View this message in context: http://lucene.472066.n3.nabble.com/Best-way-to-design-a-story-and-comments-schema-tp4062867p4062929.html Sent from the Solr - User mailing list archive at Nabble.com.
Solr Faceting doesn't return values.
Hello,
I have a field defined in my schema.xml like so:
string is a type :
When I run the query for faceting data by the city:
http://XX.XX.XX.XX/solr/collection1/select?q=mm_state_code&wt=json&indent=true&facet=true&facet.field=sa_site_city
I get empty result like so:
{
"responseHeader":{
"status":0,
"QTime":1,
"params":{
"facet":"true",
"indent":"true",
"q":"mm_state_code",
"facet.field":"sa_site_city",
"wt":"json"}},
"response":{"numFound":0,"start":0,"docs":[]
},
"facet_counts":{
"facet_queries":{},
"facet_fields":{
"sa_site_city":[]},
"facet_dates":{},
"facet_ranges":{}}}
I wonder what am I doing wrong?
--
View this message in context:
http://lucene.472066.n3.nabble.com/Solr-Faceting-doesn-t-return-values-tp4065276.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Faceting doesn't return values.
Ok after I added debug=all to the query, I get:
{
"responseHeader":{
"status":0,
"QTime":11,
"params":{
"facet":"true",
"indent":"true",
"q":"mm_state_code",
"debug":"all",
"facet.field":"sa_site_city",
"wt":"json"}},
"response":{"numFound":0,"start":0,"docs":[]
},
"facet_counts":{
"facet_queries":{},
"facet_fields":{
"sa_site_city":[]},
"facet_dates":{},
"facet_ranges":{}},
"debug":{
"rawquerystring":"mm_state_code",
"querystring":"mm_state_code",
"parsedquery":"sa_property_id:mm_state_code",
"parsedquery_toString":"sa_property_id:mm_state_code",
"explain":{},
"QParser":"LuceneQParser",
"timing":{
"time":4.0,
"prepare":{
"time":2.0,
"query":{
"time":0.0},
"facet":{
"time":0.0},
"mlt":{
"time":0.0},
"highlight":{
"time":0.0},
"stats":{
"time":0.0},
"debug":{
"time":0.0}},
"process":{
"time":1.0,
"query":{
"time":0.0},
"facet":{
"time":0.0},
"mlt":{
"time":0.0},
"highlight":{
"time":0.0},
"stats":{
"time":0.0},
"debug":{
"time":1.0}
I have not defined any default facet filed, in the handler in solrconfig.xml
file.
Also, there is plenty of data available and the field sa_site_city
What I am trying to understand is this:
"parsedquery":"sa_property_id:mm_state_code"
I have a field sa_property_id in the schema but i have not defined it in the
query nor in solrconfig.xml, but why is it still evaluated? Any help in
solving this problem will be greatly appreciated.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Solr-Faceting-doesn-t-return-values-tp4065276p4065294.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Faceting doesn't return values.
Ok my bad. I do have a default field defined in the /select handler in the config file. explicit 10 sa_property_id But then how do I change my query now? -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Faceting-doesn-t-return-values-tp4065276p4065298.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Faceting doesn't return values.
Thanks for your reply. I have my request url modified like this: http://xx.xx.xx.xx/solr/collection1/select?q=TX&df=mm_state_code&wt=xml&indent=true&facet=true&facet.field=sa_site_city&debug=all Facet Filed = sa_site_city ( city wise facet) Default Filed = mm_state_code Query= TX When I run this query, I get something like this: 0 3 true sa_site_city true TX 1369238921109 all sa_site_city xml TX TX sa_site_city:TX sa_site_city:TX LuceneQParser 2.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.0 1.0 1.0 0.0 0.0 0.0 0.0 I do have the data in my index and that I verified by running other queries. I can't figure out what I am missing. -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Faceting-doesn-t-return-values-tp4065276p4065360.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Faceting doesn't return values.
When I use your query, I get :
400
12
true
mm_state_code
true
*mm_state_code:(**TX)*
1369244078714
all
sa_site_city
xml
org.apache.solr.search.SyntaxError: Cannot parse
'*mm_state_code:(**TX)*': Encountered " ":" ": "" at line 1, column 14.
Was expecting one of:
...
...
...
"+" ...
"-" ...
...
"(" ...
"*" ...
"^" ...
...
...
...
...
...
...
"[" ...
"{" ...
...
...
400
Not sure why the data wont show up. Almost all the records has the field
sa_site_city has data and is also indexed. :(
--
View this message in context:
http://lucene.472066.n3.nabble.com/Solr-Faceting-doesn-t-return-values-tp4065276p4065406.html
Sent from the Solr - User mailing list archive at Nabble.com.
RE: filter query from external list of Solr unique IDs
Does anything exists already in solr 4.3 to meet this usecase scenario? -- View this message in context: http://lucene.472066.n3.nabble.com/filter-query-from-external-list-of-Solr-unique-IDs-tp1709060p4070874.html Sent from the Solr - User mailing list archive at Nabble.com.
Solr - Spatial Search for Specif Areas on Map
Hi, I am new to Solr Spatial Search and would like to understand if Solr can be used successfully for very large data sets in the range of 4Billion records. I need to search some filtered data based on a region - maybe a set of lat/lons or polygon area. is that possible in solr? How fast is it with such data size? Will it be able to handle the load for 1 req/sec? If so, how? Do you think solr can beat the performance of PostGIS? As I am about to choose the right technology for my new project, I need some expert comments from the community. Regards Sam -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Spatial-Search-for-Specif-Areas-on-Map-tp3995051.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr - Spatial Search for Specif Areas on Map
David, Thanks for such a detailed response. The data volume I mentioned is the total set of records we have - but we would never ever need to search the entire base in one query; we would divide the data by region or zip code. So, in that case I assume that for a single region, we would not have more than 200M records (this is real , we have a region with that many records). So, I can assume that I can create shards based on regions and the requests would get distributed among these region servers, right? You also mentioned about ~20 concurrent queries per shard - do you have links to some benchmarks? I am very interested to know about the hardware sizing details for such a setup. About setting up Solr for a single shard, I think I will go by your advice. Will see how much a single shard can handle in a decent machine :) The reason why I came up with that figure was, I have a user base of 500k and theres a lot of activity which would happen on the map - every time someone moves the tiles, zooms in/out, scrolls, we are going to send a server side request to fetch some data ( I agree we can benefit much using caching but I believe Solr itself has its own local cache). I might be a bit unrealistic with my 10K rps projections but I have read about 9K rps to map servers from some sources on the internet. And, NO, I don't work for Google :) But who knows we might be building something that can get so much traffic to us in a while. :D BTW, my question still remains - can we do search on polygonal areas on the map? If so, do you have any link where i can get more details? Bounding Box thing wont work for me I guess :( Sam -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Spatial-Search-for-Specif-Areas-on-Map-tp3995051p3995209.html Sent from the Solr - User mailing list archive at Nabble.com.
