Re: Getting a large number of documents by id

2013-07-18 Thread Brian Hurt
Thanks everyone for the response. On Thu, Jul 18, 2013 at 11:22 AM, Alexandre Rafalovitch wrote: > You could start from doing id:(12345 23456) to reduce the query length and > possibly speed up parsing. > I didn't know about this syntax- it looks useful. > You could also move the query from 'q

Re: Getting a large number of documents by id

2013-07-18 Thread Alexandre Rafalovitch
And I guess, if only a subset of fields is being requested but there are other large fields present, there could be the cost of loading those extra fields into memory before discarding them. In which case, using enableLazyFieldLoading may help. Regards, Alex. Personal website: http://www.outer

Re: Getting a large number of documents by id

2013-07-18 Thread Roman Chyla
Look at speed of reading the data - likely, it takes long time to assemble a big response, especially if there are many long fields - you may want to try SSD disks, if you have that option. Also, to gain better understanding: Start your solr, start jvisualvm and attach to your running solr. Start

Re: Getting a large number of documents by id

2013-07-18 Thread Michael Della Bitta
Brian, Have you tried the realtime get handler? It supports multiple documents. http://wiki.apache.org/solr/RealTimeGet Michael Della Bitta Applications Developer o: +1 646 532 3062 | c: +1 917 477 7906 appinions inc. “The Science of Influence Marketing” 18 East 41st Street New York, NY 1

Re: Getting a large number of documents by id

2013-07-18 Thread Jack Krupansky
separated list of document IDs (keys). Check it out: http://wiki.apache.org/solr/RealTimeGet -- Jack Krupansky -Original Message- From: Brian Hurt Sent: Thursday, July 18, 2013 10:46 AM To: solr-user@lucene.apache.org Subject: Getting a large number of documents by id I have a situation

Re: Getting a large number of documents by id

2013-07-18 Thread Alexandre Rafalovitch
You could start from doing id:(12345 23456) to reduce the query length and possibly speed up parsing. You could also move the query from 'q' parameter to 'fq' parameter, since you probably don't care about ranking ('fq' does not rank). If these are unique every time, you could probably look at not

Getting a large number of documents by id

2013-07-18 Thread Brian Hurt
I have a situation which is common in our current use case, where I need to get a large number (many hundreds) of documents by id. What I'm doing currently is creating a large query of the form "id:12345 OR id:23456 OR ..." and sending it off. Unfortunately, this query is taking a long time, espe