threads get all jumbled together. (For instance, "Solr and
word frequencies?", "Solr 1.3 expected release date", and "Solr,
search result format" are all now mixed together in my client.)
Thanks!
-Mike
Fetch your 70,000 results in 70 chunks of 1000 results. Parse each chunk
and add it to your internal list.
If you are allowed to parse Python results, why can't you use a diffetent
XML parser?
What sort of "more work" are you doing? I've implemented lots of stuff
on top of a paged model, includin
>> I think your biggest problem is requesting 70,000 records from Solr.
>> That is not going to be fast.
I know it, but the limits on the development don't lend themselves to putting
all of the fields into lucene so a proper search can be conducted. We need to
return them all because more wor
Owens, Martin wrote:
Hello everyone,
I'm looking for a better solution that the current xml output we're currently
getting; if you return more than 70k records the webserver can no longer cope
with parsing the xml and the machine falls over out of memory.
Ideally what we'd like is for the sea
I think your biggest problem is requesting 70,000 records from Solr.
That is not going to be fast.
Two suggestions:
1. Use paging. Get the results in chunks, 10, 25, 100, whatever.
2. Since you are running out of memory parsing XML, I'm guessing
that you're using a DOM-style parser. Don't do tha
