>> I think your biggest problem is requesting 70,000 records from Solr. >> That is not going to be fast.
I know it, but the limits on the development don't lend themselves to putting all of the fields into lucene so a proper search can be conducted. We need to return them all because more work is done on the results webserver side (much to my chagrin) so paging is out of the question. >> 2. Since you are running out of memory parsing XML, I'm guessing >> that you're using a DOM-style parser. Don't do that. You do not >> need to create elaborate structures, strip mine the data, then >> throw those structures away. Instead, us a streaming parser, like Stax. Oh I know there are better ways of doing it, I just can't do any of them. constraints and all that. I was looking at the PythonResponseWriter, I'm trying to find a howto since a response writer would be responsible for writing the response after a search right? Best regards, Martin Owens
