On 6/15/2014 2:54 AM, Floyd Wu wrote: > Thank you Alex. > I'm doing commit every 100 fiels. > Maybe there is a better way to do this job, something like DIH(possible?) > Sometimes i have much bigger xml file (2MB) and post to SOLR(jetty enabled) > may encounter slow or exceed limitation.
If you are getting OOM exceptions on your Solr server, then you need to increase your Java heap size. I have never seen the "pending too many" error that you mentioned at the beginning of the thread, and Google didn't turn up anything useful, so I don't know what needs to be done for that. If you can post the entire exception stacktrace for this error, perhaps we can figure it out. We would also need the exact Solr version. Solr has a default 2MB limit on POST requests. This can be increased with the formdataUploadLimitInKB parameter on the requestDispatcher tag in solrconfig.xml -- assuming that you're running 4.1 or later. Previous versions required changing the request size in the servlet container config, but there was a bug in the example Jetty included in 4.0.0 that made it impossible to change the size. https://bugs.eclipse.org/bugs/show_bug.cgi?id=397130 https://issues.apache.org/jira/browse/SOLR-4223 Regarding the advice to disable the updateLog: I never like to do this, but if you are sending a very large number of updates in a single request, it might be advisable until indexing is complete, so that Solr restart times are not excessive. Thanks, Shawn
