Hi Shawn, important note ahead: I appreciate your help very much! > it's too proprietary for public eyes That's not the reason for not posting all the code. I just tried to extract the "relevant parts" in order to prevent not seeing the forest for the trees. And yes " addMultipleDocuments" ends up in solrClient.add( batch ). There are no other substitutions/modifications, except for it "being executed in a lambda". If you desire I can send you the code of our SolrUtil class...?
> For efficiency reasons, you might try "batch.clear();" instead of creating a > new ArrayList Will do that, allthough I am not hunting for nano's, at least not at the moment ;) -----Ursprüngliche Nachricht----- Von: Shawn Heisey [mailto:apa...@elyograg.org] Gesendet: Montag, 22. Februar 2016 15:57 An: solr-user@lucene.apache.org Betreff: Re: AW: AW: OutOfMemory when batchupdating from SolrJ On 2/22/2016 1:55 AM, Clemens Wyss DEV wrote: > SolrClient solrClient = getSolrClient( coreName, true ); > Collection<SolrInputDocument> batch = new > ArrayList<SolrInputDocument>(); while ( elements.hasNext() ) { > IIndexableElement elem = elements.next(); > SolrInputDocument doc = createSolrDocForElement( elem, provider, locale ); > // [1] > if ( doc != null ) > { > batch.add( doc ); > if ( batch.size() == 100 ) > { > solrClient.add( documents ); // [2] > batch = new ArrayList<SolrInputDocument>(); // [3] > } > } > } > if ( !batch.isEmpty() ) > { > addMultipleDocuments( uniqueProviderName, solrClient, batch ); > batch = null; > } Did you type the above code as a paraphrase of the actual code, or is that a copy/paste of actual code? I'm guessing that it was typed or hand-edited to something more simple, not pasted as-is, because in the loop you have an add() call on the client, but outside the loop you have something entirely different -- a locally defined method called addMultipleDocuments. I would expect these calls to be identical. Your followup, where you pointed out an error in the add statement, suggests even more that you did not provide actual code. I understand the desire to protect your investment in your work, but partial information makes it difficult to offer help. You could send the code to me unicast if you think it's too proprietary for public eyes, but if you do that, I will keep the discussion on the list and only talk about the code in general terms. The mailing list will generally eat any attachments, other means are necessary to relay code. For efficiency reasons, you might try "batch.clear();" instead of creating a new ArrayList. It doesn't seem likely that this would actually fix the problem, but since the true code is probably different, I can't say for sure. Thanks, Shawn
