> solrClient.add( documents ); // [2] is of course: solrClient.add( batch ); // [2]
-----Ursprüngliche Nachricht----- Von: Clemens Wyss DEV [mailto:clemens...@mysign.ch] Gesendet: Montag, 22. Februar 2016 09:55 An: solr-user@lucene.apache.org Betreff: AW: AW: OutOfMemory when batchupdating from SolrJ Find attached the relevant part of the batch-update: ... SolrClient solrClient = getSolrClient( coreName, true ); Collection<SolrInputDocument> batch = new ArrayList<SolrInputDocument>(); while ( elements.hasNext() ) { IIndexableElement elem = elements.next(); SolrInputDocument doc = createSolrDocForElement( elem, provider, locale ); // [1] if ( doc != null ) { batch.add( doc ); if ( batch.size() == 100 ) { solrClient.add( documents ); // [2] batch = new ArrayList<SolrInputDocument>(); // [3] } } } if ( !batch.isEmpty() ) { addMultipleDocuments( uniqueProviderName, solrClient, batch ); batch = null; } ... IIndexableElement is part of our index/search framework. [1] creating a single SolrInputDocument [2] handing the SIDs to SolrJ/SolrClient [3] creating a new batch, i.e. releasing the SolrInputDocuments The above code is being executed in an ExecutorService handed in as a lambda. I.e.: executorService.submit( () -> { <the code> } ); Thanks for any advices. If needed, I can also provide the OOM-heapdump ... -----Ursprüngliche Nachricht----- Von: Shawn Heisey [mailto:apa...@elyograg.org] Gesendet: Freitag, 19. Februar 2016 18:59 An: solr-user@lucene.apache.org Betreff: Re: AW: OutOfMemory when batchupdating from SolrJ On 2/19/2016 3:08 AM, Clemens Wyss DEV wrote: > The logic is somewhat this: > > SolrClient solrClient = new HttpSolrClient( coreUrl ); while ( got > more elements to index ) { > batch = create 100 SolrInputDocuments > solrClient.add( batch ) > } How much data is going into each of those SolrInputDocument objects? If the amount of data is very small (a few kilobytes), then this sounds like your program has a memory leak. Can you provide more code detail? Ideally, you would make the entire code available by placing it on the Internet somewhere and providing a URL. If there's anything sensitive in the code, like passwords or public IP addresses, feel free to redact it, but try not to remove anything that affects how the code operates. Thanks, Shawn
