On 05/06/2013 09:32 AM, Rogowski, Britta wrote:
Hi!
When I write from our database to a HttpSolrServer, (using a
LinkedBlockingQueue to write just one document at a time), I run into memory
problems (due to various constraints, I have to remain on a 32-bit system, so I
can use at most 2 GB RAM).
If I use an EmbeddedSolrServer (to write locally), I have no such problems.
Just now, I tried out ConcurrentUpdateSolrServer (with a queue size of 1, but 3
threads to be safe), and that worked out fine too. I played around with various
GC options and monitored memory with jconsole and jmap, but only found out that
there's lots of byte arrays, SolrInputFields and Strings hanging around.
Since ConcurrentUpdateSolrServer works, I'm happy, but I was wondering if
people were aware of the memory issue around HttpSolrServer.
Regards,
Britta Rogowski
We are not memory constrained so can not confirm the problem with
HttpSolrServer, but how often do you commit ?
Having an autocommit set to a few minutes may help reduce memory usage
during indexation.
Is the memory usage on the Solr server side, or in your feeder code ?
--
André Bois-Crettez
Search technology, Kelkoo
http://www.kelkoo.com/
Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 8, rue du Sentier 75002 Paris
425 093 069 RCS Paris
Ce message et les pièces jointes sont confidentiels et établis à l'attention
exclusive de leurs destinataires. Si vous n'êtes pas le destinataire de ce
message, merci de le détruire et d'en avertir l'expéditeur.
