Hello Erick,

Thanks for your answer.

After reading previous subjects on the user list, we had already tried to 
change the parameters we mentioned.

- concurrent warming searchers : we have set the maxWarmingSearchers attribute 
to 2 
<maxWarmingSearchers>2</maxWarmingSearchers>

- we have tried 32 and 64 for the ramBufferSizeMB attribute

- there is no other load on the Solr server, or search when we index

- the autocommit is defined with openSearcher=false, maxTime=600000ms, 
maxDocs=6000 - the autoSoftCommit is defined with maxTime=1000
We have already tried to change the softcommit and the commit parameters in 
several ways. We have also tried to commit on the client size.
Ok I try to commit more often.

- we have used cache sizes defined in the example : size=512

The documents size is not too big, I think : 1 million documents produce a 6Go 
index.

Thanks for your answer on multiple collections. I thought multiple collections 
should have the same schema in Zk after reading a wiki page : 
http://wiki.apache.org/solr/NewSolrCloudDesign : "The entire cluster must have 
a single schema and solrconfig"
Maybe is this page deprecated ?
I also thought that because OOM errors occur only when we index a second 
collection. There is no problem when indexing a single collection.

Going with 4.1 would not be easy for now... We'll think about it.

Thanks.

Joel


-----Message d'origine-----
De : Erick Erickson [mailto:erickerick...@gmail.com] 
Envoyé : jeudi 31 janvier 2013 14:00
À : solr-user@lucene.apache.org
Objet : Re: Indexing problems

I'm really surprised you're hitting OOM errors, I suspect you have something 
else pathological in your system. So, I'd start checking things like
- how many concurrent warming searchers you allow
- How big your indexing RAM is set to (we find very little gain over 128M BTW).
- Other load on your Solr server. Are you, for instance, searching on it too?
- what your autocommit characterstics are (think about autocommitting fairly 
often with openSearcher=false).
- have you defined huge caches?
- .....

How big are these documents anyway? With 12G of ram, they'd have to be 
absolutely _huge_ to matter much.

Multiple collections should work fine in ZK. I really think you have some 
innocent-looking configuration setting thats bollixing you up, this is not 
expected behavior.

If at all possible, I'd also go with 4.1. I don't really think it's relevant to 
your situation, but there have been a lot of improvements in the code....

Best
Erick

Reply via email to