Hi there Yes I have tried that, and it makes a small difference but still getting the slowdown, just slightly later. No hints from the logs about the slowdown, no errors or useful info in there even if I set all the logging on.
Willem On Fri, Oct 7, 2011 at 2:38 PM, Jan Høydahl <jan....@cominvent.com> wrote: > Hi, > > Have you tried to do a commit after the deleteByQuery only? > Also, what seems to cause the slowdown? Any hints from the logs? > > -- > Jan Høydahl, search solution architect > Cominvent AS - www.cominvent.com > Solr Training - www.solrtraining.com > > On 7. okt. 2011, at 10:04, Willem Basson wrote: > > > Hi there > > > > We are currently moving from Solr 1.4 to 3.4 and we are seeing a few > issues > > with adding documents. > > We do a delete by query and then do a lot of adds, about 100k before we > do a > > commit and optimise. > > With 1.4 this was all fine, not super quick but didn't see any problems. > > With 3.4 the rate of adding documents seriously degrades. For our one > index > > at about 80% it severely slows down but struggles and completes. > > For the other index which has quite a few more fields (up to 6000+ for > some > > documents) it slows down at about 20%. > > > > If we do periodic commits then we don't see the slowdown, but that causes > us > > some other issues with replication etc. and while we can go down that > route > > if we really must we would like to know what has changed from 1.4 to 3.4 > to > > cause this behaviour. I have tried changing the LuceneMatchVersion to > > LUCENE_34 and upped to memory from 2GB to 4GB on a machine with 8GB ram > but > > it really doesn't make any difference to the behaviour. Don't see any > errors > > in the log files. > > > > Any ideas of what we could try to diagnose or fix the problem? > > > > -- > > Willem Basson > > -- Willem Basson
