Most probably I need to play around UpdateHandler(s); I am using DirectUpdateHandler with allowDuplicates = false: solrj.SolrServer.add(docs, overwrite=true)
Use case: I have a timestamp on a document; documents in an index get expired by timestamp; same document could be added to the index many times with different timestamp. Most probably heavy 'deletes' cause this problem; will try to implement specific UpdateHandler (can we update just a non-tokenized timestamp or other modified field of an existing document instead of delete/insert?) But "merge" shouldn't lock isn't it? Why new docs can't be added? Thanks -----Original Message----- From: Fuad Efendi Sent: August-11-09 1:09 PM To: solr-user@lucene.apache.org Subject: NativeFSLockFactory, ConcurrentMergeScheduler: why locks? 1. I always have files lucene-xxxx-write.lock and lucene-xxxx-n-write.lock which I believe shouldn't be used with NativeFSLockFactory 2. I use mergeFactor=100 and ramBufferSizeMB=256, few GB indes size. I tried mergeFactor=10 and mergeFactor=1000. It seems ConcurrentMergeScheduler locks everything instead of using separate thread on background... So that my configured system spents half an hour to UPDATE (probably existing in the index) million of documents, then it stops and waits few hours for index merge which is extremely slow (a lot of deletes?) With mergeFactor=1000 I had extremely performant index updates (50,000,000 a first day), and then I was waiting more than 2 days when merge complete (and was forced to kill process). Why it locks everything? Thanks, Fuad
