Re: effect of continuous deletes on index's read performance

2012-02-06 Thread Michael McCandless
On Mon, Feb 6, 2012 at 8:20 AM, prasenjit mukherjee wrote: > Pardon my ignorance, Why can't the IndexWriter and IndexSearcher share > the same underlying in-memory datastructure so that IndexSearcher need > not be reopened with every commit. Because the semantics of an IndexReader in Lucene guar

Re: effect of continuous deletes on index's read performance

2012-02-06 Thread prasenjit mukherjee
Pardon my ignorance, Why can't the IndexWriter and IndexSearcher share the same underlying in-memory datastructure so that IndexSearcher need not be reopened with every commit. On 2/6/12, Erick Erickson wrote: > Your continuous deletes won't affect performance > noticeably, that's true. > > But

Re: effect of continuous deletes on index's read performance

2012-02-06 Thread Nagendra Nagarajayya
You could also try Solr 3.4 with RankingAlgorithm as this offers NRT. You can get more information about NRT for Solr 3.4 from here: http://solr-ra.tgels.org/wiki/en/Near_Real_Time_Search_ver_3.x Regards, - Nagendra Nagarajayya http://solr-ra.tgels.org http://rankingalgorithm.tgels.org On 2/

Re: effect of continuous deletes on index's read performance

2012-02-06 Thread Erick Erickson
Your continuous deletes won't affect performance noticeably, that's true. But you're really doing bad things with the commit after every add or delete. You haven't said whether you have a master/ slave setup or not, but assuming you're searching on the same machine you're indexing to, each time yo

Re: effect of continuous deletes on index's read performance

2012-02-05 Thread prasenjit mukherjee
Thanks Otis. commitWithin will definitely work for me ( as I currently am using 3.4 version, which doesnt have NRT yet ). Assuming that I use commitWithin=10secs, are you saying that the continuous deletes ( without commit ) wont have any affect on performance ? I was under the impression that de

Re: effect of continuous deletes on index's read performance

2012-02-05 Thread Otis Gospodnetic
Hi Prasenjit, It sounds like at this point your main enemy might be those per-doc-add commits.  Don't commit until you need to see your new docs in results.  And if you need NRT then use softCommit option with Solr trunk (http://search-lucene.com/?q=softcommit&fc_project=Solr) or use commitWith