You've got it. That's the post I was talking about, I was rushed and couldn't find it quickly...
LucidWorks Enterprise uses a trunk version of Solr, so DWPT is in that code in 2.0. For Solr-only, you can just check out a trunk build. Best Erick On Thu, Apr 5, 2012 at 7:54 PM, Mike O'Leary <tmole...@uw.edu> wrote: > First of all, what I was seeing was different from what I thought I was > seeing because a few weeks ago I uncommented the <autoCommit> block in the > solrconfig.xml file and I didn't realize it until yesterday just before I > went home, so that was controlling the commits more than the add and commit > calls that I was making. When I commented that block out again, the times for > index with add(docs, commitWithinMs) and with add(docs) and commit(false, > false) were very similar. Both of them were about 20 minutes faster (38 > minutes instead of about an hour) than indexing with <autoCommit> set to > commit after every 1,000 documents or fifteen minutes. > > Is this the blog post you are talking about: > http://www.searchworkings.org/blog/-/blogs/gimme-all-resources-you-have-i-can-use-them!/? > It seems to be about the right topic. > > I am using Solr 3.5. The feature matrix on one of the Lucid Imagination web > pages says that DocumentWriterPerThread is available in Solr 4.0 and > LucidWorks 2.0. I assume that means LucidWorks Enterprise. Is that right? > Thanks, > Mike > > -----Original Message----- > From: Erick Erickson [mailto:erickerick...@gmail.com] > Sent: Thursday, April 05, 2012 2:45 PM > To: solr-user@lucene.apache.org > Subject: Re: waitFlush and waitSearcher with SolrServer.add(docs, > commitWithinMs) > > Solr version? I suspect your outlier is due to merging segments, if so this > should have happened quite some time into the run. See Simon Wilnauer's blog > post on DocumenWriterPerThread (trunk) code. > > What commitWithin time are you using? > > > Best > Erick > > On Wed, Apr 4, 2012 at 7:50 PM, Mike O'Leary <tmole...@uw.edu> wrote: >> I am indexing some database contents using add(docs, commitWithinMs), and >> those add calls are taking over 80% of the time once the database begins >> returning results. I was wondering if setting waitSearcher to false would >> speed this up. Many of the calls take 1 to 6 seconds, with one outlier that >> took over 11 minutes. >> Thanks, >> Mike >> >> -----Original Message----- >> From: Mark Miller [mailto:markrmil...@gmail.com] >> Sent: Wednesday, April 04, 2012 4:15 PM >> To: solr-user@lucene.apache.org >> Subject: Re: waitFlush and waitSearcher with SolrServer.add(docs, >> commitWithinMs) >> >> >> On Apr 4, 2012, at 6:50 PM, Mike O'Leary wrote: >> >>> If you index a set of documents with SolrJ and use >>> StreamingUpdateSolrServer.add(Collection<SolrInputDocument> docs, int >>> commitWithinMs), it will perform a commit within the time specified, and it >>> seems to use default values for waitFlush and waitSearcher. >>> >>> Is there a place where you can specify different values for waitFlush >>> and waitSearcher, or if you want to use different values do you have >>> to call StreamingUpdateSolrServer.add(Collection<SolrInputDocument> >>> docs) and then call StreamingUpdateSolrServer.commit(waitFlush, >>> waitSearcher) explicitly? >>> Thanks, >>> Mike >> >> >> waitFlush actually does nothing in recent versions of Solr. waitSearcher >> doesn't seem so important when the commit is not done explicitly by the user >> or a client. >> >> - Mark Miller >> lucidimagination.com >> >> >> >> >> >> >> >> >> >> >>
