On Tue, Aug 12, 2008 at 11:51:12AM -0400, Yonik Seeley wrote: > On Tue, Aug 12, 2008 at 11:19 AM, Jason Rennie <[EMAIL PROTECTED]> wrote: > > On Mon, Aug 11, 2008 at 6:41 PM, Yonik Seeley <[EMAIL PROTECTED]> wrote: > > > >> It's safe... the adds will block until the commit or optimize has finished. > >> > > > > By block, do you mean that the update connection(s) will be held open? > > HTTP calls are synchronous, so yes it will hold a connection open > (unless the container is configured to time out responses after a > while). > > > Our > > optimizes take many minutes to complete. I'm thinking that this could cause > > a large pile of threads to accumulate if we're not careful... > > Many HTTP clients block at a certain number of open connections to the > same server, acting as a natural throttling mechanism.
The route we've taken is to use <autocommit> with 25000 docs or 15 minutes, in the solrconfig.xml and continually add new data. Then 1/night we stop adding new data, and run an optimize. For us this takes about 90 minutes across all our cores. So far this is working well. enjoy, -jeremy -- ======================================================================== Jeremy Hinegardner [EMAIL PROTECTED]
