Re: Reindexing strategy

2013-06-03 Thread Dotan Cohen
On Fri, May 31, 2013 at 3:57 AM, Michael Sokolov gt wrote: > On UNIX platforms, take a look at vmstat for basic I/O measurement, and > iostat for more detailed stats. One coarse measurement is the number of > blocked/waiting processes - usually this is due to I/O contention, and you > will want to

Re: Reindexing strategy

2013-05-30 Thread Michael Sokolov
On 5/30/2013 8:30 AM, Dotan Cohen wrote: On Wed, May 29, 2013 at 5:37 PM, Shawn Heisey wrote: It's impossible for us to give you hard numbers. You'll have to experiment to know how fast you can reindex without killing your servers. A basic tenet for such experimentation, and something you hop

Re: Reindexing strategy

2013-05-30 Thread Dotan Cohen
On Wed, May 29, 2013 at 5:37 PM, Shawn Heisey wrote: > It's impossible for us to give you hard numbers. You'll have to > experiment to know how fast you can reindex without killing your > servers. A basic tenet for such experimentation, and something you > hopefully already know: You'll want to

Re: Reindexing strategy

2013-05-29 Thread Shawn Heisey
On 5/29/2013 6:01 AM, Dotan Cohen wrote: > I mean 'overload' Solr in the sense that it cannot read, process, and > write data fast enough because too much data is being handled. I > remind you that this system is writing hundreds of documents per > minute. Certainly there is a limit to what Solr ca

Re: Reindexing strategy

2013-05-29 Thread Dotan Cohen
On Wed, May 29, 2013 at 2:41 PM, Upayavira wrote: > I presume you are running Solr on a multi-core/CPU server. If you kept a > single process hitting Solr to re-index, you'd be using just one of > those cores. It would take as long as it takes, I can't see how you > would 'overload' it that way. >

Re: Reindexing strategy

2013-05-29 Thread Upayavira
I presume you are running Solr on a multi-core/CPU server. If you kept a single process hitting Solr to re-index, you'd be using just one of those cores. It would take as long as it takes, I can't see how you would 'overload' it that way. I guess you could have a strategy that pulls 100 documents