Re: Injest pauses

2012-08-29 Thread Otis Gospodnetic
quot;solr-user@lucene.apache.org" > Cc: > Sent: Wednesday, August 29, 2012 1:05 PM > Subject: RE: Injest pauses > > Interestingly it is not pausing during every commit so at least a portion of > the > time the async commit code is working.  Trying to track down t

Re: Injest pauses

2012-08-29 Thread Alexey Serba
Original Message- > From: Voth, Brad (GE Corporate) > Sent: Wednesday, August 29, 2012 12:32 PM > To: solr-user@lucene.apache.org > Subject: RE: Injest pauses > > Thanks, I'll continue with my testing and tracking down the block. > > -Original Message- > F

RE: Injest pauses

2012-08-29 Thread Voth, Brad (GE Corporate)
: solr-user@lucene.apache.org Subject: RE: Injest pauses Thanks, I'll continue with my testing and tracking down the block. -Original Message- From: ysee...@gmail.com [mailto:ysee...@gmail.com] On Behalf Of Yonik Seeley Sent: Wednesday, August 29, 2012 12:28 PM To: solr-user@lucene.apach

RE: Injest pauses

2012-08-29 Thread Voth, Brad (GE Corporate)
Thanks, I'll continue with my testing and tracking down the block. -Original Message- From: ysee...@gmail.com [mailto:ysee...@gmail.com] On Behalf Of Yonik Seeley Sent: Wednesday, August 29, 2012 12:28 PM To: solr-user@lucene.apache.org Subject: Re: Injest pauses On Wed, Aug 29, 20

Re: Injest pauses

2012-08-29 Thread Yonik Seeley
On Wed, Aug 29, 2012 at 11:58 AM, Voth, Brad (GE Corporate) wrote: > Anyone know the actual status of SOLR-2565, it looks to be marked as resolved > in 4.* but I am still seeing long pauses during commits using 4.* SOLR-2565 is definitely committed - adds are no longer blocked by commits (at lea

RE: Injest pauses

2012-08-29 Thread Voth, Brad (GE Corporate)
. -Original Message- From: Voth, Brad (GE Corporate) Sent: Wednesday, August 29, 2012 10:17 AM To: solr-user@lucene.apache.org Subject: RE: Injest pauses Very interesting links, after much more digging yesterday this appears to be exactly what I'm seeing. I am using 4.0 beta currently f

RE: Injest pauses

2012-08-29 Thread Voth, Brad (GE Corporate)
exey Serba [ase...@gmail.com] Sent: Wednesday, August 29, 2012 6:45 AM To: solr-user@lucene.apache.org Subject: Re: Injest pauses Hey Brad, > This leads me to believe that a single merge thread is blocking indexing from > occuring. > When this happens our producers, which distribute th

Re: Injest pauses

2012-08-29 Thread Alexey Serba
Hey Brad, > This leads me to believe that a single merge thread is blocking indexing from > occuring. > When this happens our producers, which distribute their updates amongst all > the shards, pile up on this shard and wait. Which version of Solr you are using? Have you tried 4.0 beta? * http

RE: Injest pauses

2012-08-28 Thread Voth, Brad (GE Corporate)
cers to see if the flushing is able to keep up at some point. From: Shawn Heisey [s...@elyograg.org] Sent: Tuesday, August 28, 2012 1:37 PM To: solr-user@lucene.apache.org Subject: Re: Injest pauses On 8/27/2012 2:54 PM, Voth, Brad (GE Corporate) wrote: &g

Re: Injest pauses

2012-08-28 Thread Shawn Heisey
On 8/27/2012 2:54 PM, Voth, Brad (GE Corporate) wrote: Hello all, I'm working on implementing a solution in a very high index rate, lower query rate project. We have a very small record size, 2 small strings, 6 longs, 7 ints, and 2 dates, indexing on 8 of those fields. We need to be able to

RE: Injest pauses

2012-08-28 Thread Voth, Brad (GE Corporate)
onday, August 27, 2012 4:54 PM To: solr-user@lucene.apache.org Subject: Injest pauses Hello all, I'm working on implementing a solution in a very high index rate, lower query rate project. We have a very small record size, 2 small strings, 6 longs, 7 ints, and 2 dates, indexing on 8 of those f

Injest pauses

2012-08-27 Thread Voth, Brad (GE Corporate)
Hello all, I'm working on implementing a solution in a very high index rate, lower query rate project. We have a very small record size, 2 small strings, 6 longs, 7 ints, and 2 dates, indexing on 8 of those fields. We need to be able to maintain an injest rate of 50k records/sec average. Thr