Re: Solr indexing configuration help

2008-06-02 Thread Mike Klaas
om: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Yonik Seeley Sent: Sunday, June 01, 2008 6:47 AM To: solr-user@lucene.apache.org Subject: Re: Solr indexing configuration help On Sun, Jun 1, 2008 at 4:43 AM, Gaku Mak <[EMAIL PROTECTED]> wrote: I have tried Yonik's suggest

RE: Solr indexing configuration help

2008-06-02 Thread Norskog, Lance
rg Subject: Re: Solr indexing configuration help On Sun, Jun 1, 2008 at 4:43 AM, Gaku Mak <[EMAIL PROTECTED]> wrote: > I have tried Yonik's suggestions with the following: > 1) all autowarming are off > 2) commented out firstsearch and newsearcher event handlers > 3) incre

Re: Solr indexing configuration help

2008-06-01 Thread Yonik Seeley
On Sun, Jun 1, 2008 at 4:43 AM, Gaku Mak <[EMAIL PROTECTED]> wrote: > I have tried Yonik's suggestions with the following: > 1) all autowarming are off > 2) commented out firstsearch and newsearcher event handlers > 3) increased autocommit interval to 600 docs and 30 minutes (previously 50 > docs a

Re: Solr indexing configuration help

2008-06-01 Thread Gaku Mak
Hi Yonik and other solr dev/users, I have tried Yonik's suggestions with the following: 1) all autowarming are off 2) commented out firstsearch and newsearcher event handlers 3) increased autocommit interval to 600 docs and 30 minutes (previously 50 docs and 5 minutes) In addition, I updated the

Re: Solr indexing configuration help

2008-05-30 Thread Yonik Seeley
Some things to try: - turn off autowarming on the master - turn off autocommit, unless you really need it, or change it to be less agressive: autocommitting every 50 docs is bad if you are rapidly adding documents. - set maxWarmingSearchers to 1 to prevent the buildup of searchers -Yonik On Fri,

Re: Solr indexing configuration help

2008-05-30 Thread Gaku Mak
I started running the test on 2 other machines with similar specs but more RAM (4G). One of them now has about 60k docs and still running fine. On the other machine, solr died at about 43k docs. A short while before solr died, I saw that there were 5 searchers at the same time. Do any of you know

Re: Solr indexing configuration help

2008-05-29 Thread Gaku Mak
Looking further at the java error, those crashes are mostly related to GC. VM_Operation (0x41b429e0): parallel gc failed allocation, mode: safepoint, requested by thread 0x2aab1988c400 I'm following the http://java.sun.com/javase/6/webnotes/trouble/TSG-VM/html/gbyzo.html and see if

Re: Solr indexing configuration help

2008-05-29 Thread Yonik Seeley
It's most likely a 1) hardware issue: bad memory OR 2) incompatible libraries (most likely libc version for the JVM). If you have another box around, try that. -Yonik On Thu, May 29, 2008 at 9:51 PM, Gaku Mak <[EMAIL PROTECTED]> wrote: > > Hi Yonik and others, > > I'm getting this java error af

Re: Solr indexing configuration help

2008-05-29 Thread Gaku Mak
Hi Yonik and others, I'm getting this java error after switching to JVM 1.6.0_3. This error occurs after the stress test has been going for a while and failed at 12K docs level and at 18K again. Am I doing something wrong? Please help! Thanks! # # An unexpected error has been detected by Jav

Re: Solr indexing configuration help

2008-05-28 Thread Gaku Mak
That's good to know, I will get the 1.6.0_3 version of JDK and try it out. Do you guys have any comments on my other questions? Any help is greatly appreciated!! Thanks! -Gaku Yonik Seeley wrote: > > On Wed, May 28, 2008 at 10:30 PM, Gaku Mak <[EMAIL PROTECTED]> wrote: >> I used the admin GU

Re: Solr indexing configuration help

2008-05-28 Thread Otis Gospodnetic
o: solr-user@lucene.apache.org > Sent: Wednesday, May 28, 2008 10:30:39 PM > Subject: Re: Solr indexing configuration help > > > I used the admin GUI to get the java info. > java.vm.specification.vendor = Sun Microsystems Inc. > > Any suggestion? Thanks a lot for your

Re: Solr indexing configuration help

2008-05-28 Thread Yonik Seeley
On Wed, May 28, 2008 at 10:30 PM, Gaku Mak <[EMAIL PROTECTED]> wrote: > I used the admin GUI to get the java info. > java.vm.specification.vendor = Sun Microsystems Inc. Well, your original email listed IcedTea... but that is mostly Sun code, so maybe that's why the vendor is still listed as Sun.

Re: Solr indexing configuration help

2008-05-28 Thread Gaku Mak
I used the admin GUI to get the java info. java.vm.specification.vendor = Sun Microsystems Inc. Any suggestion? Thanks a lot for your help!! -Gaku Yonik Seeley wrote: > > Not sure why you would be getting an OOM from just indexing, and with > the 1.5G heap you've given the JVM. > Have you tr

Re: Solr indexing configuration help

2008-05-28 Thread Yonik Seeley
Not sure why you would be getting an OOM from just indexing, and with the 1.5G heap you've given the JVM. Have you tried Sun's JVM? -Yonik On Wed, May 28, 2008 at 7:35 PM, gaku113 <[EMAIL PROTECTED]> wrote: > > Hi all Solr users/developers/experts, > > I have the following scenario and I apprecia