Re: Doc add limit

2006-07-26 Thread sangraal aiken
I removed everything from the Add xml so the docs looked like this: 187880 187852 and it still hung at 6,144... -S On 7/26/06, Yonik Seeley <[EMAIL PROTECTED]> wrote: If you narrow the docs down to just the "id" field, does it still happen at the same place? -Yonik On 7/26/06, sangraa

Re: Recompilation of latest lucene seems to break update of Solr - Solution

2006-07-26 Thread Chris Hostetter
:I had to use the latest lucene version, not available on their : site, but only on the subversioning system. (Version called 2.1 - : 419723 2006-07-06 22:14:07) Just to clarify: Solr is definitely a little more "bleeding edge" then the stable releases of Lucene. The Lucene JARs in solr's li

Re: Doc add limit

2006-07-26 Thread Yonik Seeley
If you narrow the docs down to just the "id" field, does it still happen at the same place? -Yonik On 7/26/06, sangraal aiken <[EMAIL PROTECTED]> wrote: I see the problem on Mac OS X/JDK: 1.5.0_06 and Debian/JDK: 1.5.0_07. I don't think it's a socket problem, because I can initiate additional

Re: Doc add limit

2006-07-26 Thread sangraal aiken
I see the problem on Mac OS X/JDK: 1.5.0_06 and Debian/JDK: 1.5.0_07. I don't think it's a socket problem, because I can initiate additional updates while the server is hung... weird I know. Thanks for all your help, I'll send a post if/when I find a solution. -S On 7/26/06, Yonik Seeley <[EMA

Re: Doc add limit

2006-07-26 Thread Yonik Seeley
Tomcat problem, or a Solr problem that is only manifesting on your platform, or a JVM or libc problem, or even a client update problem... (possibly you might be exhausting the number of sockets in the server by using persistent connections with a long timeout and never reusing them?) What is your

Re: Doc add limit

2006-07-26 Thread sangraal aiken
Right now the heap is set to 512M but I've increased it up to 2GB and yet it still hangs at the same number 6,144... Here's something interesting... I pushed this code over to a different server and tried an update. On that server it's hanging on #5,267. Then tomcat seems to try to reload the web

Re: Doc add limit

2006-07-26 Thread Yonik Seeley
So it looks like your client is hanging trying to send somethig over the socket to the server and blocking... probably because Tomcat isn't reading anything from the socket because it's busy trying to restart the webapp. What is the heap size of the server? try increasing it... maybe tomcat could

Re: Doc add limit

2006-07-26 Thread sangraal aiken
Thanks for you help Yonik, I've responded to your questions below: On 7/26/06, Yonik Seeley <[EMAIL PROTECTED]> wrote: It's possible it's not hanging, but just takes a long time on a specific add. This is because Lucene will occasionally merge segments. When very large segments are merged, it

Re: Doc add limit

2006-07-26 Thread Yonik Seeley
It's possible it's not hanging, but just takes a long time on a specific add. This is because Lucene will occasionally merge segments. When very large segments are merged, it can take a long time. In the log file, add commands are followed by the number of milliseconds the operation took. Next

Doc add limit

2006-07-26 Thread sangraal aiken
Hey there... I'm having an issue with large doc updates on my solr installation. I'm adding in batches between 2-20,000 docs at a time and I've noticed Solr seems to hang at 6,144 docs every time. Breaking the adds into smaller batches works just fine, but I was wondering if anyone knew why this w

Recompilation of latest lucene seems to break update of Solr - Solution

2006-07-26 Thread Tom Weber
Hi again, sorry to spam the list, but wanted to share the solution which let the system compile. I had to use the latest lucene version, not available on their site, but only on the subversioning system. (Version called 2.1 - 419723 2006-07-06 22:14:07) To get this version, use th

Re: Recompilation of latest lucene seems to break update of Solr

2006-07-26 Thread Yonik Seeley
Hi Tom, I had fixed the LUCENE-545 backward incompatability in Lucene here: http://issues.apache.org/jira/browse/LUCENE-609 Although it shouldn't be neccessary, maybe it would work if you put the new lucene libs in solr's lib dir and rebuild solr? -Yonik On 7/26/06, Tom Weber <[EMAIL PROTECTED]

Recompilation of latest lucene seems to break update of Solr - Addum

2006-07-26 Thread Tom Weber
Hi again, with the latest nightbuild of solr and the latest lucene, the error still persists, but the error message is different, jhust to be complete here the actual error message: java.lang.NoSuchMethodError: org.apache.lucene.document.Document.add (Lorg/apache/lucene/document/Fieldabl

Recompilation of latest lucene seems to break update of Solr

2006-07-26 Thread Tom Weber
Hello, jhust compiled the latest version of lucene (), updated the webapps/solr/WEB-INF/lib/ with the 3 jar files: lucene-core-2.0.1-dev.jar lucene-snowball-2.0.1-dev.jar lucene-highlighter-2.0.1-dev.jar Restarted solr, The Admin interface of solr is still running, but trying

Own Similarity Class in Solr

2006-07-26 Thread Tom Weber
Hello, I would like to alter the similarity behaviour of solr to remove the fieldnorm factor in the similarity calculations. As far as I read, I need to recreate my own similarity class and import it into solr using the config in schema.xml. Has anybody already tweaked or played wit