On 11/09/2007, at 7:21 AM, Ryan McKinley wrote:
The other problem is that after some time we get a "Too Many Open
Files" error when autocommit fires.
Have you checked your ulimit settings?
http://wiki.apache.org/lucene-java/
LuceneFAQ#head-48921635adf2c968f7936dc07d51dfb40d638b82
ulimit -n <number>.
Yeah I'm aware of the ulimit, I'm just keen to identify what's
causing it to happen before starting to increase limits. Given the
write.lock errors as well I'm particularly suspicious of it. That
said, most likely it happens whenever a search and a write are
happening at the same time and two sets of the files get opened which
is enough to kick it over the limit. The fact that it fixes itself is
a good indication that it's not a file handle leak.
As mike mentioned, you may also want to use 'single' as the
lockType. In solrconfig set:
<indexDefaults>
...
<lockType>single</lockType>
</indexDefaults>
I'll give that a go. Looks like it didn't make it into Solr 1.2 so
I'll try upgrading to the nightly build.
I could of course switch to using the Solr webapp since we're
running in Tomcat anyway, however I really like the ability to
have a single WAR file that contains everything and also not have
to worry about actually making HTTP requests and the complexity
that adds.
This sounds like a good candidate to try solrj:
http://wiki.apache.org/solr/Solrj
This way you write your app independent of how you connect to
solr. It also takes care of the XML parsing for you and lets you
work with objects rather then strings.
Just when you think you know everything on the wiki.... This looks
fantastic I wish I knew about it a couple of months ago. Not sure if
I can resolve the JVM 1.4 vs 1.5 issue for that API though (we have
multiple deploy targets, it's painful but needed).
ryan
Lots of stuff to play with.
Thanks,
Adrian Sutton
http://www.symphonious.net
