Adrian Sutton wrote:
On 11/09/2007, at 7:21 AM, Ryan McKinley wrote:
The other problem is that after some time we get a "Too Many Open
Files" error when autocommit fires.
Have you checked your ulimit settings?
http://wiki.apache.org/lucene-java/LuceneFAQ#head-48921635adf2c968f7936dc07d51dfb40d638b82
ulimit -n <number>.
Yeah I'm aware of the ulimit, I'm just keen to identify what's causing
it to happen before starting to increase limits. Given the write.lock
errors as well I'm particularly suspicious of it. That said, most likely
it happens whenever a search and a write are happening at the same time
and two sets of the files get opened which is enough to kick it over the
limit. The fact that it fixes itself is a good indication that it's not
a file handle leak.
lucene opens a lot of files. It can easily get beyond 1024. (I think
the default). I'm no expert on how the file handling works, but I think
more files are open if you are searching and writing at the same time.
If you can't increase the limit you can try:
<useCompoundFile>true</useCompoundFile>
It is slower, but if you are unable to change the ulimit on the deployed
machines....
As mike mentioned, you may also want to use 'single' as the lockType.
In solrconfig set:
<indexDefaults>
...
<lockType>single</lockType>
</indexDefaults>
I'll give that a go. Looks like it didn't make it into Solr 1.2 so I'll
try upgrading to the nightly build.
If you need to use this in production soon, I'd suggest sticking with
1.2 for a while. There has been a LOT of action in trunk and it may be
good to let it settle before upgrading a production system.
You should not need to upgrade to fix the write.lock and Too Many Open
Files problem. Try increasing ulimit or using a compoundfile before
upgrading.
Just when you think you know everything on the wiki....
someone finally updates it!
