This is probably related to "using Solr/Lucene embeddedly" See the warning at the top of http://wiki.apache.org/solr/EmbeddedSolr
It does sound like your SolrIndexSearcher objects aren't being closed. Solr (via SolrCore) doesn't rely on garbage collection to close the searchers (since gc unfortunately can't be triggered by low descriptors). SolrIndexSearcher objects are reference counted and closed when no longer in use. This means that SolrQueryRequest objects must always be closed or the refcount will be off. Not sure where you could start except perhaps trying to verify the number of live SolrIndexSearcher objects. -Yonik On Dec 20, 2007 8:20 AM, amamare <[EMAIL PROTECTED]> wrote: > > I have an application consisting of three web applications running on JBoss > 1.4.2 on a Linux Redhat server. I'm using Solr/Lucene embeddedly to create > and maintain a frequently updated index. Once updated, the index is copied > to another directory used for searching. Old index-files in the search > directory are then deleted. The streams used to copy the files are closed in > finally-blocks. After a few days an IOException occurs because of "too many > open files". When I run the linux command > > ls -l /proc/26788/fd/ > > where 26788 is jboss' process id, it gives me a seemingly ever-increasing > list of deleted files (1 per update since I optimize on every update and use > compound file format), marked with 'deleted' in parantheses. They are all > located in the search directory. From what I understand this means that > something still holds a reference to the file, and that the file will be > permanently deleted once this something loses its reference to it. > > Only SolrIndexSearcher objects are in direct contact with these files in the > search application. The searchers are local objects in search-methods, and > are closed after every search operation. In theory, the garbage collector > should collect these objects later (though while profiling other > applications I've noticed that it often doesn't garbage collect until the > allocated memory starts running out). > > The other objects in contact with the files are the FileOutputStreams used > to copy them, but as stated above, these are closed in finally-blocks and > thus should hold no reference to the files. > > I need to get rid of the "too many open files"-problem. I suspect that it is > related to the almost-deleted files in the proc-dir, but I know too little > of Linux to be sure. Does the problem ring a bell to anyone, or do you have > any ideas as to how I can get rid of the problem? > > All help is greatly appreciated. > -- > View this message in context: > http://www.nabble.com/SolrIndexWriter-holding-reference-to-deleted-file--tp14436326p14436326.html > Sent from the Solr - User mailing list archive at Nabble.com.
