I haven't been able to get a profiler at the server yet, but I thought I
might show how my code works, because it's quite different from the example
in the link you provided...
public synchronized ResultItem[] search(String query) throws
CorruptIndexException, IOException{
SolrIndexSearcher searcher = new SolrIndexSearcher(solrCore.getSchema(),
"MySearcher", solrCore.getIndexDir(), true);
Hits hits = search(searcher, query);
for(int i =0; i < hits.length(); i++){
parse(hits.doc(i));
//add to result-array
}
searcher.close();
//return result-array
}
private Hits search(SolrIndexSearcher searcher, String pQuery){
try {
SolrQueryParser parser = new SolrQueryParser(solrCore.getSchema(),
"text");
//default search field is called "text"
Query query = parser.parse(pQuery);
return searcher.search(query);
}
//catch exceptions
}
This is the code that does the searching. The searcher is passed as a
parameter to the search-method, because it needs to be open while I'm
parsing the documents in the hits. I know I should move the closure of the
search-operation to a finally-block, will do that in any case, but I doubt
it will solve the problem because I've never had any exceptions in this
code. Might the problem be that I'm not using SolrQueryRequest objects?
Best regards,
Yonik Seeley wrote:
>
> This is probably related to "using Solr/Lucene embeddedly"
> See the warning at the top of http://wiki.apache.org/solr/EmbeddedSolr
>
> It does sound like your SolrIndexSearcher objects aren't being closed.
> Solr (via SolrCore) doesn't rely on garbage collection to close the
> searchers (since gc unfortunately can't be triggered by low
> descriptors). SolrIndexSearcher objects are reference counted and
> closed when no longer in use. This means that SolrQueryRequest
> objects must always be closed or the refcount will be off.
>
> Not sure where you could start except perhaps trying to verify the
> number of live SolrIndexSearcher objects.
>
> -Yonik
>
> On Dec 20, 2007 8:20 AM, amamare <[EMAIL PROTECTED]> wrote:
>>
>> I have an application consisting of three web applications running on
>> JBoss
>> 1.4.2 on a Linux Redhat server. I'm using Solr/Lucene embeddedly to
>> create
>> and maintain a frequently updated index. Once updated, the index is
>> copied
>> to another directory used for searching. Old index-files in the search
>> directory are then deleted. The streams used to copy the files are closed
>> in
>> finally-blocks. After a few days an IOException occurs because of "too
>> many
>> open files". When I run the linux command
>>
>> ls -l /proc/26788/fd/
>>
>> where 26788 is jboss' process id, it gives me a seemingly ever-increasing
>> list of deleted files (1 per update since I optimize on every update and
>> use
>> compound file format), marked with 'deleted' in parantheses. They are all
>> located in the search directory. From what I understand this means that
>> something still holds a reference to the file, and that the file will be
>> permanently deleted once this something loses its reference to it.
>>
>> Only SolrIndexSearcher objects are in direct contact with these files in
>> the
>> search application. The searchers are local objects in search-methods,
>> and
>> are closed after every search operation. In theory, the garbage collector
>> should collect these objects later (though while profiling other
>> applications I've noticed that it often doesn't garbage collect until the
>> allocated memory starts running out).
>>
>> The other objects in contact with the files are the FileOutputStreams
>> used
>> to copy them, but as stated above, these are closed in finally-blocks and
>> thus should hold no reference to the files.
>>
>> I need to get rid of the "too many open files"-problem. I suspect that it
>> is
>> related to the almost-deleted files in the proc-dir, but I know too
>> little
>> of Linux to be sure. Does the problem ring a bell to anyone, or do you
>> have
>> any ideas as to how I can get rid of the problem?
>>
>> All help is greatly appreciated.
>> --
>> View this message in context:
>> http://www.nabble.com/SolrIndexWriter-holding-reference-to-deleted-file--tp14436326p14436326.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
>
>
--
View this message in context:
http://www.nabble.com/SolrIndexWriter-holding-reference-to-deleted-file--tp14436326p14594325.html
Sent from the Solr - User mailing list archive at Nabble.com.
