Hi,
The custom code we have is something like this :
public class MySearchHandlerextends SearchHandler {
@Override public void handleRequestBody(SolrQueryRequest req, SolrQueryResponse
rsp)throws Exception {
SolrIndexSearcher searcher =req.getSearcher();
try{
// Do stuff with the searcher....
}finally {
req.close();
}
}
}
Despite the fact that we always close the request each time we get
a SolrIndexSearcher from the request, the number of SolrIndexSearcher
instances is increasing. Each time a new commit is done on the index, a
new Searcher is created (this is normal) but the old one remains. Is
there something wrong with this custom code ? Shall we try something
explained there :
http://stackoverflow.com/questions/20515493/solr-huge-number-of-open-searchers
? Thanks, Gérald Reinhart (working with Elodie on the subject) On
03/07/2017 05:45 PM, Elodie Sannier wrote:
Thank you Erick for your answer.
The files are deleted even without JVM restart but they are still seen
as DELETED by the kernel.
We have a custom code and for the migration to Solr 6.4.0 we have added
a new code with req.getSearcher() but without "close".
We will decrement the reference count on a resource for the Searcher
(prevent the Searcher remains open after a commit) and see if it fixes
the problem.
Elodie
On 03/07/2017 03:55 PM, Erick Erickson wrote:
Just as a sanity check, if you restart the Solr JVM, do the files
disappear from disk?
Do you have any custom code anywhere in this chain? If so, do you open
any searchers but
fail to close them? Although why 6.4 would manifest the problem but
other code wouldn't
is a mystery, just another sanity check.
Best,
Erick
On Tue, Mar 7, 2017 at 6:44 AM, Elodie Sannier <elodie.sann...@kelkoo.fr> wrote:
Hello,
We have migrated from Solr 5.4.1 to Solr 6.4.0 and the disk usage has
increased.
We found hundreds of references to deleted index files being held by solr.
Before the migration, we had 15-30% of disk space used, after the migration
we have 60-90% of disk space used.
We are using Solr Cloud with 2 collections.
The commands applied on the collections are:
- for incremental indexation mode: add, deleteById with commitWithin of 30
minutes
- for full indexation mode: add, deleteById, commit
- for switch between incremental and full mode: deleteByQuery, createAlias,
reload
- there is also an autocommit every 15 minutes
We have seen the email "Solr leaking references to deleted files"
2016-05-31 which describe the same problem but the mentioned bugs are fixed.
We manually tried to force a commit, a reload and an optimize on the
collections without effect.
Is a problem of configuration (merge / delete policy) or a possible
regression in the Solr code ?
Thank you
Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 158 Ter Rue du Temple 75003 Paris
425 093 069 RCS Paris
Ce message et les pièces jointes sont confidentiels et établis à l'attention
exclusive de leurs destinataires. Si vous n'êtes pas le destinataire de ce
message, merci de le détruire et d'en avertir l'expéditeur.
--
Elodie Sannier
Software engineer
<http://www.kelkoo.com/>
*E*elodie.sann...@kelkoo.fr* Skype*kelkooelodies
*T*+33 (0)4 56 09 07 55
*A*Parc Sud Galaxie, 6, rue des Méridiens, 38130 Echirolles
Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 158 Ter Rue du Temple 75003 Paris
425 093 069 RCS Paris
Ce message et les pièces jointes sont confidentiels et établis à l'attention
exclusive de leurs destinataires. Si vous n'êtes pas le destinataire de ce
message, merci de le détruire et d'en avertir l'expéditeur.
--
Gérald ReinhartSoftware Engineer
<http://www.kelkoo.com/>
*E*gerald.reinh...@kelkoo.com*A*Parc Sud Galaxie, 6, rue des Méridiens,
38130 Echirolles, FR
Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 158 Ter Rue du Temple 75003 Paris
425 093 069 RCS Paris
Ce message et les pièces jointes sont confidentiels et établis à l'attention
exclusive de leurs destinataires. Si vous n'êtes pas le destinataire de ce
message, merci de le détruire et d'en avertir l'expéditeur.
