Re: Reduction of open files

2008-10-17 Thread Grant Ingersoll
That is weird. Can you try running Lucene's CheckIndex tool on the index: http://hudson.zones.apache.org/hudson/job/Lucene-trunk/javadoc//org/apache/lucene/index/CheckIndex.html It should be in the Lucene core library that is shipped w/ Solr. On Oct 16, 2008, at 4:27 PM, Paul deGrandis wrot

Re: Reduction of open files

2008-10-16 Thread Otis Gospodnetic
e.apache.org > Sent: Thursday, October 16, 2008 3:28:29 PM > Subject: Reduction of open files > > I have been working with SOLR for a few months now. According to some > documentation I read, segment files only have one set of all the other > lingustic module type of stuff (normali

Re: Reduction of open files

2008-10-16 Thread Paul deGrandis
My biggest concern is why do the remaining files stay open even if my mergeFactor is 2. I would expect to see one or two segment files and one or two sets of accompanying file (.nrm, .frq, etc), based on the documentation. Paul On Thu, Oct 16, 2008 at 4:23 PM, Paul deGrandis <[EMAIL PROTECTED]>

Re: Reduction of open files

2008-10-16 Thread Paul deGrandis
I currently am not. The document collection is highly volatile (3000 modifications a minute) and from reading thought it would be too much of a performance penalty but never tested it. What behavior in terms of file creation and open fd is seen when useCompoundFile is set to true? Paul On Thu,

Re: Reduction of open files

2008-10-16 Thread Grant Ingersoll
Are you using the compound file format? -Grant On Oct 16, 2008, at 3:28 PM, Paul deGrandis wrote: I have been working with SOLR for a few months now. According to some documentation I read, segment files only have one set of all the other lingustic module type of stuff (normalization, frequen

Reduction of open files

2008-10-16 Thread Paul deGrandis
I have been working with SOLR for a few months now. According to some documentation I read, segment files only have one set of all the other lingustic module type of stuff (normalization, frequency), is there a way to remove/reduce the files not associated with a segment besides optimizing the ind