I am not sure if it was completely fixed. (This was related to a Lucene bug)
But you can try w/ a recent build and confirm it for us.
I have never encountered these during our tests in windows XP/Linux
I have attached a patch which logs the names of the files which could
not get deleted (which may help us diagnose the problem). If you are
comfortable applying a patch you may try it out.
--Noble
On Fri, Jan 23, 2009 at 12:05 AM, Jeff Newburn <jnewb...@zappos.com> wrote:
> Few weeks ago is our version. Does this contribute to the directory issues
> and extra files that are left?
>
>
> On 1/22/09 10:33 AM, "Noble Paul നോബിള് नोब्ळ्" <noble.p...@gmail.com>
> wrote:
>
>> This was reported by another user and was fixed recently.Are you using
>> a recent version?
>> --Noble
>>
>> On Fri, Jan 23, 2009 at 12:00 AM, Jeff Newburn <jnewb...@zappos.com> wrote:
>>> We have both. A majority of them are just empty but others have almost a
>>> full index worth of files. I have also noticed that during a lengthy index
>>> update the system will throw errors about how it cannot move one of the
>>> index files. Essentially on reindex the system does not replicate until an
>>> optimize is done which changes all of the file names allowing the file error
>>> go away.
>>>
>>> Jan 22, 2009 10:17:15 AM org.apache.solr.handler.SnapPuller copyAFile
>>> SEVERE: Unable to move index file from: /data/index.20090122101604/_8n.tvx
>>> to: /data/index/_8n.tvx
>>>
>>>
>>>
>>> On 1/22/09 10:23 AM, "Noble Paul നോബിള് नोब्ळ्" <noble.p...@gmail.com>
>>> wrote:
>>>
>>>> Jeff ,
>>>> Do you see both the empty index.xxxx dirs as well as the extra files
>>>> in the index?
>>>> --Noble
>>>>
>>>> On Thu, Jan 22, 2009 at 10:37 PM, Jeff Newburn <jnewb...@zappos.com> wrote:
>>>>> We are seeing something very similar. Ours is intermittent and usually
>>>>> happens a great deal on random days. Often it seems to occur during large
>>>>> index updates on the master.
>>>>>
>>>>>
>>>>> On 1/22/09 8:58 AM, "Shalin Shekhar Mangar" <shalinman...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> On Thu, Jan 22, 2009 at 10:18 PM, Jaco <jdevr...@gmail.com> wrote:
>>>>>>
>>>>>>> Hm, I don't know what to do anymore. I tried this:
>>>>>>> - Run Tomcat service as local administrator to overcome any
>>>>>>> permissioning
>>>>>>> issues
>>>>>>> - Installed latest nightly build (I noticed that item I mentioned before
>>>>>>> (
>>>>>>> http://markmail.org/message/yq2ram4f3jblermd) had been committed which
>>>>>>> is
>>>>>>> good
>>>>>>> - Build a small master and slave core to try it all out
>>>>>>> - With each replication, the number of files on slave grows, and the
>>>>>>> directories index.xxxxxxx.. are not removed
>>>>>>> - I tried sending explicit commit commands to the slave, assuming it
>>>>>>> wouldn't help, which was true.
>>>>>>> - I don't see any reference to SolrDeletion in the log of the slave
>>>>>>> (it's
>>>>>>> there in the log of the master)
>>>>>>>
>>>>>>> Can anybody recommend some action to be taken? I'm building up some
>>>>>>> quite
>>>>>>> large production cores right now, and don't want the slaves to eat up
>>>>>>> all
>>>>>>> hard disk space of course..
>>>>>>
>>>>>>
>>>>>> How frequently do you optimize your index? Does the number of files
>>>>>> decrease
>>>>>> after an optimize?
>>>>>>
>>>>>> Can you execute the "indexversion" command:
>>>>>> /replication?command=indexversion
>>>>>> and then issue the following command with the returned index version:
>>>>>>
> /replication?command=filelist&indexversion=<INDEX_VERSION_BY_FIRST_COMMAND>>>>>
>>
>>>>>> The above will give the list of files being used by that commit point.
>>>>>>
>>>>>> Can you compare the list of files given by the above command and with the
>>>>>> files you see in the solr/data/index directory? How many are extra?
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>
--
--Noble Paul
Index: src/java/org/apache/solr/handler/SnapPuller.java
===================================================================
--- src/java/org/apache/solr/handler/SnapPuller.java (revision 736216)
+++ src/java/org/apache/solr/handler/SnapPuller.java (working copy)
@@ -587,21 +587,27 @@
static boolean delTree(File dir) {
if (dir == null || !dir.exists())
return false;
+ boolean isSuccess = true;
File contents[] = dir.listFiles();
if (contents != null) {
for (File file : contents) {
if (file.isDirectory()) {
boolean success = delTree(file);
- if (!success)
- return false;
+ if (!success){
+ LOG.error("Unable to delete directory : "+ file);
+ isSuccess = false;
+ }
} else {
boolean success = file.delete();
- if (!success)
+ if (!success){
+ LOG.error("Unable to delete file : "+ file);
+ isSuccess = false;
return false;
+ }
}
}
}
- return dir.delete();
+ return isSuccess && dir.delete();
}
/**
@@ -853,6 +859,7 @@
//close the file
fileChannel.close();
} catch (Exception e) {/* noop */
+ LOG.error("Error closing the file stream: "+ this.saveAs ,e);
}
try {
post.releaseConnection();
