Can you post the full logs leading up to the corruption (including
full stack traces)?  Ie, after reboot when the permissions problem
started.

I'm very surprised this led to index corruption.  If Lucene is unable
to write any of the files for a new commit, that commit aborts and
those partially written files should not be referenced.  So I don't
understand how a non-existent _X.del file became referenced.  Are you
sure nothing else was touching the index directory?  Is there any
chance two writers were opened at once?

Can you run CheckIndex on the index, to see how many segments are
corrupt (it has a main method)?  You can use CheckIndex -fix to then
remove the broken segments, but that removes all docs in those
segments from the index.

Mike

On Fri, May 1, 2009 at 1:50 AM, Jacob Singh <jacobsi...@gmail.com> wrote:
> We rebooted a machine, and the permissions on the external drive where
> the index was stored had changed.  We didn't realize it immediately,
> because searches were working and updates were not throwing errors
> back to the client.
>
> These ended up in catalina.out
>
> Apr 22, 2009 11:57:12 PM org.apache.solr.common.SolrException log
> SEVERE: java.io.FileNotFoundException: /vol/CDJL-13578/index/_kw.fdt
> (Permission denied)
>
>
> This is because of autocommit.  None of the update requests were
> returning 500s or 403s because Solr hadn't tried to commit yet.  So
> the client thought everything was groovy.
>
> Now, the index cannot update, even though the perms are fixed.  We now
> get an error like this:
>
> Apr 23, 2009 11:51:05 PM
> org.apache.solr.update.DirectUpdateHandler2$CommitTracker run
> SEVERE: auto commit error...
>
> java.io.FileNotFoundException: /vol/CDJL-13578/index/_kq_1.del (No
> such file or directory)
>
> Is there anything we can do to salvage the situation?
>
> Thanks,
> Jacob
>
>
> --
>
> +1 510 277-0891 (o)
> +91 9999 33 7458 (m)
>
> web: http://pajamadesign.com
>
> Skype: pajamadesign
> Yahoo: jacobsingh
> AIM: jacobsingh
> gTalk: jacobsi...@gmail.com
>

Reply via email to