Well, at least it's _some_ progress ;). Agreed, the segments hanging around is still something of a mystery although if I really stretch I could relate them, maybe.
I believe there's clean-up logic when a core starts up to nuke cruft in the index directory. If the cruft was created after a core swap on the core where Solr couldn't write the core.properties file, then when the core started back up is it possible that it was looking in the wrong directory to clean stuff up. This a total and complete guess though as I don't know that bit of code so.... If the undeleted segment files were in a directory related to the core whose core.properties file wasn't persisted, that would lend some credence to the idea though. FWIW, Erick On Tue, Apr 7, 2015 at 12:18 PM, Shawn Heisey <apa...@elyograg.org> wrote: > On 4/7/2015 10:54 AM, Erick Erickson wrote: >> I'm pretty clueless why you would be seeing this, and slammed with >> other stuff so I can't dig into this right now. >> >> What do the "core.properties" files look like when you see this? They >> should be re-written when you swap cores. Hmmm, I wonder if there's >> some condition where the files are already open and the persistence >> fails? If so we should be logging that error, I have no proof either >> way whether we are or not though. >> >> Guessing that your log files in the problem case weren't all that >> helpful, but let's have a look at them if this occurs again? > > I hadn't had a chance to review the logs, but when I did just now, I > found this: > > ERROR - 2015-04-07 11:56:15.568; > org.apache.solr.core.CorePropertiesLocator; Couldn't persist core > properties to /index/solr4/cores/sparkinc_0/core.properties: > java.io.FileNotFoundException: > /index/solr4/cores/sparkinc_0/core.properties (Permission denied) > > That's fairly clear. I guess my permissions were wrong. My best guess > as to why -- things owned by root from when I created the > core.properties files. Solr does not run as root. I didn't think to > actually look at the permissions before I ran a script that I maintain > which fixes all the ownership on my various directories involved in my > full search installation. > > I don't think this explains the not-deleted segment files problem. > Those segment files were written by solr running as the regular user, so > there couldn't have been a permission problem. > > Thanks, > Shawn >
