corrupted svn repository with “serialized hash missing terminator” error
I have a local subversion repository. I am trying to convert it to Mercurial. In the process I discovered that repository is corrupted. I have tried the following 1. svnadmin verify - confirmed that repository is corrupted at revision 1265. The verify stops at this revision. 2. svnadmin recover - was unable to recover the repository. 3. tried to dump the repository content. svnadmin dump fails at revision 1265. So dumped till 1264. Took another dump from revision 1322. (all revision from 1265 to 1321 are giving the serialized hash terminator error). 4. I tried to combine the dumps and load it into a new repository. However, some files were added in the 'missing' revisions (1265 to 1321) and while loading dump I am getting errors. I googled for error and advice seems to be dump the repository and reload the contents in new repository. However, that does not work for me. I tried to find what exactly is the meaning of the error. However, I could not get any information. I tried to read the documentation of FSFS structure ( http://svn.apache.org/repos/asf/subversion/trunk/subversion/libsvn_fs_fs/structure). I tried to manually check revision file for 1265 and see what is wrong. But could not find anything visibly wrong. Is there any way to correct this error in the repository (manually/programmatically) ? If someone can give me a hint on what is wrong in the file and underwhat conditions this error occurs, I will try to write a script to correct it. Regards, Nitin
Re: corrupted svn repository with “serialized hash missing terminator” error
Hi Stefan, Thanks for the hint. I tried the fsfsverify and fsfixer also. Both did not work. Looking back I think the problem started when I upgraded to from 1.7.x to 1.8.1. What exactly is 'serialized hash terminator' error ? Regards, Nitin On Sun, Sep 1, 2013 at 4:33 PM, Stefan Sperling wrote: > On Sun, Sep 01, 2013 at 12:02:18PM +0530, Nitin Bhide wrote: > > I have a local subversion repository. I am trying to convert it to > > Mercurial. In the process I discovered that repository is corrupted. I > have > > tried the following > > > > 1. svnadmin verify - confirmed that repository is corrupted at revision > > 1265. The verify stops at this revision. > > 2. svnadmin recover - was unable to recover the repository. > > 3. tried to dump the repository content. svnadmin dump fails at revision > > 1265. So dumped till 1264. Took another dump from revision 1322. (all > > revision from 1265 to 1321 are giving the serialized hash terminator > error). > > 4. I tried to combine the dumps and load it into a new repository. > However, > > some files were added in the 'missing' revisions (1265 to 1321) and while > > loading dump I am getting errors. > > > > I googled for error and advice seems to be dump the repository and reload > > the contents in new repository. However, that does not work for me. > > > > I tried to find what exactly is the meaning of the error. However, I > could > > not get any information. I tried to read the documentation of FSFS > > structure ( > > > http://svn.apache.org/repos/asf/subversion/trunk/subversion/libsvn_fs_fs/structure > ). > > I tried to manually check revision file for 1265 and see what is wrong. > But > > could not find anything visibly wrong. > > > > Is there any way to correct this error in the repository > > (manually/programmatically) ? > > > > If someone can give me a hint on what is wrong in the file and underwhat > > conditions this error occurs, I will try to write a script to correct it. > > > > Regards, > > Nitin > > Give this script a try: > https://svn.apache.org/repos/asf/subversion/trunk/contrib/server-side/fsfsverify.py > It can fix some instances of corruption which was probably caused by > older Subversion/APR versions. > -- In theory, there is NO difference between theory and practice. But, in practice, there IS Yogi Berra My company's new product : BootstrapToday - Simple and Sensible Application Lifecycle Management http://www.bootstraptoday.com
Re: corrupted svn repository with “serialized hash missing terminator” error
Hi Philip, Thanks for the clarification. Does the message mean either in db/revs or in db/revprops 'END' tag is missing ? Regards, Nitin On Thu, Sep 5, 2013 at 11:50 PM, Philip Martin wrote: > Nitin Bhide writes: > > > Hi Stefan, > > > > Thanks for the hint. I tried the fsfsverify and fsfixer also. Both did > not > > work. Looking back I think the problem started when I upgraded to from > > 1.7.x to 1.8.1. > > > > What exactly is 'serialized hash terminator' error ? > > It means one of the repository files is corrupt. It could be a revision > files in db/revs or it could be a revprop file in db/revprops. A > serialized hash is a series of K/V pairs followed by END: > > K 10 > svn:author > V 2 > pm > K 8 > svn:date > V 27 > 2013-09-05T18:00:22.881511Z > K 7 > svn:log > V 1 > m > END > > -- > Philip Martin | Subversion Committer > WANdisco // *Non-Stop Data* > -- In theory, there is NO difference between theory and practice. But, in practice, there IS Yogi Berra My company's new product : BootstrapToday - Simple and Sensible Application Lifecycle Management http://www.bootstraptoday.com
Re: corrupted svn repository with “serialized hash missing terminator” error
Philip/Stefan/Andreas, Thanks for the help. I was able to write a small python script specific to my needs and recover the repository. It was corrupted revision property file. I have written the details in blog post. http://nitinbhide.blogspot.in/2013/09/recovering-from-corrupted-subversion.html Thanks again (especially Philip), Regards, Nitin On Fri, Sep 6, 2013 at 2:36 PM, Philip Martin wrote: > Nitin Bhide writes: > > > Thanks for the clarification. Does the message mean either in db/revs or > in > > db/revprops 'END' tag is missing ? > > It means the file ended before the END tag was seen, but it's not > possible to say exactly what is missing. > > -- > Philip Martin | Subversion Committer > WANdisco // *Non-Stop Data* > -- In theory, there is NO difference between theory and practice. But, in practice, there IS Yogi Berra My company's new product : BootstrapToday - Simple and Sensible Application Lifecycle Management http://www.bootstraptoday.com
Re: corrupted svn repository with “serialized hash missing terminator” error
Hi Andreas,
Thanks.
I think such recovery tools should be there. Probably not as part of
'svnadmin' but at least in contrib. The FSFS verify helped me. I will post
the script I used to my website. I am not sure if the attachments are
allowed in this list. If allowed, I will post here as well.
I am a reasonably good python developer. However I don't know much about
svn internals. So if someone is interested in writing such tool, I am
willing to contribute.
Regards,
Nitin
On Sat, Sep 7, 2013 at 6:49 PM, Andreas Mohr wrote:
> Hi,
>
> On Sat, Sep 07, 2013 at 05:50:29PM +0530, Nitin Bhide wrote:
> >Philip/Stefan/Andreas,
> >
> >Thanks for the help. I was able to write a small python script
> >specific to
> >my needs and recover the repository. It was corrupted revision
> >property
> >file. I have written the details in blog post.
> >[1]
> http://nitinbhide.blogspot.in/2013/09/recovering-from-corrupted-subversion.html
>
> Thanks for openly documenting/sharing your experience!
> (and of course congrats for your smashing success)
>
> Now a question that would spring from this would be whether this is
> something that could be more automated in Subversion project circles
> (for this one user who was nicely able to draw the necessary conclusions
> and then even write his own tool, there's probably a dozen more users
> who gave up in despair).
> That's probably not a feature that svnadmin ought to provide ("form
> intermediate revisions out of thin air when encountering corruption in
> certain revisions"), but rather an external recovery tool that's made
> for that purpose.
> But those are just random thoughts of someone who's quite external to all
> this... ;)
>
> Andreas Mohr
>
> --
> GNU/Linux. It's not the software that's free, it's you.
>
>
--
In theory, there is NO difference between theory and practice.
But, in practice, there IS
Yogi Berra
My company's new product : BootstrapToday - Simple and Sensible Application
Lifecycle Management
http://www.bootstraptoday.com
