Hi Johan,
Sorry for the delayed response as I was in long vacation.
Yes, I don’t think we are hitting the real sha-1 collision in our repository
and as you said it might be another bug in the sha-1 collision detection code.
I am sure that no one committed the sha-1-colliding files from
https://shattered.io
Our repository is very huge with 1 Million + revisions and I am seeing the
issue only while loading the specific revision 724413. I am able to load the
revisions before and after it.
As you suggested I have now loaded the dump till 724412 which is the revision
before the problematic one with rep sharing enabled.
It's not allowing me to load the problematic revision with rep-sharing disabled
and getting below message.
FYI,
[root@vitech-svn-slave-test-01 svndump]# svnadmin load /u01/svn/repos <
incdump724413.txt
svnadmin: E200002: Error while parsing config file: /u01/svn/repos/db/fsfs.conf:
svnadmin: E200002: line 39: Option expected
[root@vitech-svn-slave-test-01 svndump]#
If I re-enable the rep-sharing and trying to load the problematic revision, I
get the error while loading the specific file"SecurityServiceImpl.java" of that
revision number. Please let me know if you need any further details.
Just to reiterate my source SVN is 1.9.5 and target is 1.9.7 version
FYI,
[root@vitech-svn-slave-test-01 svndump]# svnadmin load /u01/svn/repos <
incdump724413.txt
<<< Started new transaction, based on original revision 724413
* editing path :
v3-web-product/branches/8.3.0.279_IPERS/benefits/Scripts/DML_Exception/web_dml_admin_businessEntityRelationsHistory.sql
... done.
* editing path :
v3-web-product/branches/8.3.0.279_IPERS/benefits/Scripts/DML_Exception/web_dml_admin_securityApplicationUserHistory.sql
... done.
* editing path :
v3-web-product/branches/8.3.0.279_IPERS/benefits/Scripts/DML_Exception/web_dml_admin_securityRoleUserHistory.sql
... done.
* editing path :
v3-web-product/branches/8.3.0.279_IPERS/benefits/Scripts/DML_Exception/web_dml_admin_userMaintenance_audit_Labels.sql
... done.
* editing path :
v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/admin/model/BusinessEntityRelationHist.hbm
... done.
* editing path :
v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/admin/model/BusinessEntityRelationHist.java
... done.
* editing path :
v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/admin/model/SecurityApplicationUsrHist.hbm
... done.
* editing path :
v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/admin/model/SecurityApplicationUsrHist.java
... done.
* editing path :
v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/admin/model/SecurityRoleUserHist.hbm
... done.
* editing path :
v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/admin/model/SecurityRoleUserHist.java
... done.
* editing path :
v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/admin/model/base/BaseBusinessEntityRelationHist.java
... done.
* editing path :
v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/admin/model/base/BaseSecurityApplicationUsrHist.java
... done.
* editing path :
v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/admin/model/base/BaseSecurityRoleUserHist.java
... done.
* editing path :
v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/admin/services/SecurityService.java
... done.
* editing path :
v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/admin/services/SecurityServiceImpl.java
...svnadmin: E160000: SHA1 of reps '669684 7 1143 195972
efbdb058ce857b2860cfa245f014f0b9 9db457be74545c184242e57208bf1d56db1f15b2
724412-fiyk/_16' and '-1 0 929 195972 efbdb058ce857b2860cfa245f014f0b9
9db457be74545c184242e57208bf1d56db1f15b2 724412-fiyk/_16' matches
(9db457be74545c184242e57208bf1d56db1f15b2) but contents differ
svnadmin: E160004: Filesystem is corrupt
svnadmin: E200014: Checksum mismatch while reading representation:
expected: efbdb058ce857b2860cfa245f014f0b9
actual: 04a53277f405bcbab8a3b1fff9f8c6e0
Thanks,
Santosh.
-----Original Message-----
From: Johan Corveleyn [mailto:jcor...@gmail.com]
Sent: Thursday, January 25, 2018 4:27 PM
To: Santosh Kondapuram <skondapu...@vitechinc.com>
Cc: users@subversion.apache.org
Subject: Re: Issue while loading the SVN Dump SVN version 1.9.7
On Thu, Jan 25, 2018 at 10:40 AM, Santosh Kondapuram
<skondapu...@vitechinc.com> wrote:
> Hi Johan,
>
> I have tried reloading the dump as you suggested with -M 0 option but still
> I am running in to the same issue.
> Seems like the svn admin could not load the dump due to sha1 collision files.
> So the question is how do I identify the sha1 colliding files ? does the
> error stack trace say something about it ?
> As per the sha1-advisory note if we create a Subversion permission
> rule(authz) that block access to the one or both of the files, then tools
> like svnsync will not send the colliding content.
> So If someone can help me in identifying the colliding files then I would
> like to block them and sync my mirror repository. Below is the error I am
> hitting.
> Any help much appreciated and I am very much new to the subversion.
>
> https://subversion.apache.org/security/sha1-advisory.txt
>
> FYI,
> svnadmin: E160000: SHA1 of reps '669684 7 1143 195972
> efbdb058ce857b2860cfa245f014f0b9
> 9db457be74545c184242e57208bf1d56db1f15b2 724412-fiyk/_16' and '-1 0
> 929 195972 efbdb058ce857b2860cfa245f014f0b9
> 9db457be74545c184242e57208bf1d56db1f15b2 724412-fiyk/_16' matches
> (9db457be74545c184242e57208bf1d56db1f15b2) but contents differ
> svnadmin: E160004: Filesystem is corrupt
> svnadmin: E200014: Checksum mismatch while reading representation:
> expected: efbdb058ce857b2860cfa245f014f0b9
> actual: 04a53277f405bcbab8a3b1fff9f8c6e0
>
> Thanks,
> Santosh.
Hi Santosh,
Hmmm, I chatted a bit about this on irc with Andreas Stieger. It seems unlikely
that you have a real sha-1 collision in your repository.
Unless someone committed those on purpose (for instance by committing files
from https://shattered.io/). It's also possible that there is another bug in
the "sha-1 collision detection code" (one that doesn't get bypassed with the
-M0 trick).
So, are you sure someone committed such sha-1-colliding files?
Also: the revision where it fails, is that the last revision in the dumpstream?
Something you could try to get further, or to get more information about what's
going on: if you disable rep-sharing SVN should be able to store the sha-1
collision (you won't be able to check out both colliding files in a working
copy though). So you could 'svnadmin load' until the revision right before the
problematic one, then disable rep-sharing (putting the option
enable-rep-sharing = false in db/fsfs.conf), then load the problematic
revision. Then you might be able to find out at least the name of the file
that's causing the problem.
HTH,
--
Johan
This e-mail message and any files transmitted with it may contain confidential
and proprietary information and are intended solely for the use of the
individual or entity to which they are addressed. Any unauthorized review, use,
disclosure or distribution is strictly prohibited. If you have received this
e-mail in error please notify the sender by reply email and destroy all copies
of the original message. Thank you for your cooperation.
