Awesome! Thanks for looking in to this over the weekend guys. I look forward to 1.8.19.
Regards, David -----Original Message----- From: Stefan Fuhrmann [mailto:stef...@apache.org] Sent: Sunday, July 30, 2017 10:14 AM To: Bert Huijben <b...@qqmail.nl>; David Engel <den...@magnitude.com>; users@subversion.apache.org Subject: Re: Checksum mismatch bug in 1.8.18 Hi David, thanks for the report; the issue turned out to be quite serious. The underlying bug has been present since at least as early as 1.8.0 but only surfaced with the added SHA1-collision detection code in 1.8.18. Under certain circumstances, yet-to-be-committed data would be cached under the wrong key. The impact ranges from bogus errors as you experienced them to silent repository corruption that only a 'svnadmin verify' run would detect. So far, no such corruption has been demonstrated but it remains a plausible scenario. Luckily, the problem seems to be mostly restricted to 'svnadmin load', with the "rep sharing" enable for the respective repo. 'svnadmin load -M 0' will also not have that issue. I fixed the code and the fix should be shipped with the upcoming 1.8.19 release. -- Stefan^2. On 29.07.2017 14:37, Bert Huijben wrote: > This specific error message was added in the last release, so yes it is new > in > the last versions. The last 1.9.x version also added it and I'm surprised > that > you see the error on 1.8 and not 1.9 (or vice versa). > > It tries to tell you that you have two files with an identical SHA-1 hash, > but > different contents. A case that we accidentally didn't catch before, that > might > have caused some data loss under extreme circumstances. > > This scenario is pretty unlikely to trigger, unless you specifically try to > do > that. I'll try to verify what exactly is in your dump file to see if you > really > hit this scenario, or that we have a bug in our code that tries to catch this > scenario. Assuming you properly verified with 1.9.6 (and not 1.9.5 or > earlier) I > assume that this is a bug in 1.8.18, as other scenarios are really extremely > unlikely. > > I'm not sure I'll have time to look into this before Monday though, so > perhaps > one of the other developers beats me to it. > > Bert > > *From:*David Engel [mailto:den...@magnitude.com] > *Sent:* zaterdag 29 juli 2017 00:01 > *To:* users@subversion.apache.org > *Subject:* Checksum mismatch bug in 1.8.18 > > Hi, > > I think I'm running into a bug in svnadmin in the latest 1.8 release > (1.8.18). > It is not present in 1.8.17 nor 1.9.5 nor 1.9.6. I'm guessing it's related > to > the recent changes around strict repsharing. I'm not subscribed to the > mailing > list, so please CC me on any responses. > > Here are the relevant details: > > OS: Windows 10 and Windows 2012 R2 > > Subversion release: 1.8.18 > > Both SlikSVN release: > > svnadmin, version 1.8.18-SlikSvn-1.8.18-X64 (SlikSvn/1.8.18) X64 > > compiled Jul 17 2017, 13:03:37 on x86_64-microsoft-windows6.2.9200 > > And Collabnet release: > > svnadmin, version 1.8.18 (r1800620) > > compiled Jul 7 2017, 05:51:59 on x86_64-microsoft-windows6.2.9200 > > Performing an svnadmin load results in the following error: > > ... > > <<< Started new transaction, based on original revision 650 > > * editing path : > Dev/Common/Utils/External/StampVerData/MetaBuilderVersion.inf ... done. > > * editing path : > Dev/Common/Utils/External/StampVerData/MetaBuilderVersionDBG.inf ... done. > > * editing path : > Dev/Common/Utils/External/StampVerData/OracleVersion.inf > ...svnadmin: E160000: SHA1 of reps '-1 0 45 132 > a6ea37d29094deece56250ebe167ce46 > 6a2d6bc4c7e58c0d2627a1b0dd5b23e542aec4d1 650-i2/_8' and '-1 136 45 132 > a6ea37d29094deece56250ebe167ce46 6a2d6bc4c7e58c0d2627a1b0dd5b23e542aec4d1 > 650-i2/_8' matches (6a2d6bc4c7e58c0d2627a1b0dd5b23e542aec4d1) but contents > differ > > svnadmin: E160004: Filesystem is corrupt > > svnadmin: E200014: Checksum mismatch while reading representation: > > expected: a6ea37d29094deece56250ebe167ce46 > > actual: 5f696f5d0755f3bcb5e927bdfba4bfa8 > > In order to reproduce the error, you can use the attached VersionStamps3 > repository dump file. It was taken with the same version of svnadmin from a > multi-GB repository that's been around for over a decade and pruned > (svndumpfilter) to a small set of files and revisions in one folder while > still > encountering the error. I first encountered the error another place very > quickly in an incremental load on top of a repo that was loaded from 1.8.14, > but > that one was on a revision north of 120,000+. This is way easier to > reproduce. > I tried filtering the dump further and dropping empty revisions, but the > error > did not occur in those cases. The error also did not occur using 1.8.17, > 1.9.5., and 1.9.6. It also did not occur in 1.8.18 if I set > enable-rep-sharing > = false before performing the load. > > Repro.bat script: > > @echo off > > :defineCommands > > rem You might need to adjust these lines to point to your > > rem compiled-from-source Subversion binaries, if using those: > > for %%i in (svn.exe) do set SVN="%%~$PATH:i" > > for %%i in (svnadmin.exe) do set SVNADMIN="%%~$PATH:i" > > :defineUrls > > rem Only supported access method: file:// <file:///>. If http:// or svn://, > then > > rem you'll have to configure it yourself first. > > set URL=file:///%CD%/repos > > set URL=%URL:\=/% > > echo Base url for repo: %URL% > > :cleanAllDirsAndCreateRepo > > if exist repos rmdir /s /q repos > > if exist import-me rmdir /s /q import-me > > if exist wc rmdir /s /q wc > > %SVNADMIN% create repos > > pause > > :prepareGreekTree > > echo Making a Tree for import... > > mkdir import-me\Dev\Common\Utils\External > > echo Importing it... > > cd import-me > > %SVN% import -q -m "Initial import." %URL% > > cd .. > > rem This is where your reproduction recipe goes. > > svnadmin --version > > svnadmin load repos < VersionStamps3 > > goto :eof > > Thanks, > > David >
