> *From:* Ronald Taneza [mailto:ronald.tan...@gmail.com]
> *Sent:* dinsdag 21 november 2017 15:44
> *To:* users@subversion.apache.org
>
> I got the error below while running "svnadmin load -M 0" to load a dump
> file created by "svnrdump dump".
>
> svnadmin: E140001: Sum of subblock sizes larger than total block
> content length
>
> This error was reported when "svnadmin load" was loading a big file
> (around 2 GB) from a revision in the dump file.
[...]
> Checking the dump file produced by svnrdump (svn version 1.8.19), I
> noticed that the Content-length for the 2GB file is a negative value!
[...]
> SVN-fs-dump-format-version: 3
> Node-path: TheBigFile
> Node-kind: file
> Node-action: add
> Prop-delta: true
> Prop-content-length: 59
> Text-delta: true
> Text-content-length: 2238208329
> Text-content-md5: d2f79377abb0db99b37460c8156727fb
> Content-length: -2056758908
Thank you for finding this!
I can see this bug existed in svnrdump up to 1.8.19. (For 1.9 I refactored this
to use common code shared with 'svnadmin dump' which does not have this bug.)
In 1.8.19, subversion/svnrdump/svnrdump.c:close_file() contains:
if (fb->dump_text)
...
SVN_ERR(svn_stream_printf(eb->stream, pool,
SVN_REPOS_DUMPFILE_TEXT_CONTENT_LENGTH
": %lu\n",
(unsigned long)info->size));
...
if (fb->dump_props)
SVN_ERR(svn_stream_printf(eb->stream, pool,
SVN_REPOS_DUMPFILE_CONTENT_LENGTH
": %ld\n\n",
(unsigned long)info->size +
propstring->len));
else if (fb->dump_text)
SVN_ERR(svn_stream_printf(eb->stream, pool,
SVN_REPOS_DUMPFILE_CONTENT_LENGTH
": %ld\n\n",
(unsigned long)info->size));
...
info->size is apr_off_t ... probably 64 bits on most systems.
propstring->len is apr_size_t ... probably 64 bits on most systems.
It uses "%lu" for the text content, which thus work OK up to 4 GB, and "%ld"
for the overall content length which thus only works up to 2 GB.
Earlier in this file, the property content length is printed correctly:
buf = apr_psprintf(pool, SVN_REPOS_DUMPFILE_CONTENT_LENGTH
": %" APR_SIZE_T_FMT "\n", len);
The attached patch should fix it; not yet tested.
- Julian
On the '1.8.x' branch: Fix 2GB limit on content-length header in svnrdump.
On platforms where 'long' is 32 bits, such as at least some versions of
Windows, svnrdump printed bad headers for file-representations more than 2GB
in length, up to version 1.8.19.
Attempting to load such a dump file gave this error: "svnadmin: E140001: Sum
of subblock sizes larger than total block content length".
This code was refactored before version 1.9.0 such that the bug was no
longer present.
Found by: Ronald Taneza <ronald.taneza{_AT_}gmail.com>
* subversion/svnrdump/dump_editor.c
(close_file): Print lengths using APR_SIZE_T_FMT instead of 'long'.
--This line, and those below, will be ignored--
Index: subversion/svnrdump/dump_editor.c
===================================================================
--- subversion/svnrdump/dump_editor.c (revision 1816054)
+++ subversion/svnrdump/dump_editor.c (working copy)
@@ -1037,14 +1037,14 @@ close_file(void *file_baton,
": %s\n",
fb->base_checksum));
/* Text-content-length: 39 */
SVN_ERR(svn_stream_printf(eb->stream, pool,
SVN_REPOS_DUMPFILE_TEXT_CONTENT_LENGTH
- ": %lu\n",
- (unsigned long)info->size));
+ ": %" APR_SIZE_T_FMT "\n",
+ info->size));
/* Text-content-md5: 82705804337e04dcd0e586bfa2389a7f */
SVN_ERR(svn_stream_printf(eb->stream, pool,
SVN_REPOS_DUMPFILE_TEXT_CONTENT_MD5
": %s\n",
text_checksum));
@@ -1052,19 +1052,19 @@ close_file(void *file_baton,
/* Content-length: 1549 */
/* If both text and props are absent, skip this header */
if (fb->dump_props)
SVN_ERR(svn_stream_printf(eb->stream, pool,
SVN_REPOS_DUMPFILE_CONTENT_LENGTH
- ": %ld\n\n",
- (unsigned long)info->size + propstring->len));
+ ": %" APR_SIZE_T_FMT "\n\n",
+ info->size + propstring->len));
else if (fb->dump_text)
SVN_ERR(svn_stream_printf(eb->stream, pool,
SVN_REPOS_DUMPFILE_CONTENT_LENGTH
- ": %ld\n\n",
- (unsigned long)info->size));
+ ": %" APR_SIZE_T_FMT "\n\n",
+ info->size));
/* Dump the props now */
if (fb->dump_props)
{
SVN_ERR(svn_stream_write(eb->stream, propstring->data,
&(propstring->len)));
