Re: SVNSERVE Error on Commits

[Nico Kadel-Garcia]

On Thu, May 11, 2017 at 12:20 PM, Graham, Jonathan - US
 wrote:
> We have recently gone through and done some software updates to our CentOS
> 6.9 server that hosts SVN for us. None of the updates appeared to of
> affected anything for SVN overall, however after these patches occurred we
> started getting  org.apache.subversion.javahl.ClientException: E210004:
> Commit failed (details follow): E210004: Malformed network data to occur on
> nearly every commit we have attempted to make to the machine. This also
> caused our machine to completely crash with a kernel panic. After doing some
> digging around and getting core dump files to be generated we were able to
> ascertain that it appears to be something that svnserve is calling that is
> causing this issue. The following is from the core dump file: #0
> 0x003a752eaebb in accept4 (fd=, addr=...,
> addr_len=0x7ffd6d4e9270, flags=524288) at
> ../sysdeps/unix/sysv/linux/accept4.c:34 34 return INLINE_SYSCALL (accept4,
> 4, fd, addr.__sockaddr__, addr_len, flags); Any thoughts on what is going
> on? I’m new to submitting issues, so what are the next steps I need to
> preform to get this escalated up as currently we are unable to do commits at
> all unless I completely reinstall SVN on the machine, then it will let some
> through before erroring out again.

If the machine is crashing, you have a serious kernel and/or hardware
issue. Subversion should *not* be able to crash the host, especially
if it is not running as a root user.

Why would you be afraid to re-install the Subversion software? And why
do you think it would help? Can you boot with an older kernel, of
which the last 2 or so should still be available at boot time, and see
if that helps? Is this real hardware, or a virtualized host on some
instance that may be incompatible with the latest kernel?

Re: SVNSERVE Error on Commits

[Daniel Shahaf]

Nico Kadel-Garcia wrote on Fri, May 12, 2017 at 06:22:15 -0400:
> On Thu, May 11, 2017 at 12:20 PM, Graham, Jonathan - US
>  wrote:
> > We have recently gone through and done some software updates to our CentOS
> > 6.9 server that hosts SVN for us. None of the updates appeared to of
> > affected anything for SVN overall, however after these patches occurred we
> > started getting  org.apache.subversion.javahl.ClientException: E210004:
> > Commit failed (details follow): E210004: Malformed network data to occur on
> > nearly every commit we have attempted to make to the machine. This also
> > caused our machine to completely crash with a kernel panic. After doing some
> > digging around and getting core dump files to be generated we were able to
> > ascertain that it appears to be something that svnserve is calling that is
> > causing this issue. The following is from the core dump file: #0
> > 0x003a752eaebb in accept4 (fd=, addr=...,
> > addr_len=0x7ffd6d4e9270, flags=524288) at
> > ../sysdeps/unix/sysv/linux/accept4.c:34 34 return INLINE_SYSCALL (accept4,
> > 4, fd, addr.__sockaddr__, addr_len, flags); Any thoughts on what is going
> > on? I’m new to submitting issues, so what are the next steps I need to
> > preform to get this escalated up as currently we are unable to do commits at
> > all unless I completely reinstall SVN on the machine, then it will let some
> > through before erroring out again.
> 
> If the machine is crashing, you have a serious kernel and/or hardware
> issue.

Until you can fix the kernel and/or hardware to stop the panics, you
might try switching svnserve from --daemon mode to --inetd mode or
--tunnel mode, so that accept(2) will be called not by svnserve but by
inetd or sshd.  (It's possible that something in how svnserve
calls accept(8) tickles a kernel bug, and that sshd / inetd will not
tickle that bug.)

Also, look at the libapr versions if you haven't already.  Try upgrading
(or if you had, try downgrading).  (libapr abstracts accept(2) for libsvn.)