On Monday, April 27, 2020 at 5:56:52 PM UTC-7, Ian Lance Taylor wrote:
>
> On Mon, Apr 27, 2020 at 5:10 PM Liam <[email protected] <javascript:>>
> wrote:
> >
> > On Monday, April 27, 2020 at 4:22:41 PM UTC-7, Ian Lance Taylor wrote:
> >>
> >> On Sun, Apr 26, 2020 at 4:55 PM Liam <[email protected]> wrote:
> >> >
> >> > During an io.Copy() where the Writer is a TCPConn and the Reader is a
> 200K disk file, my code may concurrently Write() on the same TCPConn.
> >> >
> >> > I see the result of the Write() inserted into the result of the
> io.Copy(). I had the impression that was impossible, but I must be
> mistaken, as the sendfile(2) docs read:
> >> >
> >> > Note that a successful call to sendfile() may write fewer bytes than
> requested; the caller should be prepared to retry the call if there were
> unsent bytes.
> >> >
> >> > Could someone confirm that one must indeed synchronize concurrent use
> of tcpConn.Write() and io.Copy(tcpConn, file)?
> >>
> >> Synchronization should not be required. internal/poll.Sendfile
> >> acquires a write lock on dstFD, which is the TCP socket. That should
> >> ensure that the contents of an ordinary Write (which also acquires a
> >> write lock) should not interleave with the sendfile data.
> >>
> >> That said, if the sendfile system call cannot be used for whatever
> >> reason, the net package will fall back on doing ordinary Read and
> >> Write calls. And those Write calls can be interleaved with other
> >> Write calls done by a different goroutine. I think that is probably
> >> permitted, in that io.Copy doesn't promise to not interleave with
> >> simultaneous Write calls on the destination.
> >>
> >> So in the general case you should indeed use your own locking to avoid
> >> interleaving between io.Copy and a concurrent Write.
> >
> >
> > Thanks for the details. Where could I add a Println() to reveal why it
> doesn't call poll.Sendfile()?
> >
> > I expect this system to use sendfile(2). The file is a normal file on a
> local partition (running on a Digital Ocean Droplet).
> >
> >
> > /etc/fstab has:
> > UUID=[omitted] / ext4 defaults 1 1
> >
> >
> > $ df -h
> > Filesystem Size Used Avail Use% Mounted on
> > devtmpfs 981M 0 981M 0% /dev
> > tmpfs 996M 0 996M 0% /dev/shm
> > tmpfs 996M 436K 995M 1% /run
> > tmpfs 996M 0 996M 0% /sys/fs/cgroup
> > /dev/vda1 59G 5.7G 51G 11% /
> > tmpfs 200M 0 200M 0% /run/user/0
>
>
> Well, I don't know for that it doesn't use sendfile, I just can't
> explain the results you're seeing if it does use sendfile.
>
> The place to start is to find out why (or whether)
> internal/poll.Sendfile is returning 0, nil.
>
Erm, feeling stupid, but I didn't find any invocations of poll.Sendfile()
in stdlib...
$ grep -r poll.Sendfile /usr/local/go/src/
$ grep -r 'Sendfile(' /usr/local/go/src/
/usr/local/go/src/net/sendfile_test.go:func TestSendfile(t *testing.T) {
/usr/local/go/src/net/http/fs_test.go:func TestLinuxSendfile(t *testing.T) {
/usr/local/go/src/internal/poll/sendfile_linux.go: n, err1 :=
syscall.Sendfile(dst, src, nil, n)
/usr/local/go/src/internal/poll/sendfile_solaris.go: n, err1 :=
syscall.Sendfile(dst, src, &pos1, n)
/usr/local/go/src/internal/poll/sendfile_bsd.go: n, err1 :=
syscall.Sendfile(dst, src, &pos1, n)
/usr/local/go/src/syscall/syscall_js.go:func Sendfile(outfd int, infd int,
offset *int64, count int) (written int, err error) {
/usr/local/go/src/syscall/syscall_nacl.go:func Sendfile(outfd int, infd
int, offset *int64, count int) (written int, err error) {
/usr/local/go/src/syscall/syscall_unix.go:func Sendfile(outfd int, infd
int, offset *int64, count int) (written int, err error) {
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/05c2cde4-9768-45bf-9d00-586cf4486081%40googlegroups.com.
