On Monday, April 27, 2020 at 5:56:52 PM UTC-7, Ian Lance Taylor wrote:
>
> On Mon, Apr 27, 2020 at 5:10 PM Liam <[email protected] <javascript:>>
> wrote:
> >
> > On Monday, April 27, 2020 at 4:22:41 PM UTC-7, Ian Lance Taylor wrote:
> >>
> >> On Sun, Apr 26, 2020 at 4:55 PM Liam <[email protected]> wrote:
> >> >
> >> > During an io.Copy() where the Writer is a TCPConn and the Reader is a
> 200K disk file, my code may concurrently Write() on the same TCPConn.
> >> >
> >> > I see the result of the Write() inserted into the result of the
> io.Copy(). I had the impression that was impossible, but I must be
> mistaken, as the sendfile(2) docs read:
> >> >
> >> > Note that a successful call to sendfile() may write fewer bytes than
> requested; the caller should be prepared to retry the call if there were
> unsent bytes.
> >> >
> >> > Could someone confirm that one must indeed synchronize concurrent use
> of tcpConn.Write() and io.Copy(tcpConn, file)?
> >>
> >> Synchronization should not be required. internal/poll.Sendfile
> >> acquires a write lock on dstFD, which is the TCP socket. That should
> >> ensure that the contents of an ordinary Write (which also acquires a
> >> write lock) should not interleave with the sendfile data.
> >>
> >> That said, if the sendfile system call cannot be used for whatever
> >> reason, the net package will fall back on doing ordinary Read and
> >> Write calls. And those Write calls can be interleaved with other
> >> Write calls done by a different goroutine. I think that is probably
> >> permitted, in that io.Copy doesn't promise to not interleave with
> >> simultaneous Write calls on the destination.
> >>
> >> So in the general case you should indeed use your own locking to avoid
> >> interleaving between io.Copy and a concurrent Write.
> >
> >
> > Thanks for the details. Where could I add a Println() to reveal why it
> doesn't call poll.Sendfile()?
> >
> > I expect this system to use sendfile(2). The file is a normal file on a
> local partition (running on a Digital Ocean Droplet).
> >
> >
> > /etc/fstab has:
> > UUID=[omitted] / ext4 defaults 1 1
> >
> >
> > $ df -h
> > Filesystem Size Used Avail Use% Mounted on
> > devtmpfs 981M 0 981M 0% /dev
> > tmpfs 996M 0 996M 0% /dev/shm
> > tmpfs 996M 436K 995M 1% /run
> > tmpfs 996M 0 996M 0% /sys/fs/cgroup
> > /dev/vda1 59G 5.7G 51G 11% /
> > tmpfs 200M 0 200M 0% /run/user/0
>
>
> Well, I don't know for that it doesn't use sendfile, I just can't
> explain the results you're seeing if it does use sendfile.
>
> The place to start is to find out why (or whether)
> internal/poll.Sendfile is returning 0, nil.
>
Found the invocations, wrong letter case...
$ grep -r 'SendFile(' /usr/local/go/src/
/usr/local/go/src/net/sendfile_windows.go: done, err :=
poll.SendFile(&fd.pfd, syscall.Handle(f.Fd()), n)
/usr/local/go/src/net/sendfile_linux.go: written, werr =
poll.SendFile(&c.pfd, int(fd), remain)
/usr/local/go/src/net/sendfile_unix_alt.go: written, werr =
poll.SendFile(&c.pfd, int(fd), pos, remain)
/usr/local/go/src/internal/poll/sendfile_windows.go:func SendFile(fd *FD,
src syscall.Handle, n int64) (int64, error) {
/usr/local/go/src/internal/poll/sendfile_linux.go:func SendFile(dstFD *FD,
src int, remain int64) (int64, error) {
/usr/local/go/src/internal/poll/sendfile_solaris.go:func SendFile(dstFD
*FD, src int, pos, remain int64) (int64, error) {
/usr/local/go/src/internal/poll/sendfile_bsd.go:func SendFile(dstFD *FD,
src int, pos, remain int64) (int64, error) {
$ grep -r 'sendFile(' /usr/local/go/src/net/
/usr/local/go/src/net/sendfile_windows.go:func sendFile(fd *netFD, r
io.Reader) (written int64, err error, handled bool) {
/usr/local/go/src/net/sendfile_linux.go:func sendFile(c *netFD, r
io.Reader) (written int64, err error, handled bool) {
/usr/local/go/src/net/sendfile_unix_alt.go:func sendFile(c *netFD, r
io.Reader) (written int64, err error, handled bool) {
/usr/local/go/src/net/sendfile_stub.go:func sendFile(c *netFD, r io.Reader)
(n int64, err error, handled bool) {
/usr/local/go/src/net/tcpsock_posix.go: if n, err, handled :=
sendFile(c.fd, r); handled {
Should I check the 'handled' result of sendFile() in net/tcpsock_posix.go ?
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/9076006e-92f2-4fca-bc2b-cf75e920b92f%40googlegroups.com.
