On Thu, Jan 18, 2018 at 12:12 PM, Sowmini Varadhan <sowmini.varad...@oracle.com> wrote: > On (01/18/18 08:53), Eric Dumazet wrote: >> >> The thing is : MSG_PEEK 'support' will also need SO_PEEK_OFF support. > > sure, I'll drop the MSG_PEEK idea (which I wasnt very thrilled > about anyway) > >> So lets properly design things, and not re-use legacy stuff that is >> proven to be not multi-thread ready and too complex. >> >> If you want to design a new channel of communication, do it, and >> maintain it. > > My instinct is to go with the fixed size ancillary data- which itself > allows 2 options: > > 1. cmsg_data has a sock_extended_err preamble > with ee_origin = SO_EE_ORIGIN_ZEROCOPY_COOKIE (or similar), > and the ee_data is an array of 32 bit cookies (can pack at most 8 > 32-bit cookies, if we want to pack this into an skb->cb) > > Using the sock_extended_err as preamble will allow this to be usable by > existing tcp zcopy applications (they can use the ee_origin to find > out if this a batch of cookies or the existing hi/lo values). > > 2. If we have the option of passing completion-notification up as ancillary > data on the pollin/recvmsg channel itself (instead of MSG_ERRQUEUE)
This assumes a somewhat symmetric workload, where there are enough recv calls to reap the notification associated with the send calls. > we dont have to try to retain "backward compat" to the > SO_EE_ORIGIN_ZEROCOPY API: we can just use a completely new data > struct for the notification and potentially pack more cookies into > 48 bytes (RDS could be the first guinea pig for this- doesnt even > have to be done across all protocol families on day-1). > > I think the shmem channel suggestion would be an optional optimization > that can be added later- it may not even be necessary, since most > applications will likely be sending *and* receiving data, so passing up > cookies with recvmsg should be "good enough" to save syscall overhead > for the common case. > > I can work #2, if there are no objections to it. I would stay with MSG_ERRQUEUE processing. One option is to pass data up to userspace in the data portion of the notification skb instead of encoding it in ancillary data, like tcp_get_timestamping_opt_stats.
