On Wed, Jan 17, 2018 at 7:20 AM, Sowmini Varadhan
<[email protected]> wrote:
> If the MSG_ZEROCOPY flag is specified with rds_sendmsg(), and,
> if the SO_ZEROCOPY socket option has been set on the PF_RDS socket,
> application pages sent down with rds_sendmsg() are pinned.
>
> The pinning uses the accounting infrastructure added by
> Commit a91dbff551a6 ("sock: ulimit on MSG_ZEROCOPY pages")
>
> The payload bytes in the message may not be modified for the
> duration that the message has been pinned. A multi-threaded
> application using this infrastructure may thus need to be notified
> about send-completion so that it can free/reuse the buffers
> passed to rds_sendmsg(). Notification of send-completion will
> identify each message-buffer by a cookie that the application
> must specify as ancillary data to rds_sendmsg().
> The ancillary data in this case has cmsg_level == SOL_RDS
> and cmsg_type == RDS_CMSG_ZCOPY_COOKIE.
>
> Signed-off-by: Sowmini Varadhan <[email protected]>
> ---
> include/uapi/linux/rds.h | 1 +
> net/rds/message.c | 44 +++++++++++++++++++++++++++++++++++++++++++-
> net/rds/rds.h | 3 ++-
> net/rds/send.c | 27 ++++++++++++++++++++++++---
> 4 files changed, 70 insertions(+), 5 deletions(-)
> +static int rds_cmsg_zcopy(struct rds_sock *rs, struct rds_message *rm,
> + struct cmsghdr *cmsg)
> +{
> + unsigned int *cookie;
Use fixed-width types across the ABI.
> +
> + if (cmsg->cmsg_len < CMSG_LEN(sizeof(*cookie)))
> + return -EINVAL;
> + cookie = CMSG_DATA(cmsg);
> + rm->data.op_mmp_znotifier->z_cookie = *cookie;
> + return 0;
> +}
> +
> @@ -1107,12 +1126,14 @@ int rds_sendmsg(struct socket *sock, struct msghdr
> *msg, size_t payload_len)
>
> /* Attach data to the rm */
> if (payload_len) {
> - rm->data.op_sg = rds_message_alloc_sgs(rm, ceil(payload_len,
> PAGE_SIZE));
> + int num_sgs = ceil(payload_len, PAGE_SIZE);
> +
> + rm->data.op_sg = rds_message_alloc_sgs(rm, num_sgs);
Unrelated change. Leftover from revising the patch?
Also, with user pages, data is no longer guaranteed to be page
aligned, so this may
need more sgs.
