Re: splice() performance for TCP socket forwarding

2018-12-13 Thread Eric Dumazet
On 12/13/2018 06:03 AM, Eric Dumazet wrote: > > > On 12/13/2018 05:33 AM, Marek Majkowski wrote: >> Ok, 4.19 does seem to kinda fix the SO_RCVLOWAT with splice, but I >> don't fully understand it: >> >> fcntl(8, F_SETPIPE_SZ, 1048576) = 1048576 <0.33> >> setsockopt(4, SOL_SOCKET, S

Re: splice() performance for TCP socket forwarding

2018-12-13 Thread Willy Tarreau
On Thu, Dec 13, 2018 at 02:17:20PM +0100, Marek Majkowski wrote: > > splice code will be expensive if less than 1MB is present in receive queue. > > I'm not sure what you are suggesting. I'm just shuffling data between > two sockets. Is there a better buffer size value? Is it possible to > keep sp

Re: splice() performance for TCP socket forwarding

2018-12-13 Thread Eric Dumazet
On 12/13/2018 05:33 AM, Marek Majkowski wrote: > Ok, 4.19 does seem to kinda fix the SO_RCVLOWAT with splice, but I > don't fully understand it: > > fcntl(8, F_SETPIPE_SZ, 1048576) = 1048576 <0.33> > setsockopt(4, SOL_SOCKET, SO_RCVLOWAT, [131072], 4) = 0 <0.14> > splice(4, NULL

Re: splice() performance for TCP socket forwarding

2018-12-13 Thread Willy Tarreau
Hi Eric! On Thu, Dec 13, 2018 at 05:37:11AM -0800, Eric Dumazet wrote: > Maybe mlx5 driver is in LRO mode, packing TCP payload in 4K pages ? I could be wrong but I don't think so : I remember having been used to LRO on myri10ge a decade ago giving me good performance which would degrade with conc

Re: splice() performance for TCP socket forwarding

2018-12-13 Thread Eric Dumazet
On 12/13/2018 04:55 AM, Willy Tarreau wrote: > > It's quite strange, it doesn't match at all what I'm used to. In haproxy > we're using splicing as well between sockets, and for medium to large > objects we always get much better performance with splicing than without. > 3 years ago during a t

Re: splice() performance for TCP socket forwarding

2018-12-13 Thread Marek Majkowski
Ok, 4.19 does seem to kinda fix the SO_RCVLOWAT with splice, but I don't fully understand it: fcntl(8, F_SETPIPE_SZ, 1048576) = 1048576 <0.33> setsockopt(4, SOL_SOCKET, SO_RCVLOWAT, [131072], 4) = 0 <0.14> splice(4, NULL, 9, NULL, 1048576, SPLICE_F_MOVE) = 121435 <71.039385> splice

Re: splice() performance for TCP socket forwarding

2018-12-13 Thread Marek Majkowski
On Thu, Dec 13, 2018 at 2:17 PM Marek Majkowski wrote: > > Eric, > > On Thu, Dec 13, 2018 at 1:49 PM Eric Dumazet wrote: > > On 12/13/2018 03:25 AM, Marek Majkowski wrote: > > > Hi! > > > > > > I'm basically trying to do TCP splicing in Linux. I'm focusing on > > > performance of the simplest cas

Re: splice() performance for TCP socket forwarding

2018-12-13 Thread Marek Majkowski
Eric, On Thu, Dec 13, 2018 at 1:49 PM Eric Dumazet wrote: > On 12/13/2018 03:25 AM, Marek Majkowski wrote: > > Hi! > > > > I'm basically trying to do TCP splicing in Linux. I'm focusing on > > performance of the simplest case: receive data from one TCP socket, > > write data to another TCP socket

Re: splice() performance for TCP socket forwarding

2018-12-13 Thread Willy Tarreau
Hi Marek, On Thu, Dec 13, 2018 at 12:25:20PM +0100, Marek Majkowski wrote: > Hi! > > I'm basically trying to do TCP splicing in Linux. I'm focusing on > performance of the simplest case: receive data from one TCP socket, > write data to another TCP socket. I get poor performance with splice. > >

Re: splice() performance for TCP socket forwarding

2018-12-13 Thread Eric Dumazet
On 12/13/2018 03:25 AM, Marek Majkowski wrote: > Hi! > > I'm basically trying to do TCP splicing in Linux. I'm focusing on > performance of the simplest case: receive data from one TCP socket, > write data to another TCP socket. I get poor performance with splice. > > First, the naive code, pr