[tcpdump-workers] best strategy for receiving millions-packets-per-second

[Montorsi, Francesco via tcpdump-workers]

--- Begin Message ---

Hi all,

I hope this is the right mailing list for this kind of questions (also I 
couldn't find an "archive search" option so please forgive me if these 
questions have already been answered)...


I'm evaluating libpcap as an alternative to the DPDK framework 
(https://www.dpdk.org/); my use case is RX-only (no TX) in promiscuous mode, of 
wire traffic at speeds in the range [0-20] MPPS on 100Gbps NICs (note that the 
max theoretical PPS is 148 MPPS @64B frames on a 100Gbps link).


I wonder if you can give me pointers on :


a) what's the best way to load-balance received packets across a pool of 
threads / CPU cores: libpcap API is single-thread right?

Should I create a thread that calls pcap_next_ex(), computes some sort of 
RSS value on the packet and dispatches it to another thread of my app? Is this 
the best scheme?

   The problem with this approach is that the thread computing the RSS will 
typically be the bottleneck (since it cannot scale)


b) is using pcap_dispatch() a better approach compared to using pcap_next_ex() 
or it's just the same thing?



Thanks!


Francesco Montorsi
Principal Engineer - Diagnostix Technical Leader (EvA and ECC)
P: +39 059 7409350

www.empirix.com
twitter.com/empirix
linkedin.com/company/empirix

--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers

Re: [tcpdump-workers] best strategy for receiving millions-packets-per-second

[Montorsi, Francesco via tcpdump-workers]

--- Begin Message ---
Hi Anders,

> Why "as an alternative to the DPDK framework"? I think there is code to use
> DPDK in libpcap.
Actually there's no code in libpcap 1.9.1 to use DPDK... I just grepped through 
the tarball..

> I also think there has been efforts to use CP fanout and similar have you
> tried that?
I think you mean the PACKET_FANOUT_CPU option of  a Linux socket 
(http://man7.org/linux/man-pages/man7/packet.7.html) right?
Honestly I didn't see any reference in libpcap (I grepped :) )

THanks
Francesco
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers

Re: [tcpdump-workers] best strategy for receiving millions-packets-per-second

[Montorsi, Francesco via tcpdump-workers]

--- Begin Message ---

Hi Ray,

>On 04/05/2020 11:08, Montorsi, Francesco wrote:
>> a) what's the best way to load-balance received packets across a pool of 
>> threads / CPU cores: libpcap API is single-thread right?
>
>Assuming Linux (since you mentioned DPDK) 
yes, right  - I forgot to mention that.

>you can get per-packet
>load-balancing by forgetting libpcap and going straight for `AF_PACKET`
>and then use the ioctl `PACKET_FANOUT_CPU` to keep packets on whichever
>CPU core handles that RX queue.

Thanks for pointing this out, that was my same suspect.


>You need sufficient entropy on the incoming packets IP and port fields
>for the RSS algortihm to ensure effective per-queue balancing on the
>traffic.
>See slide 11 onwards from my presentation here:
>

Thanks I had a look.
My problem is that very often my application will receive packets encapsulated 
through GRE/ERSPAN or GTPU tunnelling protocols
so that the outer IP addresses that Linux will use for hashing and dispatching 
across queues risk to be always the same.
That would mean close-to-zero entropy and all packets dispatched to the same 
CPU core.
I guess I would use PACKET_FANOUT_LB to distribute traffic on a set of 
intermediate CPUs that would compute the hash using the 
tunnelled IP addresses instead of the "outer" ones and then redistribute 
packets over the actual worker threads...

Do you know if there is any plan to get AF_PACKET support in libpcap anytime 
soon?

Thanks,
Francesco
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers