[tcpdump-workers] Multiple interface capture and thread safety status in libpcap
Hi all, I need to perform packet sniffing on several interfaces at the same time. My natural approach would be to open a pcap_t object for each interface and place a "select" - considering Linux -call to deal with packet dispatching. My only constraint is that I have to treat the received packets in chronological order: indeed, I would like to process the data as it gets to the interfaces, without introducing any reordering. If I am not mistaken, it might be possible that a "select" call does not read data in temporal order, if multiple FDs are ready at the time the process is scheduled for running by the OS. Is that correct ? A work-around to this problem might be to move the capture on different threads: each thread has its own pthread_t object and captures traffic on a different interface. In this case, I do not have a clear picture about which parts of libpcap are thread-safe and which not (my version of reference is the 1.1.1); I have found really old posts about thread-safety issues in pcap_compile and pcap_setfilter (which I would need: 1 common filter for each thread) but nothing more. To sum up: 1) Could I/O multiplexing reorder packets during multiple interfaces sniffing ? 2) Which are the caveats to take into account when capturing from different interfaces using threads (1 thread per interface, same BPF filter for each thread) ? Which parts of libpcap are not thread-safe (v. >= 1.1.1) ? Thank you Alberto Balesena - This is the tcpdump-workers list. Visit https://cod.sandelman.ca/ to unsubscribe.
Re: [tcpdump-workers] Multiple interface capture and thread safety
On 10.05.2012 16:59, ri...@happyleptic.org wrote: -[ Thu, May 10, 2012 at 04:43:50PM +0200, Wiener Schnitzel ] I need to perform packet sniffing on several interfaces at the same time. My natural approach would be to open a pcap_t object for each interface and place a "select" - considering Linux -call to deal with packet dispatching. My only constraint is that I have to treat the received packets in chronological order: indeed, I would like to process the data as it gets to the interfaces, without introducing any reordering. If I am not mistaken, it might be possible that a "select" call does not read data in temporal order, if multiple FDs are ready at the time the process is scheduled for running by the OS. Is that correct ? Yes, but anyway as libpcap (and the kernel) will batch the packets sent to each pcap_t handle you will have some reordering taking place independantly of select(). Good to know. A work-around to this problem might be to move the capture on different threads: each thread has its own pthread_t object and captures traffic on a different interface. In this case, I do not have a clear picture about which parts of libpcap are thread-safe and which not (my version of reference is the 1.1.1); I have found really old posts about thread-safety issues in pcap_compile and pcap_setfilter (which I would need: 1 common filter for each thread) but nothing more. libpcap is mostly thread safe (I say mostly because I don't know for your version but I never encountered any issue with modern versions - and by modern I mean the version shipped with Debian stable). If you go with threads, you will still need to ensure that packets are processed in order, though. But do you really need to process packets received on distinct interfaces in timestamp order? What's the purpose? Are you talking about either reordering in libpcap or the merging of the packets ? As I have to deal with asymm. paths and perform flow analysis, I must ensure that the packets of a flow are analyzed in temporal order, no matter from which interface they came through. - This is the tcpdump-workers list. Visit https://cod.sandelman.ca/ to unsubscribe.
Re: [tcpdump-workers] Multiple interface capture and thread safety
On 11.05.2012 09:02, Guy Harris wrote: On May 10, 2012, at 7:43 AM, Wiener Schnitzel wrote: I need to perform packet sniffing on several interfaces at the same time. Are you processing packets from each interface independently, so that a packet on interface A is not looked at when processing packets from interface B, or are you processing the packets from all of the interfaces as a single stream, so that you need to see packets from multiple interfaces in order? At a certain point, I'd like to treat the packets as they came from a single source. Even if you're sniffing on *one* interface, I think that, with at least some versions of the Linux kernel, "as it gets to the interfaces" and "as it gets delivered to the PF_PACKET socket from which libpcap reads" are not necessarily the same thing on multi-core machines. I seem to remember that some people have seen packets with out-of-order timestamps, and have the impression that the problem is that if two packets are processed on different cores, the packet that arrived second might be queued up on the socket before the packet that arrived first if, for whatever reason, the thread on the second core manages to get its job done faster. I don't know whether this is still a problem with reasonably recent versions of the kernel. I'd would be very interested in this kind of details. Do you think it is documented somewhere ? Also, does that mean that PCAP timestamps are normally reliable (if the NIC cannot expose its own RX timestamp) ? Now, if you put all the pcap_t's into non-blocking mode, and pass a count of 1 to pcap_dispatch(), so it processes only one packet, or if you use pcap_next() or pcap_next_ex(), you could try reading from each of the interfaces, process the packet with the lowest time stamp, and, in the next trip through the loop, read another packet from the interface from which the packet you processed came and re-check the packets read previously from the other interfaces, you'd process the packets in time stamp order (modulo any out-of-order delivery from the kernel on any single interface). Nice suggestion. If each thread is processing packets independently, so that you don't have to worry about processing packets from multiple interfaces in the right order for all of those interfaces, then you could do it in one thread - for each call to pcap_dispatch(), do the processing for packets from the interface in question. Doing it in multiple threads would make better use of multiple cores in your application, however. If that's *not* the case, doing the capture in different threads still requires some scheme to process packets from different interfaces in order. I see. As I said, I might need to merge the data coming from the interfaces, so I need an algorithm to compare the age of packets with different sources. pcap_compile() uses YACC and Lex, or uses replacements thereof in YACC-compatible/Lex-compatible mode, so they're *not* thread safe - the lexical analyzer and parser have non-thread-safe state. pcap_setfilter(), however, should be thread-safe, and the rest of the APIs are thread-safe as long as any given pcap_t is only being processed in one thread at a time; Hence, if each thread wants to compile a different BPF, I need an external lock to the function. Otherwise, I can compile a shared BPF in the main thread and set it in the sniffing threads without any issues. Am I right ? Thanks - This is the tcpdump-workers list. Visit https://cod.sandelman.ca/ to unsubscribe.
