[tcpdump-workers] Linux mmap support and nonblocking mode
Hi, I just spent a few days debugging unexpected packet loss in my application. The first packet that passed the filter would often be missed, but the response that came immediately after would always make it through, as well as subsequent packets. This failure was inconsistent and difficult to reproduce. After trying several other things, I upgraded from pcap 1.1.0 to pcap 1.6.1. This made the issue much worse. With the new version, the first packet was always lost instead of only sometimes, and future packets would frequently get lost as well. I found that disabling the mmap support worked around the problem. I think I understand why this was happening, at least in the context of TPACKET_V3. The issue is similar to #335. The kernel closes blocks after a certain timeout, handing them over to userspace. This happens whether the blocks contain any packets or not. My application uses libpcap in nonblocking mode with a separate event loop, and it didn't get notified of activity on the packet socket file descriptor when blocks expired. Eventually, a packet would pass the filter, but by this point, all blocks would be assigned to userspace, and the kernel would have nowhere to store it. Calling pcap_dispatch() from a timer seems to work around this by returning the blocks to the kernel. I'm now setting the pcap timeout to 50 ms (I wasn't setting it before), and running a timer every 50 ms that calls pcap_dispatch(). This looks more like a kernel issue than a libpcap issue to me, but I would appreciate a warning in the documentation that pcap_dispatch() needs to be called periodically, even if there's no activity on the selectable fd. Of course, if you can think of a way to work around this inside libpcap, that would be even better. Thanks, Aaron ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
[tcpdump-workers] libpcap BPF implementation issue
Hello, A few weeks ago, I reported a problem with TPACKET v3 missing packets when using libpcap in nonblocking mode. I managed to work around this reliably, but I continued seeing missed packets on platforms which don't support TPACKET v3. The problem seems to be due to the way libpcap sometimes filters the first few packets in the ring: /* run filter on received packet * If the kernel filtering is enabled we need to run the * filter until all the frames present into the ring * at filter creation time are processed. * In this case, blocks_to_filter_in_userland is used * as a counter for the packet we need to filter. * Note: alternatively it could be possible to stop applying * the filter when the ring became empty, but it can possibly * happen a lot later... */ bp = frame + tp_mac; if (handlep->filter_in_userland && handle->fcode.bf_insns && (bpf_filter(handle->fcode.bf_insns, bp, tp_len, tp_snaplen) == 0)) return 0; I found that the packets being missed were passing the kernel filter, but bpf_filter() was returning 0 on these packets. My filter string is generated as follows: /* We have a somewhat complicated filter string to handle fragmented packets. Handle the following cases: * port %d: Unfragmented packet with the port number of interest (IPv4 or IPv6, TCP or UDP). * ip[6:2] & 0x3fff != 0: Fragmented IPv4 packet * Fragmented IPv6 packet: no direct check because the kernel BPF implementation can't iterate over the optional headers. Check for the following "next header" values in the fixed header: - 0: Hop-by hop options (may precede fragment header) - 60: Destination options (may precede fragment header) - 43: Routing (may precede fragment header) - 44: Fragment */ snprintf(filter_str, sizeof(filter_str), "port %d or ip[6:2] & 0x3fff != 0 or ip6 proto 0 or ip6 proto 60 or ip6 proto 43 or ip6 proto 44", port); Patching libpcap to always set blocks_to_filter_in_userland and filter_in_userland to 0 successfully works around the problem. It seems that libpcap's BPF implementation mishandles this filter string somehow. I haven't been able to figure out what exactly goes wrong, but I'd happy to do more debugging or provide packet captures if that would be helpful. The packets involved are completely normal IPv4 UDP packets with the destination port set to the port specified at the beginning of the filter string. Aaron ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers