[tcpdump-workers] Linux mmap support and nonblocking mode

2014-07-23 Thread Aaron Lehmann
Hi,

I just spent a few days debugging unexpected packet loss in my
application. The first packet that passed the filter would often be
missed, but the response that came immediately after would always make
it through, as well as subsequent packets. This failure was inconsistent
and difficult to reproduce.

After trying several other things, I upgraded from pcap 1.1.0 to pcap
1.6.1. This made the issue much worse. With the new version, the first
packet was always lost instead of only sometimes, and future packets
would frequently get lost as well. I found that disabling the mmap
support worked around the problem.

I think I understand why this was happening, at least in the context of
TPACKET_V3. The issue is similar to #335. The kernel closes blocks after
a certain timeout, handing them over to userspace. This happens whether
the blocks contain any packets or not. My application uses libpcap in
nonblocking mode with a separate event loop, and it didn't get notified
of activity on the packet socket file descriptor when blocks expired.
Eventually, a packet would pass the filter, but by this point, all
blocks would be assigned to userspace, and the kernel would have nowhere
to store it.

Calling pcap_dispatch() from a timer seems to work around this by
returning the blocks to the kernel. I'm now setting the pcap timeout to
50 ms (I wasn't setting it before), and running a timer every 50 ms that
calls pcap_dispatch().

This looks more like a kernel issue than a libpcap issue to me, but I
would appreciate a warning in the documentation that pcap_dispatch()
needs to be called periodically, even if there's no activity on the
selectable fd. Of course, if you can think of a way to work around this
inside libpcap, that would be even better.

Thanks,
Aaron
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


[tcpdump-workers] libpcap BPF implementation issue

2014-09-02 Thread Aaron Lehmann
Hello,

A few weeks ago, I reported a problem with TPACKET v3 missing packets
when using libpcap in nonblocking mode. I managed to work around this
reliably, but I continued seeing missed packets on platforms which don't
support TPACKET v3.

The problem seems to be due to the way libpcap sometimes filters the
first few packets in the ring:

/* run filter on received packet
 * If the kernel filtering is enabled we need to run the
 * filter until all the frames present into the ring
 * at filter creation time are processed.
 * In this case, blocks_to_filter_in_userland is used
 * as a counter for the packet we need to filter.
 * Note: alternatively it could be possible to stop applying
 * the filter when the ring became empty, but it can possibly
 * happen a lot later... */
bp = frame + tp_mac;
if (handlep->filter_in_userland && handle->fcode.bf_insns &&
(bpf_filter(handle->fcode.bf_insns, bp,
tp_len, tp_snaplen) == 0))
return 0;

I found that the packets being missed were passing the kernel filter,
but bpf_filter() was returning 0 on these packets. My filter string is
generated as follows:

/* We have a somewhat complicated filter string to handle fragmented
   packets. Handle the following cases:

   * port %d: Unfragmented packet with the port number of interest
 (IPv4 or IPv6, TCP or UDP).

   * ip[6:2] & 0x3fff != 0: Fragmented IPv4 packet

   * Fragmented IPv6 packet: no direct check because the kernel BPF
 implementation can't iterate over the optional headers. Check for
 the following "next header" values in the fixed header:

- 0: Hop-by hop options (may precede fragment header)
- 60: Destination options (may precede fragment header)
- 43: Routing (may precede fragment header)
- 44: Fragment
*/

snprintf(filter_str, sizeof(filter_str), "port %d or ip[6:2] & 0x3fff 
!= 0 or ip6 proto 0 or ip6 proto 60 or ip6 proto 43 or ip6 proto 44", port);
 
Patching libpcap to always set blocks_to_filter_in_userland and
filter_in_userland to 0 successfully works around the problem. It seems
that libpcap's BPF implementation mishandles this filter string somehow.
I haven't been able to figure out what exactly goes wrong, but I'd happy
to do more debugging or provide packet captures if that would be
helpful. The packets involved are completely normal IPv4 UDP packets
with the destination port set to the port specified at the beginning of
the filter string.

Aaron
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers