[tcpdump-workers] libpcap timeout weirdness
Hi Everyone!
I am using the built in Libpcap library that comes with MacOSX. (MacOSX 10.8.3)
I have set up a background GCD dispatch queue, where the libpcap is set up and
waits for packets to arrive on my 10Gig card.
The code is very trivial, and appears as if it should works well..
Here is the problem.
I am using "pcap_loop" to grab a specific number of packets. This is set up
using a snap size of 1504 (To match the packet size from my hardware device.)
The buffer is large, about 1Mb in size. I have the "pcap_loop" code in
"blocking" mode, so that I can wait for all of the packets to finish, before
triggering a semaphore.
When I start the pcap_loop to do its work, which grabs exactly 1879 packets.
(Precisely the number I am asking for.) I end up with a timeout being called.
(I know this is a timeout, because if I change the timeout value in
"pcap_open_live", the timeout matches the duration that I set.) As a test, I
decide to trim the number of packets used in pcap_loop by 160, and everything
speeds up dramatically! Absolutely no timeouts. Anything less that 159 packets
(determined by trial & error), the time out occurs.
This isn't the only size, either.
Now I download another (smaller) set of data, this time I download only 782
packets. This download starts to timeout, as well! (Even though the 1700 or so,
packet downloaded just fine!) If I trim it by about 97 packet, the system no
longer times out.
Weird.
Just for fun, I tried an really large data set, at 3130 packets. As expected it
stalls. But again, trimming about 200 packets removes the stall, and it moves
the data extremely quickly.
If I use pcap_stats immediately after the pcap_loop, I see that in both fast
and timed out situations, that I don't drop any packets.
The only issue seems to be that the timeout is triggered.
I am at a loss on this one. I have tried changing the 'snaplen' size, and
altered the timeout, but i can't seem to see where the problem is.
This works if I trim the number of packets read, but slows down dramatically,
if I do the full amount of packets requested.
I have also switched over to pcap_create and the assorted pcap_set commands to
try to noodle with settings.
I have even set different buffersizes with sysctl debug.bpf_bufsize and
maxbufsize. No luck there, either.
Can anyone make a suggestion?
Should I ditch the built in MacOSX libpcap and compile the one available from
the tcpdump repository?
Attached is the relevant code:
char filter_exp[] = "ether proto 0x";
char errbuf[PCAP_ERRBUF_SIZE] = {0};
int buffersize = (int)(1048576);
//handle = pcap_open_live([ethernetInterface
cStringUsingEncoding:NSASCIIStringEncoding], 65536 * 2, 0, 1000, errbuf);
handle = pcap_create([ethernetInterface
cStringUsingEncoding:NSASCIIStringEncoding], errbuf);
if (handle != NULL) {
//pcap_set_promisc(handle, 0);
pcap_set_timeout(handle, 1000);
//pcap_setdirection(handle, PCAP_D_IN);
pcap_set_snaplen(handle, 1504);
pcap_set_buffer_size(handle, buffersize);
//pcap_set_rfmon(handle, 0);
pcap_activate(handle);
/* compile the filter expression */
bpf_u_int32 mask = 0x;
if (pcap_compile(handle, &fp, filter_exp, 1, mask) == -1) {
fprintf(stderr, "Couldn't parse filter %s: %s\n",
filter_exp, pcap_geterr(handle));
exit(EXIT_FAILURE);
}
/* apply the compiled filter */
if (pcap_setfilter(handle, &fp) == -1) {
fprintf(stderr, "Couldn't install filter %s: %s\n",
filter_exp, pcap_geterr(handle));
exit(EXIT_FAILURE);
}
return true;
}
---
int err = pcap_loop(handle, blockCount, processSingleImage, (u_char
*)&dataOp);
and
void processSingleImage(u_char *args, const struct pcap_pkthdr *header, const
u_char *packet)
{
GT_DataOperation *dataOp = (GT_DataOperation *)args;
static int count = 0;
count++;
}
Thanks!
bob.
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] libpcap timeout weirdness
Ok,
So I just stumbled into a "sort of" solution. I have to do a wide-spread test,
first, to see if this is really a fix:
I just set my timeout to 1ms using pcap_set_timeout,
I then added the following to my callback:
void processSingleImage(u_char *args, const struct pcap_pkthdr *header, const
u_char *packet)
{
GT_DataOperation *dataOp = (GT_DataOperation *)args;
static int count = 0;
count++;
if (count == rtoDataOp->totalBlockCount) {
count = 0;
pcap_breakloop(dataOp->handle);
}
}
Basically, I count blocks until I get everything that I need. At that point, I
do a breakloop.
This seems to eliminate the timeout issue I saw.
Now, however, I am concerned that the timeout is going to have to be adjusted
to some value that at is "One Size Fits All". Naturally a number that degrades
performance for all of my users.
I tried a timeout of 0, using pcap_set_timeout. But this ever called my
callback.
I am curious to know if anyone has any thoughts..
Thanks!
bob.
On Mar 24, 2013, at 7:51 PM, Robert Monaghan wrote:
> Hi Everyone!
>
> I am using the built in Libpcap library that comes with MacOSX. (MacOSX
> 10.8.3)
>
> I have set up a background GCD dispatch queue, where the libpcap is set up
> and waits for packets to arrive on my 10Gig card.
>
> The code is very trivial, and appears as if it should works well..
> Here is the problem.
>
> I am using "pcap_loop" to grab a specific number of packets. This is set up
> using a snap size of 1504 (To match the packet size from my hardware device.)
> The buffer is large, about 1Mb in size. I have the "pcap_loop" code in
> "blocking" mode, so that I can wait for all of the packets to finish, before
> triggering a semaphore.
>
> When I start the pcap_loop to do its work, which grabs exactly 1879 packets.
> (Precisely the number I am asking for.) I end up with a timeout being called.
> (I know this is a timeout, because if I change the timeout value in
> "pcap_open_live", the timeout matches the duration that I set.) As a test, I
> decide to trim the number of packets used in pcap_loop by 160, and everything
> speeds up dramatically! Absolutely no timeouts. Anything less that 159
> packets (determined by trial & error), the time out occurs.
>
> This isn't the only size, either.
>
> Now I download another (smaller) set of data, this time I download only 782
> packets. This download starts to timeout, as well! (Even though the 1700 or
> so, packet downloaded just fine!) If I trim it by about 97 packet, the
> system no longer times out.
>
> Weird.
>
> Just for fun, I tried an really large data set, at 3130 packets. As expected
> it stalls. But again, trimming about 200 packets removes the stall, and it
> moves the data extremely quickly.
>
> If I use pcap_stats immediately after the pcap_loop, I see that in both fast
> and timed out situations, that I don't drop any packets.
> The only issue seems to be that the timeout is triggered.
>
> I am at a loss on this one. I have tried changing the 'snaplen' size, and
> altered the timeout, but i can't seem to see where the problem is.
> This works if I trim the number of packets read, but slows down dramatically,
> if I do the full amount of packets requested.
> I have also switched over to pcap_create and the assorted pcap_set commands
> to try to noodle with settings.
> I have even set different buffersizes with sysctl debug.bpf_bufsize and
> maxbufsize. No luck there, either.
>
> Can anyone make a suggestion?
>
> Should I ditch the built in MacOSX libpcap and compile the one available from
> the tcpdump repository?
>
>
> Attached is the relevant code:
>
> char filter_exp[] = "ether proto 0x";
>
> char errbuf[PCAP_ERRBUF_SIZE] = {0};
> int buffersize = (int)(1048576);
> //handle = pcap_open_live([ethernetInterface
> cStringUsingEncoding:NSASCIIStringEncoding], 65536 * 2, 0, 1000, errbuf);
>
> handle = pcap_create([ethernetInterface
> cStringUsingEncoding:NSASCIIStringEncoding], errbuf);
>
> if (handle != NULL) {
> //pcap_set_promisc(handle, 0);
> pcap_set_timeout(handle, 1000);
> //pcap_setdirection(handle, PCAP_D_IN);
> pcap_set_snaplen(handle, 1504);
> pcap_set_buffer_size(handle, buffersize);
> //pcap_set_rfmon(handle, 0);
> pcap_activate(handle);
>
> /* compile the filter expression */
> bpf_u_int32 mask = 0x;
> if (pcap_compile(handle, &fp, filter_exp, 1, mask) == -1) {
> fprintf(stderr, "Couldn't parse filter %s: %s\n",
> filter_exp, pcap_geterr(handle));
> exit(EXIT_FAILURE);
> }
>
> /* apply the compiled filter */
> if (pcap_setfilter(handle, &f
Re: [tcpdump-workers] tcpdump 4.4 release candidate 1
> "Romain" == Romain Francoise writes: >> please expect a new release candidate on 2013-03-09, assuming I >> can get enough Internet in St.Johns. Romain> Not sure if this was St John's in Antigua or St John's in Romain> Canada, hopefully the former this time of year (but most Romain> likely the latter). :-) It was a 7000km of air travel for a 2300km trip thanks to snow at LGA. Very tiring. Romain> In any case the rc2 tarballs are nowhere to be found. Can we Romain> get an update, please? http://www.ca.tcpdump.org/beta/ -- ] Never tell me the odds! | ipv6 mesh networks [ ] Michael Richardson, Sandelman Software Works| network architect [ ] m...@sandelman.ca http://www.sandelman.ca/| ruby on rails[ ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] libpcap timeout weirdness
On Mar 24, 2013, at 12:18 PM, Robert Monaghan wrote: > Basically, I count blocks until I get everything that I need. At that point, > I do a breakloop. Is the total block count available at the time you call pcap_loop()? If so, just pass that in to pcap_loop() as the count. > This seems to eliminate the timeout issue I saw. The combination of the smaller timeout, and not trying to read more packets than will actually arrive (so that you don't try to read packets after the download finishes), is probably what's making the difference here. > Now, however, I am concerned that the timeout is going to have to be adjusted > to some value that at is "One Size Fits All". Naturally a number that > degrades performance for all of my users. If a low timeout (or the lowest possible timeout, i.e. immediate mode, as described in my previous message, namely "no timeout, just return packets immediately") causes a problem, BPF is probably not currently capable of providing what you need. If, however, the performance is OK in immediate mode, try that. One problem with immediate mode is that it's somewhat like a socket, in that each read gives you a single packet, but it's also *not* like a socket, in that there's not a lot of buffering for packets in immediate mode. Arguably, something like Linux's PF_PACKET sockets would be best for people trying to implement protocols atop the link-layer, as (either when not in memory-mapped mode, or when in TPACKET_V1 or TPACKET_V2 memory-mapped mode) it has no timeouts, but does have a buffer, so that you don't have to *immediately* read the packet or have further packets dropped due to being out of buffer space. (In TPACKET_V3 mode, it appears to work more like, err, umm, BPF, with entire buffers full of packets being delivered, and with a timeout to keep it from waiting forever for a buffer to fill up; I think that mode was introduce for the benefit of packet capture.) > I tried a timeout of 0, using pcap_set_timeout. But this ever called my > callback. In BPF, a timeout of 0 means "no timeout, just block until the store buffer fills up, even if that takes forever". ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] libpcap timeout weirdness
On Mar 24, 2013, at 2:59 PM, Guy Harris wrote: > Arguably, something like Linux's PF_PACKET sockets would be best for people > trying to implement protocols atop the link-layer, as (either when not in > memory-mapped mode, or when in TPACKET_V1 or TPACKET_V2 memory-mapped mode) > it has no timeouts, but does have a buffer, so that you don't have to > *immediately* read the packet or have further packets dropped due to being > out of buffer space. (In TPACKET_V3 mode, it appears to work more like, err, > umm, BPF, with entire buffers full of packets being delivered, and with a > timeout to keep it from waiting forever for a buffer to fill up; I think that > mode was introduce for the benefit of packet capture.) After a bit of a dive into xnu, it appears that there might be something *somewhat* similar to PF_PACKET sockets in OS X - PF_NDRV sockets. The documentation is somewhat, umm, sparse. See, for example, /usr/include/net/ndrv.h. You'd presumably open a socket with a protocol family of PF_NDRV and type SOCK_RAW, bind it to a network adapter (the struct sockaddr_ndrv structure has an interface name in it, and that's what you'd use in a bind; set the family to AF_NDRV), and then use setsockopt() calls to do the *real* bind, i.e. binding it to a particular protocol type. The socket level for setsockopt() would be SOL_NDRVPROTO, and the option would be NDRV_SETDMXSPEC to bind and NDRV_DELDMXSPEC to unbind. They both take a struct ndrv_protocol_desc as an argument. The version member of that structure should be set to NDRV_PROTOCOL_DESC_VERS; the protocol_family member should, I guess, be set to some number you pick to identify that protocol (maybe it's only used when unbinding), and the rest is a counted list of struct ndrv_demux_desc's, each of which specifies a link-layer protocol to bind to the socket. That structure has: type, which is an indication of the type of protocol specification: NDRV_DEMUXTYPE_ETHERTYPE - an Ethertype (which is what you'd use); NDRV_DEMUXTYPE_SAP - an 802.2 header (DSAP, SSAP, and 1st byte of the packet type); NDRV_DEMUXTYPE_SNAP - a SNAP type (OUI and protocol ID); all in network byte order; length, which is the length of the protocol specification; a union for the various protocol specifications. I have not tried any of this. https://github.com/okunnig-/Foobar/blob/master/main.c is a very simple example, but it doesn't do any protocol type binding. Googling for NDRV_DEMUXTYPE_ETHERTYPE might find some better examples. (What might be Really Nice, as I've said on occasion, would be a "access to particular link-layer packets" library, for use by programs implementing protocols atop the link layer (rather than by programs running as packet capture and/or injection tools), which would use whatever mechanisms are appropriate for that. Those mechanisms might be different from the ones used for packet capture: on Linux, packet capture might use TPACKET_V3 PF_PACKET/SOCK_RAW sockets not bound to a particular protocol type value, while protocols atop the link layer might use non-memory-mapped or TPACKET_V2 PF_PACKET/SOCK_DGRAM or SOCK_RAW sockets bound to a particular protocol type value; on AIX and Solaris 11, packet capture might use BPF while protocols atop the link-layer would use DLPI and bind to particular protocol type values; on other systems with DLPI, packet capture might use SAP-promiscuous DLPI devices while protocols atop the link-layer would use DLPI and bind to particular protocol type values; on OS X, packet capture might use BPF while protocols atop the link-layer would use PF_NDRV sockets; etc..) ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
