Re: [tcpdump-workers] libpcap timeout weirdness
My current code (before using libpcap) uses this very method. I set up a raw socket, set the DEMUX type, and I capture traffic. The problem with this approach is that it seems to have a fair bit of overhead in the OS. (At least on the Mac.) I have to throttle traffic down my hardware to about 25 to 30% (ouch!) in order to get my data. Any higher, packets are dropped. Just by using libpcap/bpf, I have more than doubled the speed using Immediate mode along with my pcap_open_live session. So in this case, libpcap is the way forward for my application. But now, I want to work out a way to let pcap/bpf work at full rate, if possible. If I can balance the buffer sizes, I may be able to side-step the timeouts. - maybe. :) Thanks! bob On 2013-03-25, at 12:27 AM, Guy Harris wrote: > > On Mar 24, 2013, at 2:59 PM, Guy Harris wrote: > >> Arguably, something like Linux's PF_PACKET sockets would be best for people >> trying to implement protocols atop the link-layer, as (either when not in >> memory-mapped mode, or when in TPACKET_V1 or TPACKET_V2 memory-mapped mode) >> it has no timeouts, but does have a buffer, so that you don't have to >> *immediately* read the packet or have further packets dropped due to being >> out of buffer space. (In TPACKET_V3 mode, it appears to work more like, >> err, umm, BPF, with entire buffers full of packets being delivered, and with >> a timeout to keep it from waiting forever for a buffer to fill up; I think >> that mode was introduce for the benefit of packet capture.) > > After a bit of a dive into xnu, it appears that there might be something > *somewhat* similar to PF_PACKET sockets in OS X - PF_NDRV sockets. > > The documentation is somewhat, umm, sparse. See, for example, > /usr/include/net/ndrv.h. > > You'd presumably open a socket with a protocol family of PF_NDRV and type > SOCK_RAW, bind it to a network adapter (the struct sockaddr_ndrv structure > has an interface name in it, and that's what you'd use in a bind; set the > family to AF_NDRV), and then use setsockopt() calls to do the *real* bind, > i.e. binding it to a particular protocol type. > > The socket level for setsockopt() would be SOL_NDRVPROTO, and the option > would be NDRV_SETDMXSPEC to bind and NDRV_DELDMXSPEC to unbind. They both > take a struct ndrv_protocol_desc as an argument. The version member of that > structure should be set to NDRV_PROTOCOL_DESC_VERS; the protocol_family > member should, I guess, be set to some number you pick to identify that > protocol (maybe it's only used when unbinding), and the rest is a counted > list of struct ndrv_demux_desc's, each of which specifies a link-layer > protocol to bind to the socket. > > That structure has: > > type, which is an indication of the type of protocol specification: > > NDRV_DEMUXTYPE_ETHERTYPE - an Ethertype (which is what you'd > use); > > NDRV_DEMUXTYPE_SAP - an 802.2 header (DSAP, SSAP, and 1st byte > of the packet type); > > NDRV_DEMUXTYPE_SNAP - a SNAP type (OUI and protocol ID); > > all in network byte order; > > length, which is the length of the protocol specification; > > a union for the various protocol specifications. > > I have not tried any of this. > > https://github.com/okunnig-/Foobar/blob/master/main.c > > is a very simple example, but it doesn't do any protocol type binding. > Googling for NDRV_DEMUXTYPE_ETHERTYPE might find some better examples. > > (What might be Really Nice, as I've said on occasion, would be a "access to > particular link-layer packets" library, for use by programs implementing > protocols atop the link layer (rather than by programs running as packet > capture and/or injection tools), which would use whatever mechanisms are > appropriate for that. Those mechanisms might be different from the ones used > for packet capture: > > on Linux, packet capture might use TPACKET_V3 PF_PACKET/SOCK_RAW > sockets not bound to a particular protocol type value, while protocols atop > the link layer might use non-memory-mapped or TPACKET_V2 PF_PACKET/SOCK_DGRAM > or SOCK_RAW sockets bound to a particular protocol type value; > > on AIX and Solaris 11, packet capture might use BPF while protocols > atop the link-layer would use DLPI and bind to particular protocol type > values; > > on other systems with DLPI, packet capture might use SAP-promiscuous > DLPI devices while protocols atop the link-layer would use DLPI and bind to > particular protocol type values; > > on OS X, packet capture might use BPF while protocols atop the > link-layer would use PF_NDRV sockets; > > etc..) ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
[tcpdump-workers] Link-Layer Header Type request for USBPcap
Hello, For the USBPcap project I would like to request a new link-layer header type value: LINKTYPE_USBPCAP DLT_USBPCAP Capture format specification is available at the project website [1] and could be described as pseudo-header for USB packets captured using USBPcap on Microsoft Windows. Regards, Tomasz [1] http://desowin.org/usbpcap/captureformat.html ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Link-Layer Header Type request for USBPcap
On Mar 25, 2013, at 2:48 AM, Tomasz Moń wrote:
> For the USBPcap project I would like to request a new link-layer
> header type value:
> LINKTYPE_USBPCAP
> DLT_USBPCAP
>
> Capture format specification is available at the project website [1]
> and could be described as pseudo-header for USB packets captured using
> USBPcap on Microsoft Windows.
> #pragma pack(1)
> typedef struct
> {
> USHORT headerLen; /* This header length */
> UINT64 irpId; /* I/O Request packet ID */
So headerLen is at an offset of 0, and irpId is at an offset of 2, right?
> o irpId is merely a pointer to IRP casted to the UINT64
I.e., it's an 64-bit cookie whose bits have no particular significance, given
that, within a file, there's nothing for it to point to.
> o transfer determines the transfer type and thus the header type. See
> below for details.
> o dataLength specifies the total length of transfer data to follow
> directly after the header.
> Transfer-specific headers
Presumably the "transfer-specific header" follows the
USBPCAP_BUFFER_PACKET_HEADER in the packet data, with the transfer data
following the transfer-specific header.
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
[tcpdump-workers] Is user question allowed here?
is question about use of tcpdump allowed in this list? I am running linux in vmware and use tcpdump to capture all tcp packets send/receive by its virtual net card,the server and client are running at the same host . But ,after client send/receive several packets to/from server ,only to find 0 packet have been captured by tcpdump. My command is : tcpdump tcp what is the possible reason ? Thanks! ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Link-Layer Header Type request for USBPcap
On Mon, Mar 25, 2013 at 11:08 AM, Guy Harris wrote:
>> #pragma pack(1)
>> typedef struct
>> {
>> USHORT headerLen; /* This header length */
>> UINT64 irpId; /* I/O Request packet ID */
>
> So headerLen is at an offset of 0, and irpId is at an offset of 2, right?
Exactly.
>> o irpId is merely a pointer to IRP casted to the UINT64
> I.e., it's an 64-bit cookie whose bits have no particular significance, given
> that, within a file, there's nothing for it to point to.
Yes. It is provided so it could be used to pair the requests with
responses in analysis software.
>> o transfer determines the transfer type and thus the header type. See
>> below for details.
>> o dataLength specifies the total length of transfer data to follow
>> directly after the header.
>> Transfer-specific headers
>
> Presumably the "transfer-specific header" follows the
> USBPCAP_BUFFER_PACKET_HEADER in the packet data, with the transfer data
> following the transfer-specific header.
Actually, all transfer-specific headers inherit the base packet header.
But yes, first there is the USBPCAP_BUFFER_PACKET_HEADER, then (if
any) the transfer-specific header members not contained in
USBPCAP_BUFFER_PACKET_HEADER, and then the transfer data.
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] tcpdump 4.4 release candidate 1
On Mar 24, 2013, at 2:52 PM, Michael Richardson wrote: >Romain> In any case the rc2 tarballs are nowhere to be found. Can we >Romain> get an update, please? > > http://www.ca.tcpdump.org/beta/ 1.4.0rc2 looks good, although I just checked into the trunk, and propagated to the libpcap_1_4rel0 branch, a bug fix from Michal Labedzki to a problem with Bluetooth interfaces in 3.x kernels, along with a configuration script change from me to keep that from breaking libpcap builds on systems with 3.x kernels: https://github.com/mcr/libpcap/pull/24 and we might want to pick that up for the final release. (Michael, could you mark the pull request as closed? The actual pull isn't necessary, as I committed the patch along with the configuration script change.) 4.4.0rc2 appears to have stuff that's in the trunk but not the tcpdump_4_4rel0 branch, such as print-msnlb.c, print-vxlan.c, and print-otv.c, a change to print-bgp.c, and a change to the Makefile not to have an explicit list of tests but to run a find command to find the tests. Is that intentional? ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
[tcpdump-workers] libpcap: 64bit machine has no compatible 32-bit libpcap library
I want to compile a `.c` file to 32-bit executable using `gcc` option `-m32` with libpcap the machine is linux 64bit fedora 16 however, I get the following error [root@fdf source]# gcc -m32 -o test_tcp test_tcp.c -lpcap /usr/bin/ld: skipping incompatible /usr/lib64/libpcap.so when searching for -lpcap /usr/bin/ld: cannot find -lpcap collect2: ld returned 1 exit status how to solve this problem? thanks! ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Is user question allowed here?
> "ishare" == ishare writes: ishare> But ,after client send/receive several packets to/from ishare> server ,only to find 0 packet have been captured by tcpdump. ishare> My command is : tcpdump tcp you didn't use the -w option. ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] libpcap: 64bit machine has no compatible 32-bit libpcap library
> "wen" == wen lui writes: wen> I want to compile a `.c` file to 32-bit executable using `gcc` wen> option `-m32` with libpcap the machine is linux 64bit fedora 16 wen> how to solve this problem? thanks! install a 32-bit libpcap. I don't know how to do this on Fedora, but I'm sure it is possible. ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] tcpdump 4.4 release candidate 1
> "Guy" == Guy Harris writes: Romain> In any case the rc2 tarballs are nowhere to be found. Can we Romain> get an update, please? >> http://www.ca.tcpdump.org/beta/ Guy> 1.4.0rc2 looks good, although I just checked into the trunk, libpcap 1.4.0rc3 packaged with the bt patch. Guy> 4.4.0rc2 appears to have stuff that's in the trunk but not the Guy> tcpdump_4_4rel0 branch, such as print-msnlb.c, print-vxlan.c, Guy> and print-otv.c, a change to print-bgp.c, and a change to the Guy> Makefile not to have an explicit list of tests but to run a Guy> find command to find the tests. Is that intentional? huh? It should all be there. For about ten minutes I had a 4.4.0rc2 which did not have print-msnlb.c. tcpdump-4.4.0rc3 repackaged just to be sure. -- ] Never tell me the odds! | ipv6 mesh networks [ ] Michael Richardson, Sandelman Software Works| network architect [ ] m...@sandelman.ca http://www.sandelman.ca/| ruby on rails[ ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] tcpdump 4.4 release candidate 1
> 4.4.0rc2 appears to have stuff that's in the trunk but not the > tcpdump_4_4rel0 branch, such as print-msnlb.c, print-vxlan.c, and > print-otv.c, a change to print-bgp.c, and a change to the Makefile not to > have an explicit list of tests but to run a find command to find the tests. > Is that intentional? > There are some fixes to the recent otv, vxlan and msnlb commits in tcpdump pull requests 56 and 57. -- Denis Ovsienko ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] tcpdump 4.4 release candidate 1
On Sun, Mar 24, 2013 at 05:52:51PM -0400, Michael Richardson wrote: > > > "Romain" == Romain Francoise writes: > >> please expect a new release candidate on 2013-03-09, assuming I > >> can get enough Internet in St.Johns. > > Romain> Not sure if this was St John's in Antigua or St John's in > Romain> Canada, hopefully the former this time of year (but most > Romain> likely the latter). :-) > > It was a 7000km of air travel for a 2300km trip thanks to snow at LGA. > Very tiring. > > Romain> In any case the rc2 tarballs are nowhere to be found. Can we > Romain> get an update, please? > > http://www.ca.tcpdump.org/beta/ Picking a random spot in this thread to chime in but I've heard reports via co-workers that hitting ^C while handling multiple input files via 'find | tcpdump -V -' doesn't work as expected. I'm trying to find time to look into this before the 4.4.0 release. -- WXS ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] tcpdump 4.4 release candidate 1
On Mar 25, 2013, at 7:55 AM, Michael Richardson wrote: >Guy> 4.4.0rc2 appears to have stuff that's in the trunk but not the >Guy> tcpdump_4_4rel0 branch, such as print-msnlb.c, print-vxlan.c, >Guy> and print-otv.c, a change to print-bgp.c, and a change to the >Guy> Makefile not to have an explicit list of tests but to run a >Guy> find command to find the tests. Is that intentional? > > huh? It should all be there. In the tarball, or in the branch? That stuff *is* in the tarball. The issue is that it's not in the tcpdump_4_4rel0 branch, or, at least, not in my repository cloned from your repository and then set to the tcpdump_4_4rel0 branch. If it's supposed to be in 4.4, as I infer is the case, then the branch is wrong. Currently, the only differences I see between the rc3 tarballs and the trunk is the libpcap VERSION file (the tcpdump VERSION files both say "4.4.0"). Should we make new libpcap_1.4 and tcpdump_4.4 branches from the trunk, so that they reflect what's in the rc3 tarballs? (And then make new libpcap_1.4.0 and tcpdump_4.4.0 branches when the final 1.4.0 and 4.4.0 tarballs are made.) ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] libpcap: 64bit machine has no compatible 32-bit libpcap library
On Mar 25, 2013, at 7:22 AM, Michael Richardson wrote:
>> "wen" == wen lui writes:
>wen> I want to compile a `.c` file to 32-bit executable using `gcc`
>wen> option `-m32` with libpcap the machine is linux 64bit fedora 16
>
>wen> how to solve this problem? thanks!
>
> install a 32-bit libpcap.
> I don't know how to do this on Fedora, but I'm sure it is possible.
This post:
http://forums.fedoraforum.org/showpost.php?p=1486052&postcount=2
says
I'm far a linux guru myself, but I'll tell you how stumbled onto
installing the 32-bit libraries. Some other distros have an obvious option to
add it, but Fedora doesn't make it obvious.
Go into the GUI add/remove software app and, in the options somwhere,
uncheck the option to only show "native packages", which will then allow the
.i686 32-bit packages to become visible from the repos.
Now, search for "libc" or "glibc" (I can't remember which I used) and
choose the 32-bit, i686 library package. That should bring in most of the
core/base stuff. You may need some more packages after that for whatever you're
trying to do, but that will get you started.
The original question for that post was asked about Fedora 15, but
experimenting with my Fedora 16 with KDE seems to indicate that it applies
there as well. In its add/remove software app (Apper), the instruction set
architecture for a package ("i686" for 32-bit and "x86_64" for 64-bit; your
mileage may vary if you're not using an x86 machine) only shows up, in a
tooltip, when you hover the mouse pointer over an item.
I didn't try installing any of the 32-bit packages (you'd need both the "user"
and "developer" packages), but I presume that installing 32-bit versions of all
the "core/base stuff", along with 32-bit versions of libpcap and libpcap-devel,
should do the job.
Note, by the way, that 32-bit libpcap will *NOT* work atop a 64-bit kernel if
your kernel version is less than 2.6.27 and greater than whatever version first
included the memory-mapping support for PF_PACKET sockets - those kernel
versions only have the TPACKET_V1 layout for the memory-mapped buffer, and that
one is different in 32-bit and 64-bit kernels. 2.6.27 introduces TPACKET_V2,
in which the layout is the same for 32-bit and 64-bit kernels, and 32-bit code
dealing with memory-mapped PF_PACKET sockets works fine atop 32-bit or 64-bit
kernels.
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
