[tcpdump-workers] ipv6 addresses in pcap_addr

[Chris Morgan]

Hello.

I wasn't sure if this question was answered anywhere, I've searched
via google and looked on the mailing lists but haven't seen any
answers.

How does one get a ipv6 address from the sockaddr struct pointed to by
pcap_addr? I have devices with addresses of type AF_INET6 but since
the sockaddr struct is defined to have only 14 bytes of protocol
address how does one get a 16 byte ipv6 address out of this 14 byte
structure?

Chris
-
This is the tcpdump-workers list.
Visit https://cod.sandelman.ca/ to unsubscribe.

Re: [tcpdump-workers] ipv6 addresses in pcap_addr

[Chris Morgan]

On Thu, Jan 22, 2009 at 7:59 PM, Guy Harris  wrote:
>
> On Jan 22, 2009, at 4:05 PM, Chris Morgan wrote:
>
>> Hello.
>>
>> I wasn't sure if this question was answered anywhere, I've searched
>> via google and looked on the mailing lists but haven't seen any
>> answers.
>>
>> How does one get a ipv6 address from the sockaddr struct pointed to by
>> pcap_addr?
>
> Welcome to the wonderful world of C, the programming language that combines
> the power of assembly language with the convenience of assembly language
>
> "struct sockaddr" is a sort of place-holder structure.  There are few, if
> any, real "struct sockaddr" structures out there; instead, there are "struct
> sockaddr_XXX" structures, for various values of XXX.  "struct sockaddr" is
> an abstraction that covers all of them; all "struct sockaddr_XXX" structures
> begin with members that are the same as the non-"sa_data" members of "struct
> sockaddr", and with the same structure offsets as those members, and have,
> following that, the address-type-specific data, starting at the same offset
> as the "sa_data" member of "struct sockaddr", but not necessarily having the
> same *length* as the "sa_data" member of that structure - it could be longer
> (and possibly shorter).
>

Doh. I know exactly what you mean now. I didn't notice anything on the
pcap man page about that, probably because its something documented
elsewhere. I had no idea that the sockaddr structure was used this
way. Thank you for the tip and sorry for causing you to write such an
extensive post, I appreciate your help.

Chris
-
This is the tcpdump-workers list.
Visit https://cod.sandelman.ca/ to unsubscribe.

[tcpdump-workers] Hardware mac address with pcap/winpcap

[Chris Morgan]

I'm working on a pcap/winpcap c# library, SharpPcap. I was wondering
what the best way to get a devices mac address was, preferrably in a
manner that would work in both pcap and with winpcap. So far I've seen
code that captures packets on the interface and looks in them for the
mac address, I've also seen code that sends an arp packet I guess
using the adapter ip address. I wanted to see if these were the best
ways, if anyone had any recommendations.

Chris
-
This is the tcpdump-workers list.
Visit https://cod.sandelman.ca/ to unsubscribe.

Re: [tcpdump-workers] Hardware mac address with pcap/winpcap

[Chris Morgan]

On Tue, Mar 3, 2009 at 8:49 PM, Michael Richardson
 wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
>
>>>>>> "Chris" == Chris Morgan  writes:
>    Chris> I'm working on a pcap/winpcap c# library, SharpPcap. I was
>    Chris> wondering what the best way to get a devices mac address was,
>    Chris> preferrably in a manner that would work in both pcap and with
>    Chris> winpcap. So far I've seen code that captures packets on the
>
>  Which device?
>  Are you looking for your MAC address, or a remote one?
>

I would be looking for the local adapter mac addresses.  Under linux
with pcap and the adapters I have, ethernet and wireless, I see
hardware mac addresses in pcap_if_t.addresses. I wasn't sure if there
were any known cases where pcap_if_t.addresses wouldn't have the
hardware mac address, like  with winpcap, particular devices etc.
Users have reported difficulty getting the adapter mac addresses so it
seemed useful to ask the experts.


>  Why does it matter if this is a library for C#...
>

Oh, I was just giving some background.

Chris
-
This is the tcpdump-workers list.
Visit https://cod.sandelman.ca/ to unsubscribe.

Re: [tcpdump-workers] Hardware mac address with pcap/winpcap

[Chris Morgan]

On Tue, Mar 3, 2009 at 9:54 PM, Guy Harris  wrote:
>
> On Mar 3, 2009, at 6:44 PM, Chris Morgan wrote:
>
>> I would be looking for the local adapter mac addresses.  Under linux
>> with pcap and the adapters I have, ethernet and wireless, I see
>> hardware mac addresses in pcap_if_t.addresses. I wasn't sure if there
>> were any known cases where pcap_if_t.addresses wouldn't have the
>> hardware mac address, like  with winpcap
>
> ...or with libpcap on other flavors of UN*X.
>
> Whether pcap_findalldevs() returns MAC addresses or not is
> platform-dependent - in part because what they're *called* (i.e., what the
> sa_family value for a link-layer address is) is platform-dependent, and in
> part because the OS routine that pcap_findalldevs() uses to fetch the
> addresses is not only platform-dependent but also OS-version-dependent for
> some OSes, and the set of addresses it happens to return can differ between
> platforms.
>
> For any interface that has one or more IPv4 or IPv6 addresses assigned to
> it, you should get those addresses from pcap_findalldevs().  No other
> addresses are guaranteed to be in that list, and it's not guaranteed how you
> even tell whether one of those addresses *is* a MAC address.
>

Right. I had to look at the sa_family value to differentiate the two
under linux but I don't believe windows even has the same values for
sa_family.


>> Users have reported difficulty getting the adapter mac addresses
>
> Having done a little research when I saw your first mail, it's *definitely*
> platform-dependent.  A future version of libpcap may provide it, but not
> necessarily in the results of pcap_findalldevs() - there might be some other
> API or APIs providing a collection of adapter properties, including IPv4
> addresses, IPv6 addresses, link-layer addresses of various sorts, etc..

Hmm. Right now SharpPcap is platform independent, it should work
nearly the same on windows, mac and linux. We used to have windows
specific calls to a windows dll to retrieve adapter information but
that wouldn't work on other platforms so I removed it rather than
present code that only works sometimes. Some users have expressed the
desire to get the hardware mac addresses, I think so they can send
packets, but maybe this will be filled in by the kernel or the
hardware.

Is the development of pcap such that such a feature might be present
in the next several months? Even something that would work on WinXP
and beyond and Linux with a newish kernel would be great, although I
don't assume that pcap and winpcap are developed by the same people.

Chris
-
This is the tcpdump-workers list.
Visit https://cod.sandelman.ca/ to unsubscribe.

Re: [tcpdump-workers] Hardware mac address with pcap/winpcap

[Chris Morgan]

On Wed, Mar 4, 2009 at 1:21 PM, Guy Harris  wrote:
>
> On Mar 4, 2009, at 9:19 AM, Gianluca Varenni wrote:
>
>> In the case of Windows/WinPcap, we have an internal Packet API to get the
>> MAC address, the main problem is exposing such MAC address at the pcap API
>> level. I actually didn't know that findalldevs was returning the MAC address
>> on (some flavors of?) linux. What is the sa_family in that case?
>
> PF_PACKET, it appears; I suspect that means the address is a sockaddr_ll.
>

I can confirm that it is PF_PACKET on linux and that the values are
sockaddr_ll. It took quite a bit of searching to connect the dots,
there isn't a lot of info on the net about PF_PACKET sockaddr entries
and how to interpret them.

Chris
-
This is the tcpdump-workers list.
Visit https://cod.sandelman.ca/ to unsubscribe.

[tcpdump-workers] pcap_next/pcap_dispatch on VMware vmnet device not timing out

[Chris Morgan]

Opening a live capture as root (using sudo), on a vmware bridge device
on Linux 2.6.27, using a timeout of 1000ms. I'm seeing pcap_next() and
pcap_dispatch() getting stuck reading, no timeouts are occurring. Is
there a robust and efficient way of reading packets that won't block
forever like this?  I'm doing this from that c# library so while I've
thought of using poll() on the devices file descriptor I wanted to
avoid doing low level calls like this so the code would work with
winpcap/pcap on windows/linux platforms.

I wrote a simple c app, using an example from
http://www.tcpdump.org/pcap.htm, so I had a really simple way of
reproducing the issue. I can post this if it would be helpful to
confirm that I'm not doing anything really dumb.

Chris
-
This is the tcpdump-workers list.
Visit https://cod.sandelman.ca/ to unsubscribe.

Re: [tcpdump-workers] pcap_next/pcap_dispatch on VMware vmnet device

[Chris Morgan]

On Mon, Mar 9, 2009 at 7:51 PM, Guy Harris  wrote:
>
> On Mar 9, 2009, at 4:10 PM, Chris Morgan wrote:
>
>> Opening a live capture as root (using sudo), on a vmware bridge device
>> on Linux 2.6.27, using a timeout of 1000ms. I'm seeing pcap_next() and
>> pcap_dispatch() getting stuck reading, no timeouts are occurring. Is
>> there a robust and efficient way of reading packets that won't block
>> forever like this?
>
> Well, the first question is "why is blocking forever an issue?"
>
> Is the application also going to, for example, accept input from other
> sources while it's reading captured packets?
>

Well I noticed the issue when I was trying to shut the capture down gracefully.

Blocking may not be a huge issue. We can forcefully terminate the
capture thread if it doesn't respond within a period of time to a flag
that indicates it should shutdown, it just wasn't my first thought.
One nagging question is the issue of passing the pointer of a managed
function into pcap. I'm wondering if terminating the thread will
result in additional events from pcap_dispatch(). The man page
indicates this is possible with pcap_loop() and pcap_breakloop() so we
might as well take care to not have the handler memory be garbage
collected too early.


>> I'm doing this from that c# library so while I've
>> thought of using poll() on the devices file descriptor I wanted to
>> avoid doing low level calls like this so the code would work with
>> winpcap/pcap on windows/linux platforms.
>
> At least in the case where the app is going to accept input from other
> sources, that's what select(), poll(),
> WaitForMultipleObjects/MsgWaitForMultipleObjects, etc. are intended for -
> they'll wake up if packets arrive or if one of the other sources of input
> have input available, rather than having a hack wherein one is (incorrectly)
> expecting the timer for the timeout to
>
>        1) exist (which it doesn't, on some platforms)
>
> and
>
>        2) start timing when you start the read rather than when and *if* the
> first packet arrives (which it doesn't, on some platforms, e.g. Solaris; the
> Solaris timer starts when the first packet is seen, and, at least according
> to the bufmod(7) man page, restarts it on each packet - the man page says
>
>        To ensure that messages do not languish forever in an accumulating
> chunk, bufmod maintains a read timeout. Whenever this timeout expires, the
> module closes off the current chunk and passes it upward. The module
> restarts the timeout period when it receives a read side data message and a
> timeout is not currently active. These two rules insure that bufmod
> minimizes the number of chunks it produces during periods of intense message
> activity and that it periodically disposes of all messages during slack
> intervals, but avoids any timeout overhead when there is no activity.
>
>           indicating that the purpose of the timeout is *not* to enable
> polling, but to allow batching of packets without having packets remain
> undelivered for an indefinite period of time).
>
> Yes, it's platform-dependent, so you might have to find some way to hide
> that beneath your C# class.
>
> Given that, note also that there is at least one UN*X where, unfortunately,
> poll() *doesn't* work on the descriptor you get for a pcap_t - Mac OS X,
> where, unfortunately, in Mac OS X 10.4 and 10.5, poll() doesn't work on
> *any* character special files, including not only the BPF devices that
> libpcap uses but also ttys and pseudo-ttys (a source of a number of problems
> with various bits of software).  select() *does* work on them;
> unfortunately, a lot of apps - and libraries such as GLib, as used by GTK+,
> for example - use poll() in their event loops.
>
> There are also issues with select() and poll() and BPF devices, wherein you
> don't get a wakeup if the read completes due to the timer expiring, so you
> need to
>
>        1) put the descriptor for the BPF device in non-blocking mode;
>
>        2) use a timeout in select() or poll();
>
>        3) read from the BPF device when select() or poll() returns,
> *regardless* of whether select() or poll() indicates that a read is
> possible.
> -
> This is the tcpdump-workers list.
> Visit https://cod.sandelman.ca/ to unsubscribe.
>

We currently create a thread for each capture device and call back the
user code when packets arrive. Does this remove the requirement for
buffering with timeout since we'll pass the packets off after each
arrives?

Is the timeout something checked after each packet is received? Is
this why pcap_next() is blocking, because no packets are arriving?

Chris
-
This is the tcpdump-workers list.
Visit https://cod.sandelman.ca/ to unsubscribe.

Re: [tcpdump-workers] pcap_next/pcap_dispatch on VMware vmnet

[Chris Morgan]

On Tue, Mar 10, 2009 at 3:32 AM, Guy Harris  wrote:
> Chris Morgan wrote:
>> On Mon, Mar 9, 2009 at 7:51 PM, Guy Harris  wrote:
>>> Well, the first question is "why is blocking forever an issue?"
>>>
>>> Is the application also going to, for example, accept input from other
>>> sources while it's reading captured packets?
>>
>> Well I noticed the issue when I was trying to shut the capture down
>> gracefully.
>>
>> Blocking may not be a huge issue. We can forcefully terminate the
>> capture thread if it doesn't respond within a period of time to a flag
>> that indicates it should shutdown, it just wasn't my first thought.
>
> On UN*X systems, there are two possibilities I can think of offhand:
>
>    1) if the shutdown done as the result of a signal, one could arrange
> that the signal interrupt system calls and, in the signal handler, use
> pcap_breakloop() - the system call reading packets will return an
> error of EINTR (interrupted system call) when the signal handler
> returns, and (at least in newer versions of libpcap that have
> pcap_breakloop()) will, when they see that error, check whether the
> "break the loop" flag is set and break out of the pcap_loop() or
> pcap_dispatch() loop.
>
>    2) a poll() or select() loop (but not poll() on OS X!) with a timeout
> could be used, and the flag checked whenever select() or poll()
> returns.  (That will also work with a signal, as select() or poll()
> will give an error of EINTR.)
>

Does mac osx have epoll? I'm leaning towards using the poll()/select()
approach with timeout vs. sending signals but I can't really say why.
I guess I know less about which signal to send and the potential side
effects of sending signals.



> If you want one thread to be able to shutdown the capturing in another
> thread that's running the capture loop, that might be harder.  From a
> quick look at the Single UNIX Specification, one possibility might be that
> all threads other than the capturing thread could block a signal such as
> SIGUSR1 or SIGUSR2, the thread doing the capturing could catch that signal
> and call pcap_breakloop(), and the thread trying to shutdown the capturing
> could send that signal to the process with kill().
>
> A similar scheme to the one described in the previous paragraph might work
> on Windows - use WaitForSingleObjectEx() or WaitForMultipleObjectsEx(),
> with the "bAlertable" argument being TRUE,  and with QueueUserAPC() used
> to queue for the target thread an APC that calls pcap_breakloop().
>

You mentioned that Linux doesn't support a timeout, I see that the
call used by libpcap is a recvmsg() call. Does Windows have a timeout
on the packet read or do we need a os specific way of checking things
there as well?


>> One nagging question is the issue of passing the pointer of a managed
>> function into pcap. I'm wondering if terminating the thread will
>> result in additional events from pcap_dispatch(). The man page
>> indicates this is possible with pcap_loop() and pcap_breakloop() so we
>> might as well take care to not have the handler memory be garbage
>> collected too early.
>
> Yes.
>
>> We currently create a thread for each capture device and call back the
>> user code when packets arrive. Does this remove the requirement for
>> buffering with timeout since we'll pass the packets off after each
>> arrives?
>
> What do you mean by "requirement for buffering with timeout"?
>

Because we pass each packet off as we receive it we don't buffer
internal to SharpPcap. I thought that for performance reasons
Wireshark would group packets together and upon reaching a high water
mark or exceeding some interval, that the group of packets would be
passed somewhere. Maybe I'm not understanding your initial comments
about buffering but not holding onto packets forever if the buffer
never fills up.


>> Is the timeout something checked after each packet is received?
>
> The timeout is implemented in the OS code that libpcap uses, not in
> libpcap itself.  Some platforms don't have a timeout at all (e.g., Linux);
> some platforms have a timer that's started when libpcap (or whatever code
> is reading the packets) starts reading (*BSD, Mac OS X, Tru64 UNIX,
> Windows with WinPcap); some platforms have a timer that's started when a
> packet arrives (Solaris).
>

I know these may be dumb questions since if they made sense they would
likely have been implemented already.

Is there some reason not to implement pcap_breakloop() such that it
will send a signal to wake up the pcap_dispatch() thread?  I suppose
that opens a huge can of worms since now you are dealing wi

Re: [tcpdump-workers] pcap_next/pcap_dispatch on VMware vmnet

[Chris Morgan]

On Tue, Mar 10, 2009 at 8:02 PM, Guy Harris  wrote:
>
> On Mar 10, 2009, at 9:52 AM, Chris Morgan wrote:
>
>> Does mac osx have epoll?
>
> No.  It has poll(), but that, as noted, doesn't work with character special
> files, such as the BPF devices used for traffic capture in OS X.  (BPF
> devices *do* work with poll() in *BSD.)
>

Hmm. Yeah I'll make sure to put in a comment about mac os support.


>> You mentioned that Linux doesn't support a timeout, I see that the
>> call used by libpcap is a recvmsg() call. Does Windows have a timeout
>> on the packet read
>
> Yes.  It's similar to the BPF timeout, in that the timer starts when the
> read is done (rather than when the first packet arrives, as is the case on
> Solaris).
>

Cool. That should mean that we don't need to we don't need to do
anything special on Windows.


>> Because we pass each packet off as we receive it we don't buffer
>> internal to SharpPcap. I thought that for performance reasons
>> Wireshark would group packets together and upon reaching a high water
>> mark or exceeding some interval, that the group of packets would be
>> passed somewhere. Maybe I'm not understanding your initial comments
>> about buffering but not holding onto packets forever if the buffer
>> never fills up.
>
> For performance reasons, some in-kernel packet capture mechanisms, such as:
>
>        BPF, as used in *BSD and Mac OS X
>
>        the bufmod STREAMS module in Solaris (which is one component of the
> mechanism);
>
>        the packetfilter mechanism in Tru64 UNIX;
>
>        the WinPcap kernel code;
>
> will *NOT* return from a read as soon as the first packet arrives; they
> will, instead, try to accumulate multiple packets and hand all of them up in
> a single read.
>
> Obviously, if you get one packet an hour, you don't want to have the
> application wait several hours until enough packets have been accumulated,
> so those mechanisms also include a timer, so that, after enough time
> expires, the read will return even if a bufferful of packets hasn't been
> accumulated.
>
> This isn't done by Wireshark, or, rather, by dumpcap - and it isn't done by
> tcpdump, or any other such application, either.
>
> It's not even done by libpcap; the applications just call pcap_open_live(),
> passing it a timeout value, and pcap_open_live() passes that timeout value
> to the kernel packet capture mechanism, if it supports a timeout.
>
> In BPF and WinPcap, and, I think, in packetfilter, the timer starts when the
> read starts, and the read will return after the timer expires even if *no*
> packets have been accumulated.
>
> In Solaris, the timer starts - and is reset - whenever a packet arrives, so
> the read won't return until at least one packet has been accumulated.
>
> On those platforms, the buffering will be done regardless of what SharpPcap
> does, as it's not something that the caller of libpcap/WinPcap does - it's
> done by the kernel-mode code that libpcap and the WinPcap user-mode code
> use.
>

Right. I was thinking that there might be cases where if the timeout
wasn't supported you would end up with the os buffering packets and
never returning them to the caller.

How does it work if there is no timeout and packets get captured but
not enough for the callback to occur?



>> Is there some reason not to implement pcap_breakloop() such that it
>> will send a signal to wake up the pcap_dispatch() thread?
>
> Well, one reason not to do that might be that there *isn't* necessarily a
> separate thread doing pcap_dispatch() or pcap_loop() or
>
>> I suppose
>> that opens a huge can of worms since now you are dealing with cross
>> platform thread synchronization issues.
>
> Yes.  The code would be significantly different between UN*X and Windows.
>
> If there were a pcap_breakloop_thread() operation, which would take a pcap_t
> * and a thread, the UN*X version could use pthread_kill() to send a signal
> to the thread and the Windows version could use QueueUserAPC() to queue an
> APC for the thread.
>
> That would require that WinPcap use a WaitFor...Ex() call; I'd have to see
> whether it already uses a WaitFor...() call or not.
>

Ahh. Considering the delay in adding such a feature and getting a
release out etc it seems simpler to just handle the issue at the
application level.



>> Alternatively why not use poll() or select() inside of pcap_dispatch()
>> to avoid performing the read if there is no pending data?
>
> Because, in many cases, you don't *want* to avoid performing the read if
> there's no pending data - you want to

Re: [tcpdump-workers] pcap_next/pcap_dispatch on VMware vmnet

[Chris Morgan]

On Tue, Mar 10, 2009 at 9:01 PM, Guy Harris  wrote:
>
> On Mar 10, 2009, at 5:40 PM, Chris Morgan wrote:
>
>> Hmm. Yeah I'll make sure to put in a comment about mac os support.
>
> Note that select() *does* work with BPF devices on OS X - modulo the
> traditional BPF bug wherein the timer starts when a read is done, *not* when
> a select() is done, so you *could* block forever.  At this point I think all
> the *BSDs have fixed that in their BPF implementations, but OS X hasn't
> fixed it yet.
>

Yeah, I've heard that OS X has a lot of bugs that are reported but not
fixed so it doesn't surprise me.


> (If BPF had done Solaris-style timeouts, where the timer is an inter-packet
> timer, this wouldn't have been an issue in the first place.  It might also
> mean that the timer would have done a better job of batching packets if
> they're arriving fast and delivering them when there's a pause.)
>
>> Right. I was thinking that there might be cases where if the timeout
>> wasn't supported you would end up with the os buffering packets and
>> never returning them to the caller.
>
> No.  If the timeout isn't supported, the OS doesn't buffer any packets - it
> delivers them immediately, even if that means that a burst of packets is
> delivered one packet at a time, with a trip to the kernel done for each
> packet.
>

Ahh. That makes sense.



>> How does it work if there is no timeout and packets get captured but
>> not enough for the callback to occur?
>
> If there's no timeout, that's because there's no in-kernel buffering to
> require a timeout, so "enough for the callout to occur" is one packet.
>
>> Wouldn't using poll if the timeout was set and not if it wasn't solve
>> that problem?
>
> No, because tcpdump and dumpcap *do* want a timeout, because they don't want
> to, for example, block for several hours if you only get one packet an hour.
>  They don't care whether the read blocks forever waiting for the *first*
> packet to arrive, they just don't want it to block forever waiting for a
> bufferful of packets to arrive.
>

Right.


>>
>>
>>
>>
>>>> Wouldn't
>>>> that let you implement the timeout functionality
>>>
>>> To what timeout functionality are you referring?  A timeout that prevents
>>> reads from blocking forever even if no packets arrive is, as noted, not
>>> something that all applications need or want - that's *NOT* what the
>>> timeout
>>> specified in pcap_open_live() is intended to do, and not what it was ever
>>> guaranteed to do - and a timeout that just prevents packets from being
>>> buffered indefinitely if they're not arriving fast enough is *already*
>>> implemented.
>>> -
>>
>> Right, I'm referring to the to_ms timeout in pcap_open_live().
>
> That timeout is, at least on Solaris, "a timeout that just prevents packets
> from being buffered indefinitely if they're not arriving fast enough."  On
> some other OSes, it also happens to prevent reads from blocking forever if
> no packets arrive, but applications should not be depending on that unless
> they also call uname() and abort if the OS name string is "SunOS", so that
> they ensure that they never run on Solaris.
>
>> I see
>> what you mean, the man page certainly disclaims support for the
>> timeout on all platforms. The documentation on to_ms does sound like
>> exactly what I'm trying to do though, cause the read to timeout.
>
> I tried as hard as I could to write the documentation *NOT* to sound like
> that.  The problem is that I didn't mention that until the description of
> pcap_dispatch():
>
>       NOTE: when reading a live capture, pcap_dispatch() will not
> necessarily
>       return  when  the  read  times out; on some platforms, the read
> timeout
>       isn't supported, and, on other platforms, the timer doesn't start
> until
>       at  least  one packet arrives.  This means that the read timeout
> should
>       NOT be used in, for example, an interactive application, to  allow
>  the
>       packet capture loop to ``poll'' for user input periodically, as
> there's
>       no  guarantee  that  pcap_dispatch()  will  return  after  the
>  timeout
>       expires.
>
> In libpcap 1.x, the pcap man page discusses the timeout in some detail, and
> does explicitly note that it's *NOT* guaranteed to keep reads from blocking
> forever:
>
>       read timeout
>              If, when capturing,  packets  are  delivered  as  soo

[tcpdump-workers] Is libpcap pcap_set_buffer_size() == winpcap pcap_setbuff() ?

[Chris Morgan]

Hello.

A user of Sharppcap is asking if we support pcap_setbuff(). Apparently
this is a winpcap specific option. I was wondering if
pcap_set_buffer_size() was the same as pcap_setbuff(). If so, are
there any plans to unify the api for increased cross platform code
portability?

Chris
-
This is the tcpdump-workers list.
Visit https://cod.sandelman.ca/ to unsubscribe.

Re: [tcpdump-workers] Is libpcap pcap_set_buffer_size() == winpcap

[Chris Morgan]

On Thu, Sep 3, 2009 at 1:04 PM, Guy Harris wrote:
>
> On Sep 3, 2009, at 9:13 AM, Chris Morgan wrote:
>
>> A user of Sharppcap is asking if we support pcap_setbuff(). Apparently
>> this is a winpcap specific option.
>
> Yes.
>
> The problem is that not all platforms atop which libpcap runs can support
> setting the buffer size after you've opened a network interface for
> capturing - BPF won't let you change the buffer size on a /dev/bpf* device
> once you've bound it to an interface.
>
> The WinPcap people added pcap_setbuff(), but the code whose buffer size it
> changes is their code, so they could make it work however they wanted; the
> capture code libpcap uses is part of the UN*X systems on which it runs.
>
>> I was wondering if pcap_set_buffer_size() was the same as pcap_setbuff().
>
> "The same" in what sense?
>
> They are used differently.  Libpcap 1.x, in order to allow more options to
> be specified when a network interface is opened for capturing, split
> pcap_open_live() into pcap_create(), which creates a "non-activated" pcap_t,
> on which options can be set but upon which capturing cannot be done, and
> pcap_activate(), which "activates" the pcap_t so that you can capture on it.
>
> One option that can be set between creation and activation is the buffer
> size; that even works on systems that use BPF for capturing, as the
> /dev/bpf* device isn't opened, much less bound to an interface, until the
> pcap_t is activated.
>
> So, to set the buffer size when you open an interface, you do
>
>        pd = pcap_create(...);
>        if (pd == NULL)
>                fail;
>
>                ...
>
>        status = pcap_set_buffer_size(pd, buffer_size);
>        if (status != 0)
>                fail;
>
>                ...
>
>        status = pcap_activate(pd);
>        if (status != 0)
>                fail;
>
> pcap_setbuff() takes an opened pcap_t as an argument, so it can only be
> called *after* the interface has been opened, so, to set the buffer size on
> Windows after you open an interface, you do
>
>        pd = pcap_open_live(...);
>        if (pd == NULL)
>                fail;
>
>        if (pcap_setbuff(pd, buffer_size) == -1)
>                fail;
>
> or, in WinPcap 4.1 (at least as of 4.1b5 - I don't know which version first
> picked up pcap_create() and pcap_activate()):
>
>        pd = pcap_create(...);
>        if (pd == NULL)
>                fail;
>
>                ...
>
>        status = pcap_activate(pd);
>        if (status == 0)
>                fail;
>
>        if (pcap_setbuff(pd, buffer_size) == -1)
>                fail;
>
>> If so, are there any plans to unify the api for increased cross platform
>> code
>> portability?
>
> WinPcap 4.1 (again, at least as of 4.1b5) has pcap_set_buffer_size(), so you
> can do
>
>        pd = pcap_create(...);
>        if (pd == NULL)
>                fail;
>
>                ...
>
>        status = pcap_set_buffer_size(pd, buffer_size);
>        if (status != 0)
>                fail;
>
>                ...
>
>        status = pcap_activate(pd);
>        if (status != 0)
>                fail;
>
> on Windows with WinPcap 4.1 and on UN*Xes if you have libpcap 1.x.
>
> libpcap will not pick up pcap_setbuff() as it cannot be implemented on all
> platforms (no *BSD, AIX, or Mac OS X) and as it has pcap_set_buffer_size().
> -
> This is the tcpdump-workers list.
> Visit https://cod.sandelman.ca/ to unsubscribe.
>

Ahh.

Thank you again for this detailed information.

I'm asking the user if pcap_set_buffer_size() will work for them. If
it does we can implement that interface and we'll be able to have the
same api that works the same across windows, mac, linux platforms,
keeping things simple for everyone.

Chris
-
This is the tcpdump-workers list.
Visit https://cod.sandelman.ca/ to unsubscribe.

Re: [tcpdump-workers] Is libpcap pcap_set_buffer_size() == winpcap

[Chris Morgan]

On Thu, Sep 3, 2009 at 1:53 PM, Guy Harris wrote:
>
> On Sep 3, 2009, at 10:20 AM, Chris Morgan wrote:
>
>> I'm asking the user if pcap_set_buffer_size() will work for them. If
>> it does we can implement that interface and we'll be able to have the
>> same api that works the same across windows
>
> ...with WinPcap 4.1 or later...
>
>> mac
>
> ...with SnowLeopard or later...
>
>> linux
>
> ...with versions of distributions that have picked up libpcap 1.0 or
> later...
>
> http://oswatershed.com seems to indicate that some distributions - Arch
> Linux, Sabayon, and Ubuntu - may have 1.0.0 in their current versions.
>
> It also seems to indicate that FreeBSD has picked up 1.0.0; it looks as if
> that's only in 8.0-CURRENT, though.
> -
> This is the tcpdump-workers list.
> Visit https://cod.sandelman.ca/ to unsubscribe.
>


Yeah. Requiring the latest versions is a pain but it is a good first
step to get the support in place even if it requires the latest
versions.

Chris
-
This is the tcpdump-workers list.
Visit https://cod.sandelman.ca/ to unsubscribe.

[tcpdump-workers] How to tell if application is handling packets too slowly, causing them to be missed?

[Chris Morgan]

I have a case where it appears that packets are being missed or
dropped. I wonder if this is due to too much processing being done in
the pcap_dispatch() handler in my application in cases where there are
bursts of packets like facebook chat messages or website visits.

My question is how to detect this situation at runtime or via a test
case. I thought of using pcap_stats() but I wasn't sure if
pcap_stat.ps_drop was the number of packets that were dropped only due
to buffer overruns due to the application callback not processing
packets quick enough, or if this count referred to duplicate/error
packets and/or packets dropped due to buffer overruns.

If I could identify the issue at runtime that would be even better
since it would avoid cases where optimization now might not be good
enough if more packet analyzers are added to my code in the future,
causing things to again slow down and packets to be dropped.

Chris
-
This is the tcpdump-workers list.
Visit https://cod.sandelman.ca/ to unsubscribe.

Re: [tcpdump-workers] How to tell if application is handling packets

[Chris Morgan]

On Sat, Dec 26, 2009 at 5:02 PM, Guy Harris  wrote:
>
> On Dec 26, 2009, at 9:27 AM, Chris Morgan wrote:
>
>> I have a case where it appears that packets are being missed or
>> dropped. I wonder if this is due to too much processing being done in
>> the pcap_dispatch() handler in my application in cases where there are
>> bursts of packets like facebook chat messages or website visits.
>>
>> My question is how to detect this situation at runtime or via a test
>> case. I thought of using pcap_stats() but I wasn't sure if
>> pcap_stat.ps_drop was the number of packets that were dropped only due
>> to buffer overruns due to the application callback not processing
>> packets quick enough, or if this count referred to duplicate/error
>> packets and/or packets dropped due to buffer overruns.
>
> On what version of OS are you capturing the traffic (for a Linux 
> distribution, give the kernel version), and with what version of libpcap are 
> you doing this?  ps_drop is *supposed* to reflect only packets dropped due to 
> buffer overruns, but
>

Ahh, so ps_drop might work for this.

Users are reporting issues on Windows with the latest winpcap release
but I do a lot of my testing under Linux, Ubuntu 9.10, 2.6.31 x64. I'd
be doing the drop testing under Linux initially. On Linux its pcap
version 2.4 from pcap.h although the package is listed as version 0.8.


>        1) on some OSes, the capture mechanism doesn't make that information 
> available;
>
>        2) there were, I think, bugs in some versions of libpcap on some 
> platforms that caused ps_drop not to correctly reflect that.
>
> I don't *think* any of those bugs caused it to count packets dropped due to 
> network errors, and libpcap and the capture mechanisms it uses don't drop 
> duplicate packets.-
> This is the tcpdump-workers list.
> Visit https://cod.sandelman.ca/ to unsubscribe.
>


Chris
-
This is the tcpdump-workers list.
Visit https://cod.sandelman.ca/ to unsubscribe.

Re: [tcpdump-workers] How to tell if application is handling packets

[Chris Morgan]

On Sat, Dec 26, 2009 at 6:53 PM, Guy Harris  wrote:
>
> On Dec 26, 2009, at 3:13 PM, Chris Morgan wrote:
>
>> Ahh, so ps_drop might work for this.
>>
>> Users are reporting issues on Windows with the latest winpcap release
>> but I do a lot of my testing under Linux, Ubuntu 9.10, 2.6.31 x64. I'd
>> be doing the drop testing under Linux initially. On Linux its pcap
>> version 2.4 from pcap.h although the package is listed as version 0.8.
>
> Actually, it's file format version 2.4.  Somewhat confusingly, 
> PCAP_VERSION_MAJOR and PCAP_VERSION_MINOR are the major and minor version 
> numbers of the pcap file format, not of the pcap library.
>

Doh. I've attached a patch that adds a short comment to describe those
values so I, and maybe others, don't run into the same issue again.


> The best way to get the version number of the libpcap library is probably to 
> run "tcpdump -h":
>
>        $ tcpdump -h
>        tcpdump version 4.0.0
>        libpcap version 1.0.0
>        Usage: tcpdump [-aAdDefIKlLnNOpqRStuUvxX] [ -B size ] [ -c count ]
>                        [ -C file_size ] [ -E algo:secret ] [ -F file ] [ -G 
> seconds ]
>                        [ -i interface ] [ -M secret ] [ -r file ]
>                        [ -s snaplen ] [ -T type ] [ -w file ] [ -W filecount ]
>                        [ -y datalinktype ] [ -z command ] [ -Z user ]
>                        [ expression ]
>
> and, yes, even with a package listed as version 0.8, the libpcap version 
> might be something else - for some reason, Debian (and Ubuntu) call it 
> "libpcap 0.8" even though they've upgraded to a later version of the library.
>

Ahh. That's pretty silly of them. I should report that as a ubuntu or
debian bug...

cmor...@cmorgan-laptop:~/sharppcap_git/Test/bin/Debug$ tcpdump -h
tcpdump version 4.0.0
libpcap version 1.0.0


> Alternatively, if libpcap on your system has the pcap_lib_version() routine, 
> it returns a pointer to a character string giving the version number of 
> libpcap, so your application could use that (that's what tcpdump uses, if 
> available).  Otherwise, the library might define an external variable 
> "pcap_version", which is a char array ("extern char pcap_version[]") with a 
> string containing a version number for libpcap (again, that's what tcpdump 
> uses if it's available and pcap_lib_version() isn't).-
> This is the tcpdump-workers list.
> Visit https://cod.sandelman.ca/ to unsubscribe.
>

Yes, right. I was aware of pcap_lib_version() but it required more
effort to call that routine than look in the header file and get
confused :-)


Chris
From 56cdf207128dedf64acf2d6e57af93c4044793c0 Mon Sep 17 00:00:00 2001
From: Chris Morgan 
Date: Sat, 26 Dec 2009 22:37:15 -0500
Subject: [PATCH] Add a comment to describe PCAP_VERSION_MAJOR and PCAP_VERSION_MINOR as the file format version to avoid confusion with the library version

---
 pcap/pcap.h |1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/pcap/pcap.h b/pcap/pcap.h
index 5a571c6..61f0823 100644
--- a/pcap/pcap.h
+++ b/pcap/pcap.h
@@ -57,6 +57,7 @@
 extern "C" {
 #endif
 
+/* Pcap file format version, for the version of the library use the pcap_lib_version() function */
 #define PCAP_VERSION_MAJOR 2
 #define PCAP_VERSION_MINOR 4
 
-- 
1.6.3.3

-
This is the tcpdump-workers list.
Visit https://cod.sandelman.ca/ to unsubscribe.

Re: [tcpdump-workers] Release schedule?

[Chris Morgan]

I've got a couple of Oss projects that I release and ended up writing
some bash scripts that I keep in a subdir of the code repo that build
source and release packages given a version number. They even use git
log to build the release notes from a tag.

I can share if interested since I had the same issue of time consuming
and inconsistent naming until I automated it.

Chris


On Thursday, April 1, 2010, Michael Richardson  wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
>
>> "Darren" == Darren Reed  writes:
>     >> >> The links on http://www.tcpdump.org are broken.
>
>     Guy> The tarballs are libpcap-1.1.tar.gz and tcpdump-4.1.tar.gz,
>     Guy> rather than libpcap-1.1.0.tar.gz and tcpdump-4.1.0.tar.gz.  Are
>     Guy> we now calling the major releases 1.x and 4.x rather than 1.x.0
>     Guy> and 4.x.0, or are we continuing the convention of ".
>     >> Oops, see what happens when the non-regular guy does the release.
>     >> I'll fix them!
>
>     Darren> And now that the .tar.gz files are "1.1.0" and "4.1.0", the
>     Darren> directories inside are still "1.1" and "4.1." A fix for this
>     Darren> would be nice.
>
> okay, I will redo it all this weekend.
> It will be called 1.1.1 and 4.1.1.
>
> - --
> ]       He who is tired of Weird Al is tired of life!           |  firewalls  
> [
> ]   Michael Richardson, Sandelman Software Works, Ottawa, ON    |net 
> architect[
> ] m...@sandelman.ottawa.on.ca http://www.sandelman.ottawa.on.ca/ |device 
> driver[
>    Kyoto Plus: watch the video 
>                        then sign the petition.
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v1.4.9 (GNU/Linux)
> Comment: Finger me for keys
>
> iQEVAwUBS7Sjg4CLcPvd0N1lAQKBEggAs/dY3GxSvskczE5rGCblfu0vBwMFbSu0
> vpLuxD8NuIcRFtI6XKwPnuuM61gTFaqbFnRpBQcIaWEEbRZ2d/6Cm8lqeTtno3oW
> yMWEAZKoMc23KANph7H6SVawZwUYlOgZA5vk0MyayC2UNY6+UqSgAe+vIS9yZ8ab
> EhRv0I52DhO+3gCf58pRlkikvC9ZHiwDmSllXfSFIuufY1G8TyVLqoHD6jQpdhG4
> 7bPzaQlfMMi+0vCHLUDWbvWuvJGKNTqqlI0kT0KLI4Nvae+pSgf8M8wBWGUkQWmf
> KM+x3J2K1hIJsYvKDrJIiF6G/m709fVgdL+q/57nA23RrEwaOnDBhA==
> =jjra
> -END PGP SIGNATURE-
> -
> This is the tcpdump-workers list.
> Visit https://cod.sandelman.ca/ to unsubscribe.
>
-
This is the tcpdump-workers list.
Visit https://cod.sandelman.ca/ to unsubscribe.

Re: [tcpdump-workers] Running TCPDUMP over a web interface

[Chris Morgan]

Depending on the method of usage it should be reasonable to whip up a
php page with some jquery to handle the requests. I'm not sure about
how the the information from tcpdump would be conveyed, either via a
file or stdout, and what kinds of ways the user would interact with
the system, via a pseudo-shell, check boxes and some partial free form
text etc but it should be do able with off the shelf pieces and some
glue code.

Chris


On Sun, Aug 14, 2011 at 9:51 AM, Tek Bahadur Limbu  wrote:
> Hi all,
>
> I am not sure if this is a right list to post the following question.
>
> I need to run TCPDUMP on a Linux bridge with multiple network interfaces.
> However, instead of using a shell, I need to run it over a web interface.
>
> Any guide or suggestion will be highly appreciated.
>
>
> Thanking you...
> Best regards,
> Tek Bahadur Limbu
> -
> This is the tcpdump-workers list.
> Visit https://cod.sandelman.ca/ to unsubscribe.
>
-
This is the tcpdump-workers list.
Visit https://cod.sandelman.ca/ to unsubscribe.