Re: [tcpdump-workers] Libpcap timeout settings in tcpdump - too long when printing to a terminal?
On Thu, Jan 08, 2015 at 06:57:08PM -0800, Guy Harris wrote: > With TPACKET_V3 support, Linux users are discovering what those of us using > BSD-flavored OSes have known for quite a while: > > > http://askubuntu.com/questions/570885/can-tcpdump-on-ubuntu-14-04-show-packets-in-real-time > > Tcpdump uses a timeout of 1 second when opening a capture device; this can, on > platforms that do BPF-style buffering, cause a 1-second delay before packets > show up. TPACKET_V3 is the first Linux capture mechanism to do BPF-style > buffering, so when a Linux user switches to a distribution version that > introduces a kernel that supports TPACKET_V3 and a libpcap that uses > TPACKET_V3, they'll start seeing delays that they haven't seen before. Is there any rational behind 1s value. Not saying it is wrong, just curious. > > The longer timeout can reduce capturing overhead, and if you're capturing a > high volume of traffic to a file, it's probably the right timeout to have. > If, however, you're printing packets to the console, you're probably doomed if > it's a high volume of traffic, and may want less of a delay if it's a low > volume of traffic. Always there will be cases in which default setting are just not good enough. I think that users doing high volume captures are minority so I'd not optimize defaults for them. > > Should we reduce the timeout if -w isn't specified - or do so if -w isn't > specified *and* if we're outputting to a terminal (isatty(1) returns a > non-zero value)? Should we use immediate mode if libpcap supports it? Should > we offer a command-line option to override the default? (I'd prefer to have > the default be appropriate for most cases, which may mean a different default > if printing packets or if printing them to the console rather than a file, > but, in addition to that, a command-line option to allow more sophisticated > users to tweak the timeout might be useful.) I don't think using immediate mode is the right thing to do, because it defeats the purpose of buffering done by kernel. Can't we use new default timeout value (lower) if we detect TPACKET_V3, so we don't defeat the purpose of buffering entirely but delay is short enough that output looks realtime-ish to human observer looking at the console. If we are dumping to savefile we should use higher value. Michal > ___ > tcpdump-workers mailing list > tcpdump-workers@lists.tcpdump.org > https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] [libpcap] New DLT value Request - Wattstopper DLM (#401)
Hello Guy, >> The Family/Address/IR field contains 3-bits of family code, 2-bits of >> address mode, 2-bits of IR (infrared) routing, and 1-bit unused. >> The 8 Legrand NITOO families are 000=CAD Filaire, 001=TopDog, 010=CAD RF, >> 011=CAD PLC, 100=CAD IR, 101=DLM, 111=escape, and 111=DLM Bootloader). >> This document only describes the DLM family. > > Are the fields following the Family/Address/IR field dependent on the family > code? Or can the rest of the packet, with the possible exception of the > opcode and payload, be parsed the same way regardless of the family code? Yes, the fields following the Family code are dependent on the Family code. > Are the family codes dependent on the hardware being used? Yes, the Family codes are dependent on the hardware. The WattStopper DLM hardware uses a USB interface: http://www.wattstopper.com/products/digital-lighting-management/configuration-tools/dlm-computer-interface-tools-and-software.aspx and also tunnels the DLM packets in BACnet messages. Another Family is CAD PLC, part of Open-NITOO, but I don't know the hardware interface: http://www.myopen-legrandgroup.com/cfs-filesystemfile.ashx/__key/telligent-evolution-components-attachments/00-212-01-00-00-03-46-60/U3838B_5F00_Specifications.pdf >> The Opcode field contains the message command, which determines the Payload >> and Payload Length fields. > > Presumably the Payload Length field is the number of octets in the Payload > field. Yes, the Payload Length field is the number of octets in the Payload field. > Is there some other specification to which we can point for the values of the > Opcode field, and the corresponding payload, or can this be used for several > different families of opcodes (with the dissector presumably having to be > told which one to use)? (The spec doesn't have to itself describe the opcode > or payload values, although, if there's a spec or specs for them, references > to them might be useful.) I don't have a public specification for the Opcodes at the moment. Some are detailed in this Wireshark Lua dissector: http://kargs.net/captures/irb.lua >> The LRC Checksum excludes the Preamble octets. > > Does it cover the Dongle Code or Packet Delay field? The LRC does not cover the Dongle Code or Packet Delay or Preamble. Best Regards, Steve -- http://steve.kargs.net/ ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Libpcap timeout settings in tcpdump - too long when printing to a terminal?
Guy Harris wrote: > The longer timeout can reduce capturing overhead, and if you're > capturing a high volume of traffic to a file, it's probably the right > timeout to have. If, however, you're printing packets to the console, > you're probably doomed if it's a high volume of traffic, and may want > less of a delay if it's a low volume of traffic. > Should we reduce the timeout if -w isn't specified - or do so if -w > isn't specified *and* if we're outputting to a terminal (isatty(1) > returns a non-zero value)? Should we use immediate mode if libpcap Yes, I think that -w not specified, and isatty()==1. > supports it? Should we offer a command-line option to override the > default? (I'd prefer to have the default be appropriate for most > cases, which may mean a different default if printing packets or if I think that we should be able to override things. I had hoped to spend time on tcpdump over the holidays, but IETF nomcom work ate my brain. -- ] Never tell me the odds! | ipv6 mesh networks [ ] Michael Richardson, Sandelman Software Works| network architect [ ] m...@sandelman.ca http://www.sandelman.ca/| ruby on rails[ ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Libpcap timeout settings in tcpdump - too long when printing to a terminal?
On Jan 9, 2015, at 2:09 AM, Michal Sekletar wrote: > Can't we use new default timeout value (lower) if we detect TPACKET_V3, The first sentence of my original mail was "With TPACKET_V3 support, Linux users are discovering what those of us using BSD-flavored OSes have known for quite a while:" This is not a TPACKET_V3 issue, it's a buffering issue. I notice it when testing tcpdump on Macs, which don't have PF_PACKET sockets of any sort, they have BPF; if, for example, I test on the generally-low-traffic loopback interface by pinging 127.0.0.1, the packets don't show up continuously, they show up in batches, with a 1-second delay. > so we don't > defeat the purpose of buffering entirely but delay is short enough that output > looks realtime-ish to human observer looking at the console. If we are dumping > to savefile we should use higher value. Yes, that's what I was suggesting - have tcpdump only use the 1-second timeout when writing to a file, and use a lower timeout, such as 1/10 second or less, when writing to a terminal, *regardless of what capture mechanism is being used*. That will remove the display delay for BPF-like capture mechanisms (BPF itself - except on AIX, where we use immediate mode due to bugs, TPACKET_V3, possibly the Tru64 UNIX mechanism) and have no effect on capture mechanisms that don't do timeout-based buffering (all other Linux mechanisms, IRIX, BPF on AIX, DLPI on HP-UX). DLPI+bufmod on Solaris, as I remember, does the timeout differently - it's an inter-packet timeout - but I think it might help there. ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] [libpcap] New DLT value Request - Wattstopper DLM (#401)
On Jan 9, 2015, at 8:08 AM, Steve Karg wrote: > Hello Guy, > >>> The Family/Address/IR field contains 3-bits of family code, 2-bits of >>> address mode, 2-bits of IR (infrared) routing, and 1-bit unused. >>> The 8 Legrand NITOO families are 000=CAD Filaire, 001=TopDog, 010=CAD RF, >>> 011=CAD PLC, 100=CAD IR, 101=DLM, 111=escape, and 111=DLM Bootloader). >>> This document only describes the DLM family. >> >> Are the fields following the Family/Address/IR field dependent on the family >> code? Or can the rest of the packet, with the possible exception of the >> opcode and payload, be parsed the same way regardless of the family code? > > Yes, the fields following the Family code are dependent on the Family code. I.e., following the Family/Address/IR field, other families might not have the Sequence ID, Source and Destination MAC addresses, Opcode, and Payload Length followed by Payload? Is the checksum family-code independent or might packets with other family codes not have it (or calculate the checksum differently)? >> Are the family codes dependent on the hardware being used? > > Yes, the Family codes are dependent on the hardware. The WattStopper > DLM hardware uses a USB interface: > http://www.wattstopper.com/products/digital-lighting-management/configuration-tools/dlm-computer-interface-tools-and-software.aspx > and also tunnels the DLM packets in BACnet messages. > > Another Family is CAD PLC, part of Open-NITOO, but I don't know the > hardware interface: > http://www.myopen-legrandgroup.com/cfs-filesystemfile.ashx/__key/telligent-evolution-components-attachments/00-212-01-00-00-03-46-60/U3838B_5F00_Specifications.pdf What I meant was that you said "The 8 Legrand NITOO families are ..." - is this a protocol used only by Legrand, so that those are the only family codes used, or are there other vendors who might have their own set of families and family codes, different from the Legrand ones? > The LRC does not cover the Dongle Code or Packet Delay or Preamble. I.e., it checksums packet data starting at the Family/Address/IR field and continuing through the end of the Payload field? ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] [libpcap] New DLT value Request - Wattstopper DLM (#401)
Hello Guy, The Family/Address/IR field contains 3-bits of family code, 2-bits of address mode, 2-bits of IR (infrared) routing, and 1-bit unused. The 8 Legrand NITOO families are 000=CAD Filaire, 001=TopDog, 010=CAD RF, 011=CAD PLC, 100=CAD IR, 101=DLM, 111=escape, and 111=DLM Bootloader). This document only describes the DLM family. >>> >>> Are the fields following the Family/Address/IR field dependent on the >>> family code? Or can the rest of the packet, with the possible exception of >>> the opcode and payload, be parsed the same way regardless of the family >>> code? >> >> Yes, the fields following the Family code are dependent on the Family code. > > I.e., following the Family/Address/IR field, other families might not have > the Sequence ID, Source and Destination MAC addresses, Opcode, and Payload > Length followed by Payload? Yes, that is correct. > Is the checksum family-code independent or might packets with other family > codes not have it (or calculate the checksum differently)? I presume the checksum is family-code dependent (although by luck it might not be). > What I meant was that you said "The 8 Legrand NITOO families are ..." - is > this a protocol used only by Legrand, so that those are the only family codes > used, or are there other vendors who might have their own set of families and > family codes, different from the Legrand ones? This protocol is only used by Legrand and their subsidiary companies (i.e. WattStopper), as far as I know. >> The LRC does not cover the Dongle Code or Packet Delay or Preamble. > > I.e., it checksums packet data starting at the Family/Address/IR field and > continuing through the end of the Payload field? Yes, that is correct. Best Regards, Steve -- http://steve.kargs.net/ ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] [libpcap] New DLT value Request - Wattstopper DLM (#401)
On Jan 9, 2015, at 4:11 PM, Steve Karg wrote: >> I.e., following the Family/Address/IR field, other families might not have >> the Sequence ID, Source and Destination MAC addresses, Opcode, and Payload >> Length followed by Payload? > > Yes, that is correct. OK, so we should, for now, indicate that values other than 5 in the Family subfield mean that the subsequent data in the frame are unspecified. If you can find documentation for the frame formats for the other families, we can update the spec. >> Is the checksum family-code independent or might packets with other family >> codes not have it (or calculate the checksum differently)? > > I presume the checksum is family-code dependent (although by luck it > might not be). So that goes along with the above - we'll indicate that we only specify what it means for a Family code of 5. >> What I meant was that you said "The 8 Legrand NITOO families are ..." - is >> this a protocol used only by Legrand, so that those are the only family >> codes used, or are there other vendors who might have their own set of >> families and family codes, different from the Legrand ones? > > This protocol is only used by Legrand and their subsidiary companies > (i.e. WattStopper), as far as I know. So we should assume that the Family subfield is always 3 bits and that the values always mean: 0 - Cad Filaire 1 - TopDog 2 - CAD RF 3 - CAD PLC 4 - CAD IR 5 - DLM 6 or 7 - escape 7 or 6 - DLM Bootloader The information you quoted said "111=escape, and 111=DLM Bootloader"; I assume one of those 111's is supposed to be 110. Which one of them should be 110? ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] [libpcap] New DLT value Request - Wattstopper DLM (#401)
Hello Guy, > 6 or 7 - escape > 7 or 6 - DLM Bootloader > > The information you quoted said "111=escape, and 111=DLM Bootloader"; I > assume one of those 111's is supposed to be 110. Which one of them should be > 110? 6 - unused 7 - escape and DLM Bootloader (unfortunately). The DLM Bootloader packet from LMCI USB dongle looks like this: +-+ |Dongle Code | |(1 Octet)| +-+ | Packet Delay | |(2 Octets) | +-+ |Preamble 1 | |(1 Octet | +-+ |Preamble 2 | |(1 Octet)| +-+ |Family/Address/IR| |Family=7 | |(1 Octet)| +-+ | Payload Length | | (1 Octets)| +-+ | Payload | . . . . . . | (0-32 Octets) | +-+ | LRC Checksum | | (1 Octet) | +-+ Best Regards, Steve -- http://steve.kargs.net/ ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
