[tcpdump-workers] Requesting DLT_* values for Ethernet switches proprietary tagging protocol
Hi, (Guy, I wrote you a separate email about that nearly 2 years ago and got side tracked for that long on other stuff). A number of Ethernet switches from Broadcom, Marvell, Microchip, Qualcomm, Lantiq/Intel, etc. utilize proprietary tags that are processed by these switches in-line with the Ethernet frame being sent/received. These tags are inserted by the Ethernet switch's management interface for packets egressing the switch to a management port, and are created by a software agent (or specialized Ethernet adapter, typically from the same vendor) then sent to the management port for frames ingressing the switch. These tags serve as the foundation for Linux's Distributed Switch Architecture framework. As of Linux 4.19, there is now a way for user-space to find out which type of protocol is being configured on that management network interface. These tags all have a slightly different format, which makes it really difficult to create a reliable heuristic. Things are made largely more complicated with some tag formats having a different length for ingress and egress. DSA currently supports the following tagging protocols (details can be found under net/dsa/tag_*.c in Linux source for exactly length and offset within the Ethernet frame): DSA_TAG_PROTO_NONE = 0, DSA_TAG_PROTO_BRCM, DSA_TAG_PROTO_BRCM_PREPEND, DSA_TAG_PROTO_DSA, DSA_TAG_PROTO_EDSA, DSA_TAG_PROTO_GSWIP, DSA_TAG_PROTO_KSZ9477, DSA_TAG_PROTO_LAN9303, DSA_TAG_PROTO_MTK, DSA_TAG_PROTO_QCA, DSA_TAG_PROTO_TRAILER, I would therefore would like to add all, for a total of 10 of these to an upcoming release of libpcap, and then update tcpdump to parse those tags and pretty print these Ethernet switch tags. Please let me know if this is an appropriate request, and if you want me to elaborate in case that request does not constitute a reasonable DLT candidate. Here is a pull request to tcpdump that shows what/where the 4byte Broadcom tag format is, and how the heuristic currently works, which is IMHO non ideal: https://github.com/the-tcpdump-group/tcpdump/pull/728 and here is what the libpcap changes might look like: https://github.com/ffainelli/libpcap/commit/4a97aaee7cfa19d96ad3769e0ce7953fdf33825d Thank you! -- Florian ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Requesting DLT_* values for Ethernet switches proprietary tagging protocol
On 1/17/19 3:41 PM, Guy Harris wrote: > On Jan 17, 2019, at 2:39 PM, Florian Fainelli wrote: > >> A number of Ethernet switches from Broadcom, Marvell, Microchip, >> Qualcomm, Lantiq/Intel, etc. utilize proprietary tags that are processed >> by these switches in-line with the Ethernet frame being sent/received. >> >> These tags are inserted by the Ethernet switch's management interface >> for packets egressing the switch to a management port, and are created >> by a software agent (or specialized Ethernet adapter, typically from the >> same vendor) then sent to the management port for frames ingressing the >> switch. These tags serve as the foundation for Linux's Distributed >> Switch Architecture framework. As of Linux 4.19, there is now a way for >> user-space to find out which type of protocol is being >> configured on that management network interface. > > So does this mean that Linux is running on a processor in the switch itself, > or is this for something where you have a Linux box, one of whose Ethernet > interfaces is plugged into a management port, with the Linux kernel knowing > what type of tags are being used on that port, and with 4.19 exporting that > information to userland? The later, these Ethernet switches are either discrete components, and connected to a SoC via a version of the MII electrical interface (SGMII, RGMII, GMII, whatever) for the data path and the control path being either MDIO, I2C, SPI, GPIO, something else, or these switches are integrated into an SoC directly and memory mapped into the SoC's bus address space somehow. The management port of those switches is connected to one or more of the Ethernet NICs on the SoC, and so what that Ethernet NIC "sees" is Ethernet traffic that has been tagged by the switch hardware to allow for certain decisions to be made like: where is this packet coming from, why are you sending it to me, etc. In all of these cases, Linux runs on these SocS and has a driver for that switch and is able to fully control it. What has been added in 4.19, a sysfs attribute under /sys/class/net//dsa/tagging which tells us what kind of tagging protocol is being configured on that interface. > > And does the comment in your tcpdump change > > > /* p points to after the Ethernet header (include type/length), move it >* backwards by 2 bytes to parse the Broadcom tag and extract the actual >* type/length after it. >*/ > > mean that the Broadcom tags, at least, are inserted between the source MAC > address and type/length field (so that the frames have a modified header > containing the tag, rather than having a trailer containing the tag, for > example)? This is true for Broadcom tags, and a certain number of other Ethernet switch tags, but not universally true. Some of these tags can be "trailers" and be located right before the FCS for instance (Microchip is one of those, Marvell has one like that too). Some of these tags are between MAC SA and Type/Length field, but not 4 bytes, 2 bytes, etc... Having a reliable heuristic is IMHO mission impossible here, which is why we should have some help from libpcap to tell us the linktype/DLT. Does that make sense? -- Florian ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Requesting DLT_* values for Ethernet switches proprietary tagging protocol
Le 1/17/19 à 5:24 PM, Guy Harris a écrit : > On Jan 17, 2019, at 4:56 PM, Florian Fainelli wrote: > >> These Ethernet adapters can be regular/normal Ethernet adapters, e.g: >> e1000e/igb/ixgbe on a PC connected to an Ethernet switch via GMII (for >> data path), and controlled by that PC through GPIO/SPI/I2C/MDIO for >> instance. If the switch is an external die/package then if you looked at >> frames between that NIC and the switch, you would see the tag, but that >> would require you snooping the wires between these two dies. If the >> switch is internal, things are not visible obviously. >> >> These Ethernet switch tags *never* go out to front-panel/RJ45 >> connectors, they remain within the Ethernet switch logic and get >> processed internally and stripped, and then regular Ethernet frames go >> out these RJ45 connectors, a similar operation applies in the other >> direction (ingress). > > OK, so it sounds as if: > > 1) the special ports on which the new DLT_ values would be used would > *always* get packets with the tags corresponding to the switch type, and > would *never* get regular Ethernet packets *or* DOCSIS packets from a Cisco > CMTS (Cable Modem Termination System); > > 2) regular Ethernet ports would *never* get packets with switch tags. > > If so, that means that that: > > 1) the special ports should have only *one* DLT_ value - the one > corresponding to the switch type; > > 2) other Ethernet ports should just get DLT_EN10MB and DLT_DOCSIS. Correct. The other ports are exposed as regular Ethernet network devices. > >> OK, then requesting 10 DLT values, one per-tag is reasonable? > > Yes. I see that you answered on the pull request that I pasted the link for, thanks for doing that. Cheers -- Florian ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Requesting DLT_* values for Ethernet switches proprietary tagging protocol
Le 1/17/19 à 6:29 PM, Guy Harris a écrit :
> On Jan 17, 2019, at 6:01 PM, Florian Fainelli wrote:
>
>> Correct. The other ports are exposed as regular Ethernet network devices.
>
> OK, I've updated a comment in the pull request for that.
>
> So, in any pre-4.19 kernels, does the directory /sys/class/net/{device}/dsa
> exist even if there's no tagging file in that directory?
Neither the directory nor the file actually exist.
>
> Or is there some other way, in pre-4.19 kernels, to determine if a NIC is a
> DSA management-port NIC?
>
> If so, then, even though you can't determine the tag type from userland, you
> could offer all the DLT_s for the various tag types in the DLT list and let
> the user choose which one to use; they might be able to do so if they know
> what type of device they're capturing on. (If the tagging file *does* exist,
> you can just offer them the one DLT_ that's correct for the device.)
>
Good idea, people would certainly know which device they are capturing
on. Thanks!
--
Florian
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Requesting DLT_* values for Ethernet switches proprietary tagging protocol
On 1/17/2019 6:56 PM, Guy Harris wrote: > On Jan 17, 2019, at 2:39 PM, Florian Fainelli wrote: > >> DSA currently supports the following tagging protocols (details can be >> found under net/dsa/tag_*.c in Linux source for exactly length and offset >> within the Ethernet frame): > > Unfortunately, tag_brcm.c, for example, doesn't give *enough* details - it > has a bunch of uncommented #defines (some of which suggest overlap between > fields in the same byte of the tag) and some code that doesn't handle all the > fields in both ingress and egress tags. > > Can you construct "ASCII-art" diagrams and explanatory text, of the sort seen > in, for example: > > http://www.tcpdump.org/linktypes/LINKTYPE_LINUX_SLL.html > > for the tags? Most certainly, I can only answer for the Broadcom tags, since I am not very familiar with the other protocols, and I am hoping someone else could fill in for Intel/Qualcomm/Mediatek etc. We have two different formats, presented below. Let me know if you need more details on each field and I can expand. Egress (switch to management port) format: +--+ | Ethernet Destination address (6 octets) | +--+ +--+ | Ethernet Source address (6 octets) | +--+ +--+ | OPcode[7:5] | Traffic class[4:2] | Tag enforcement [1:0] | +--+ | Time stamp [7] | Unused | +--+ +--+ | Reserved [7:1]| Desination map[1]| +--+ +--+ | Destination map [7:0]| +--+ +--+ | Ethernet Type/Length (2 bytes) | +--+ +--+ | Ethernet payload | . . . . . . Ingress (management port to switch) format +--+ | Ethernet Destination address (6 octets) | +--+ +--+ | Ethernet Source address (6 octets) | +--+ +--+ | OPcode[7:5] | Reserved[4:0] | +--+ | Classification ID [7:0] | +--+ +--+ | Reason code [7:0]| +--+ +--+ | Traffic class [7:5] | Source port number [4:0] | +--+ +--+ | Ethernet Type/Length (2 bytes) | +--+ +--+ | Ethernet payload | . . . . . . -- Florian ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Requesting DLT_* values for Ethernet switches proprietary tagging protocol
On 1/17/2019 8:12 PM, Guy Harris wrote:
> On Jan 17, 2019, at 7:49 PM, Florian Fainelli wrote:
>
>> Le 1/17/19 à 6:29 PM, Guy Harris a écrit :
>>> On Jan 17, 2019, at 6:01 PM, Florian Fainelli wrote:
>>>
>>>> Correct. The other ports are exposed as regular Ethernet network devices.
>>>
>>> OK, I've updated a comment in the pull request for that.
>>>
>>> So, in any pre-4.19 kernels, does the directory /sys/class/net/{device}/dsa
>>> exist even if there's no tagging file in that directory?
>>
>> Neither the directory nor the file actually exist.
>>
>>>
>>> Or is there some other way, in pre-4.19 kernels, to determine if a NIC is a
>>> DSA management-port NIC?
>>>
>>> If so, then, even though you can't determine the tag type from userland,
>>> you could offer all the DLT_s for the various tag types in the DLT list and
>>> let the user choose which one to use; they might be able to do so if they
>>> know what type of device they're capturing on. (If the tagging file *does*
>>> exist, you can just offer them the one DLT_ that's correct for the device.)
>>>
>>
>> Good idea, people would certainly know which device they are capturing
>> on. Thanks!
>
> But if /sys/class/net/{device}/dsa doesn't exist in pre-4.19 kernels, how can
> you determine, from userland, whether a device is a DSA management-port NIC
> or not, in a system with a pre-4.19 kernel? If you can't determine that, you
> can't know whether to offer a list of DSA management-port tag type DLTs or
> not.
*sigh* yes, you cannot determine what type of DSA tagging protocol is
applied on that interface in that case.
So either you have a new enough tcpdump/libpcap that supports pretty
printing one of these tag formats and you can somehow force the linktype
on the command line to get the decoding to work, or everything is too
old and you just get what we currently have.
Originally, I was considering an extension of the TPACKET_V3 format and
using reserved/unused fields there, but this was not a great fit,
because unlike VLAN tags, which can change on a packet per-packet basis,
the actual format of these switching tags is known at interface
setup/configuration time, and does not change. Here is what I had in
mind originally which is no longer relevant:
https://github.com/ffainelli/linux/commits/dsa-tpacket
--
Florian
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Requesting DLT_* values for Ethernet switches proprietary tagging protocol
On 1/18/19 6:13 AM, Michael Richardson wrote: > > Guy Harris wrote: > >> OK, then requesting 10 DLT values, one per-tag is reasonable? > > > Yes. > > This is totally how it should be done. > > Guy, do we want to allocate 16 or 32 values, aligned, so that the > it be can DLT_TAG_BASE + DSA_TAG_xx maps? Are the kernel values stable? > And to leave room for growth. The kernel values are stable because they are now (as of 4.19) part of the sysfs attribute ABI. Having a base to offset from would ne neat. -- Florian ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Requesting DLT_* values for Ethernet switches proprietary tagging protocol
On 1/17/19 9:55 PM, Guy Harris wrote:
> On Jan 17, 2019, at 9:16 PM, Florian Fainelli wrote:
>
>> On 1/17/2019 8:12 PM, Guy Harris wrote:
>>
>>> But if /sys/class/net/{device}/dsa doesn't exist in pre-4.19 kernels, how
>>> can you determine, from userland, whether a device is a DSA management-port
>>> NIC or not, in a system with a pre-4.19 kernel? If you can't determine
>>> that, you can't know whether to offer a list of DSA management-port tag
>>> type DLTs or not.
>>
>> *sigh* yes, you cannot determine what type of DSA tagging protocol is
>> applied on that interface in that case.
>
> There are two questions here:
>
> 1) Can you determine, in libpcap, what type of DSA tagging protocol is
> applied on an interface?
>
> 2) Can you determine, in libpcap, whether an interface is a regular
> Ethernet interface or an interface that has some type of DSA tagging
> protocol, whether you can determine *which* tagging protocol it is?
>
> If the answer to the first question is "yes", then
>
> 1) the answer to the second question is "yes"
>
> and
>
> 2) you can precisely determine *which* DLT value to use on the
> interface.
>
> If the answer to the first question is "no", but the answer to the second
> question is "yes", then
>
> 1) you can't precisely determine *which* DLT value to use on the
> interface
>
> *but*
>
> 2) you can offer the user who's doing the capture a choice of *all* the
> DLT values used for DSA tagging protocols and, if the *user* knows which
> tagging protocol is used - even if libpcap can't determine it - they can
> choose the appropriate DLT value, and programs reading the packets will be
> able to dissect the tags correctly.
>
> If the answer to *both* questions is "no", the best you could do would be to
> offer DLT_EN10MB, DLT_DOCSIS, *and* all the DLT values on *all* Ethernet
> interfaces, and let the user choose.
>
> In 4.19 and later, the answer to the first question, and thus to both
> questions, appears to be "yes".
>
> In pre-4.19 kernels, the answer to the first question appears to be "no";
> what's the answer to the second question?
>
> What's the answer to the first question in pre-4.19 kernels?
In pre-4.19 kernels there was really no way you could reliably tell a
DSA management interface apart from a regular Ethernet device in the
system, even by scanning the network device's relationship through
ifindex etc.
>
>> So either you have a new enough tcpdump/libpcap that supports pretty
>> printing one of these tag formats and you can somehow force the linktype
>> on the command line to get the decoding to work,
>
> In the second and third cases, "[forcing] the linktype on the command line"
> would be using the -y flag to set the link type to the appropriate link type
> for the DSA tag being used. That wouldn't be necessary with a 4.19 or later
> kernel - the appropriate link type would be chosen by libpcap.
Right
>
>> or everything is too
>> old and you just get what we currently have.
>>
>> Originally, I was considering an extension of the TPACKET_V3 format and
>> using reserved/unused fields there, but this was not a great fit,
>> because unlike VLAN tags, which can change on a packet per-packet basis,
>> the actual format of these switching tags is known at interface
>> setup/configuration time, and does not change. Here is what I had in
>> mind originally which is no longer relevant:
>>
>> https://github.com/ffainelli/linux/commits/dsa-tpacket
>
> The way you'd do that - which would work with *all* TPACKET formats as well
> as with non-memory-mapped capture - would be to:
>
> 1) get new ARPHRD_ types defined in the kernel for all the DSA tag
> types;
>
> 2) have the device advertise the appropriate new ARPHRD_ type, rather
> than ARPHRD_ETHER, based on the DSA tags in the packets;
>
> 3) have libpcap map those ARPHRD_ types to the corresponding DLT_
> values.
>
> That means changing the kernel - and the 4.19 kernel appears to *already*
> have a mechanism to determine the DSA tag type, which libpcap can use, so
> changing the kernel would be useful only if you can change pre-4.19 kernels.
Which is not possible none of that qualify as a bug fix we can ask
-stable maintainers to backport to < 4.19 kernel branches but that is
fine, there is a beginning to supporting those tags properly and it
starts with 4.19 and future tcpdump/libpcap/wireshark releases hopefully.
--
Florian
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Requesting DLT_* values for Ethernet switches proprietary tagging protocol
On 1/20/2019 11:52 AM, Andrew Lunn wrote: > On Fri, Jan 18, 2019 at 10:19:04AM -0800, Florian Fainelli wrote: >> On 1/18/19 6:13 AM, Michael Richardson wrote: >>> >>> Guy Harris wrote: >>> >> OK, then requesting 10 DLT values, one per-tag is reasonable? >>> >>> > Yes. >>> >>> This is totally how it should be done. >>> >>> Guy, do we want to allocate 16 or 32 values, aligned, so that the >>> it be can DLT_TAG_BASE + DSA_TAG_xx maps? Are the kernel values stable? >>> And to leave room for growth. >> >> The kernel values are stable because they are now (as of 4.19) part of >> the sysfs attribute ABI. Having a base to offset from would ne neat. > > Hi Florian > > As Guy points out, we should probably move them into a uapi > header. That makes it clear the values are ABI. UAPI is probably a bit too much, but I can certainly extend the attribute documented here for the supported values: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/ABI/testing/sysfs-class-net-dsa -- Florian ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
