[tcpdump-workers] DLT_ request
Hi, We’ve been working to provide developers with a software packet capture solution for USB transfers at Apple. To that end, I have implemented a solution which uses BPF and is libpcap compatible but is currently using the link type DLT_USER15. Therefore, I’m requesting a DLT_ value for this. Please let me know what the proper procedure is for this or if you have any other questions/concerns. Regards, Scott Deandrea ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] DLT_ request
Hi Guy,
Nothing has shipped with any version of macOS yet; indeed I’m trying to avoid
the mistake that was made with DLT_USER2 and PKTAP.
The link-layer header format is as follows:
struct
{
// Control information
uint16_t bcdVersion;// version of this structure
uint8_t headerLength; // length of this structure
uint8_t requestType; // tAppleUSBHostPacketFilterRequestType
// Transfer information
uint32_t ioLength; // amount of data requested/transferred
uint32_t ioStatus; // IOReturn
uint32_t ioFrameCount; // number of isoch frames following
uint64_t ioID; // unique id for this I/O
// Device information
uint32_t deviceLocation;// locationID of the device
uint8_t deviceSpeed; // tEndpointSpeed
uint8_t deviceAddress; // tUSBHostDeviceAddress
uint8_t endpointAddress; // Address and Direction
uint8_t endpointType; // tEndpointType
// Additional information
} __attribute__((packed, aligned(sizeof(uint32_t;
The packet payload is the raw USB data that was sent/received for I/O request.
Thanks,
—scott
> On Dec 7, 2016, at 3:44 PM, Guy Harris wrote:
>
> On Dec 1, 2016, at 10:34 AM, Scott Deandrea wrote:
>
>> We’ve been working to provide developers with a software packet capture
>> solution for USB transfers at Apple. To that end, I have implemented a
>> solution which uses BPF and is libpcap compatible but is currently using the
>> link type DLT_USER15.
>
> (Hopefully that hasn't shipped with any version of macOS, and is only being
> used internally to Apple, so that Apple hasn't repeated the mistake they made
> with DLT_USER2 and PKTAP.)
>
>> Therefore, I’m requesting a DLT_ value for this. Please let me know what
>> the proper procedure is for this
>
> We need a description of the link-layer header for the packets, and an
> indication of what the packet payload is - either a document at Apple we can
> link to, or something we can put into a document hosted on tcpdump.org.
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] DLT_ request
Hi Guy,
For the initial release I’m planning to use 0x0100 for the bcdVersion. All
length values are in bytes.
tAppleUSBHostPacketFilterRequestType doesn’t exist in public domain at this
point. It tracks when a request was enqueued and when it completed and it is
defined as follows:
enum tAppleUSBHostPacketFilterRequestType
{
kAppleUSBHostPacketFilterRequestSubmit = 0,
kAppleUSBHostPacketFilterRequestComplete = 1
};
The deviceSpeed is an enumerated value defined in IOUSBHostFamily.h
enum
{
kUSBHostConnectionSpeedLow = 0,
kUSBHostConnectionSpeedFull = 1,
kUSBHostConnectionSpeedHigh = 2,
kUSBHostConnectionSpeedSuper= 3,
kUSBHostConnectionSpeedSuperPlus= 4,
kUSBHostConnectionSpeedCount= 5
};
The deviceAddress is the 7-bit USB device address as you described.
The endpointAddress encodes both number and direction in the same manor as the
bEndpointAddress field of an endpoint descriptor as described in section 9.6.6
of the USB 2.0 spec.
The endpointType is an enumerated value defined in IOUSBHostFamily.h and
corresponds to the transfer type portion of the bmAttributes field of an
endpoint descriptor.
enum tEndpointType
{
kEndpointTypeControl= (kEndpointDescriptorTransferTypeControl >>
kEndpointDescriptorTransferTypePhase),
kEndpointTypeIsochronous= (kEndpointDescriptorTransferTypeIsochronous
>> kEndpointDescriptorTransferTypePhase),
kEndpointTypeBulk = (kEndpointDescriptorTransferTypeBulk >>
kEndpointDescriptorTransferTypePhase),
kEndpointTypeInterrupt = (kEndpointDescriptorTransferTypeInterrupt >>
kEndpointDescriptorTransferTypePhase)
};
which evaluates to:
enum tEndpointType
{
kEndpointTypeControl = 0,
kEndpointTypeIsochronous = 1,
kEndpointTypeBulk= 2,
kEndpointTypeInterrupt = 3
};
The additional information is indeed expected to increase the header length and
bcdVersion should it ever be required.
Correct, the captured data would be the data field of a usb packet as described
in section 8.3.4.
Finally, the multi-byte fields are currently enforced to be little-endian but
this can be changed to big or host if desired.
—scott
> On Dec 9, 2016, at 5:40 PM, Guy Harris wrote:
>
> On Dec 9, 2016, at 1:37 PM, Scott Deandrea wrote:
>
>> The link-layer header format is as follows:
>> struct
>> {
>> // Control information
>> uint16_t bcdVersion;// version of this structure
>
> What's the current version? 0, 1, or something else?
>
>> uint8_t headerLength; // length of this structure
>
> Presumably length in bytes.
>
>> uint8_t requestType; // tAppleUSBHostPacketFilterRequestType
>
> So what are the possible values for requestType? (I don't see
> tAppleUSBHostPacketFilterRequestType anywhere in the public XNU source or any
> header from Xcode.)
>
>> // Transfer information
>> uint32_t ioLength; // amount of data requested/transferred
>
> Presumably length in bytes.
>
>> uint32_t ioStatus; // IOReturn
>> uint32_t ioFrameCount; // number of isoch frames following
>> uint64_t ioID; // unique id for this I/O
>>
>> // Device information
>> uint32_t deviceLocation;// locationID of the device
>> uint8_t deviceSpeed; // tEndpointSpeed
>
> Is this an enumerated value, a bits/second value, or something else?
>
>> uint8_t deviceAddress; // tUSBHostDeviceAddress
>
> So that's just a standard 7-bit USB device address, presumably, as per the
> USB 3.1 spec:
>
> A 7-bit value representing the address of a device on the USB. The
> device address is the default address (00H) when the USB device is first
> powered or the device is reset. Devices are assigned a unique device address
> by the USB system software.
>
> with the 8th bit zero?
>
>> uint8_t endpointAddress; // Address and Direction
>
> So that's the endpoint in the lower 4 bits, and the direction somewhere in
> the upper 4 bits? What are the representations for IN and OUT?
>
>> uint8_t endpointType; // tEndpointType
>
> So what are the values for bulk, control, interrupt, and isochronous?
>
>> // Additional information
>
> So the additional information's presence would be indicated by a greater
> headerLength and possibly also by a different bcdVersion value?
>
>> } __attribute__((packed, aligned(sizeof(uint32_t;
>>
>> The packet payload is the raw USB data that was sent/received for I/O
>> request.
>
> So that would be the "data field" from section 8.3.4 "Data Field" from the
> USB 2.0 spec and the DWORDs following the 16-byte header from the USB 3.1
> spec?
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] DLT_ request
Hi,
The bcdVersion field is interpreted as described by the bcdUSB field of the
standard device descriptor in section 9.6.1:
The bcdUSB field contains a BCD version number. The value of the bcdUSB field
is 0xJJMN for version JJ.M.N (JJ – major version number, M – minor version
number, N – sub-minor version number), e.g., version 2.1.3 is represented with
value 0x0213 and version 2.0 is represented with a value of 0x0200.
Yes, for every USB request issued by the host there’s a
kAppleUSBHostPacketFilterRequestSubmit packet emitted at request time and a
kAppleUSBHostPacketFilterRequestComplete packet emitted when the request
completes with the same ioID value. For control endpoints, the submit packet
also contains the setup data. For all other endpoint types, the submit packet
is only this header. The complete packet contains any data that was
transferred.
—scott
> On Dec 10, 2016, at 6:48 PM, Guy Harris wrote:
>
> On Dec 9, 2016, at 6:40 PM, Scott Deandrea wrote:
>
>> For the initial release I’m planning to use 0x0100 for the bcdVersion.
>
> So I'm guessing "bcd" implies that either octets or nibbles represent digits;
> is that the case?
>
>> tAppleUSBHostPacketFilterRequestType doesn’t exist in public domain at this
>> point. It tracks when a request was enqueued and when it completed and it
>> is defined as follows:
>> enum tAppleUSBHostPacketFilterRequestType
>> {
>> kAppleUSBHostPacketFilterRequestSubmit = 0,
>> kAppleUSBHostPacketFilterRequestComplete = 1
>> };
>
> So does that mean that, for every USB request issued by the host, there's a
> kAppleUSBHostPacketFilterRequestSubmit packet emitted at request time and a
> kAppleUSBHostPacketFilterRequestComplete packet emitted when the request is
> complete, with the same ioID value, and with each request getting its own
> ioID value (unless the system has been up long enough to make more than 2^32
> USB requests...)?
>
>> The additional information is indeed expected to increase the header length
>> and bcdVersion should it ever be required.
>
> So if the bcdVersion is 0x0100, the length should always be 32?
>
>> Correct, the captured data would be the data field of a usb packet as
>> described in section 8.3.4.
>
> So is there a way to determine whether the payload begins with Setup data?
> Would that be the case for all submitted requests with an endpointType of
> kEndpointTypeControl (0) and an endpointAddress with an endpoint number of 0
> (Default Control Pipe)?
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] DLT_ request
The deviceLocation is an Apple specific property that describes the bus
topology to which the device is connected. The most significant byte contains
the bus number; a unique value given to a host controller and also used when
generating the interface numbers. The following nibbles correspond to the port
number(s) through which the device is connected. For example:
FaceTime HD Camera (Built-in)@1a11
The bus number is 0x1a and the camera is connected to port 1 of the hub that is
connected to root port 1 on the host controller.
The ioFrameCount field, though not currently supported, is intended to be used
for isochronous endpoints as their I/O model is slightly different in that you
pass down a vector describing I/O for multiple USB frames. On completion you
would receive the header followed by and ioFrameCount array of isochronous
header followed by the packet data. The isochronous header is currently
defined as follows:
struct
{
// Control information
uint32_t frameHeaderLength;
// Frame information
uint32_t frameLength;
uint64_t frameNumber;
uint64_t ioTimestamp;
} __attribute__((packed, aligned (sizeof(uint32_t;
—scott
> On Dec 10, 2016, at 7:00 PM, Guy Harris wrote:
>
> On Dec 9, 2016, at 1:37 PM, Scott Deandrea wrote:
>
>> uint32_t deviceLocation;// locationID of the device
>
> So is a locationID something defined by a USB specification, by Apple in a
> fashion specific to Darwin, or something else? What is the format of a
> locationID?
>
>> uint32_t ioFrameCount; // number of isoch frames following
>
> So would that value affect the interpretation of the payload and, if so, how?
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] DLT_ request
The bcdVersion field is a version number of the header using the same 0xJJMN syntax. The data is part of the complete packet regardless if the host is sending or receiving data to/from the device. —scott > On Dec 11, 2016, at 6:27 PM, Guy Harris wrote: > > On Dec 11, 2016, at 8:12 AM, Scott Deandrea wrote: > >> The bcdVersion field is interpreted as described by the bcdUSB field of the >> standard device descriptor in section 9.6.1: >> The bcdUSB field contains a BCD version number. The value of the bcdUSB >> field is 0xJJMN for version JJ.M.N (JJ – major version number, M – minor >> version number, N – sub-minor version number), e.g., version 2.1.3 is >> represented with value 0x0213 and version 2.0 is represented with a value of >> 0x0200. > > The bcdUSB field in section 9.6.1 "Device" appears to be the version of the > USB specification that the device supports. Is that the case? > > If so, is the bcdVersion field also a USB specification version number, or is > it a version number of the USB-over-BPF protocol, using the same 0xJJMN > syntax as the USB specification version number in bsdUSB? > >> Yes, for every USB request issued by the host there’s a >> kAppleUSBHostPacketFilterRequestSubmit packet emitted at request time and a >> kAppleUSBHostPacketFilterRequestComplete packet emitted when the request >> completes with the same ioID value. For control endpoints, the submit >> packet also contains the setup data. For all other endpoint types, the >> submit packet is only this header. The complete packet contains any data >> that was transferred. > > So that's the case both if the host is sending data to the device and if the > host is receiving data from the device - the complete packet has the data? ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] DLT_ request
Hi Guy,
The bus number is 0 based and the port numbers are 1 based.
I decided to implement isochronous transfers today and changed the structure
slightly:
struct
{
// Control information
uint32_t frameHeaderLength; // 28
// Frame information
uint32_t frameLength; // Amount of data sent/received this frame
uint32_t frameStatus;// IOReturn result of the I/O this frame
uint64_t frameNumber; // Frame number on which this was scheduled/executed
by the controller
uint64_t ioTimestamp; // Time in which the frame completed
} __attribute__((packed, aligned (sizeof(uint32_t;
Therefore, the isochronous format for a request with type
kAppleUSBHostPacketFilterRequestComplete is as follows:
Link Header
padding, if required to force 4-byte alignment
Isochronous Frame[0] Header (frameHeaderLength bytes in length)
Isochronous Frame[0] Data (frameLength bytes)
…
padding, if required to force 4-byte alignment
Isochronous Frame[linkHeader.ioFrameCount - 1] Header (aligned to 4 bytes,
frameHeaderLength bytes in length)
Isochronous Frame[linkHeader.ioFrameCount - 1] Data (frameLength bytes)
The isochronous format for a request with type
kAppleUSBHostPacketFilterRequestSubmit is similar (no data following the
isochronous frame header):
Link Header
padding, if required to force 4-byte alignment
Isochronous Frame[0] Header (frameHeaderLength bytes in length)
…
padding, if required to force 4-byte alignment
Isochronous Frame[linkHeader.ioFrameCount - 1] Header (aligned to 4 bytes,
frameHeaderLength bytes in length)
Non-ischronous endpoints will always have an ioFrameCount of 0.
—scott
> On Dec 11, 2016, at 6:37 PM, Guy Harris wrote:
>
> On Dec 11, 2016, at 8:32 AM, Scott Deandrea wrote:
>
>> The deviceLocation is an Apple specific property that describes the bus
>> topology to which the device is connected. The most significant byte
>> contains the bus number; a unique value given to a host controller and also
>> used when generating the interface numbers. The following nibbles
>> correspond to the port number(s) through which the device is connected. For
>> example:
>>
>> FaceTime HD Camera (Built-in)@1a11 > 0x1f263, registered, matched, active, busy 0 (9 ms), retain 24>
>> The bus number is 0x1a and the camera is connected to port 1 of the hub that
>> is connected to root port 1 on the host controller.
>
> So are ports numbered from 0 or from 1? If I'm reading 8.9.3 "Port Number"
> correctly:
>
> The specific port on a hub to which the packet is directed is
> identified by the value in the Route String Port field. When addressing the
> hub controller then the Port Number field at the hub’s tier level shall be
> set to zero in the Route String. The hub’s downstream ports are addressed
> beginning with one and count up sequentially.
>
> that would indicate that they're numbered from 1, so that a port number of 0
> indicates the end of the route.
>
>> The ioFrameCount field, though not currently supported, is intended to be
>> used for isochronous endpoints as their I/O model is slightly different in
>> that you pass down a vector describing I/O for multiple USB frames. On
>> completion you would receive the header followed by and ioFrameCount array
>> of isochronous header followed by the packet data. The isochronous header
>> is currently defined as follows:
>> struct
>> {
>> // Control information
>> uint32_t frameHeaderLength;
>>
>> // Frame information
>> uint32_t frameLength;
>> uint64_t frameNumber;
>> uint64_t ioTimestamp;
>> } __attribute__((packed, aligned (sizeof(uint32_t;
>
> So the frameHeaderLength would be 24 in the current version (the length of
> that structure), and the data for the first frame would start right after the
> header and go on for array[0].frameLength bytes, with the data for the second
> frame starting right after than and going on for array[1].frameLength bytes,
> and so on?
>
> Presumably that field would be ignored if the endpoint isn't an isochronous
> endpoint, right?
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] DLT_ request
Yes, the control endpoint's submit packet contains the 8 byte setup packet immediately following the link header. —scott > On Dec 12, 2016, at 6:09 PM, Guy Harris wrote: > > On Dec 12, 2016, at 10:21 AM, Scott Deandrea wrote: > >> The data is part of the complete packet regardless if the host is sending or >> receiving data to/from the device. > > So the only submit packets with data would be packets sent to control > endpoints, which would contain setup data? ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] DLT_ request
Yes, if there are fewer tiers of hubs, the unused nibbles will be zero.
Correct, the padding is only added if frameLength isn’t a multiple of 4. This
should be considered the final version. The timestamps are in Mach Absolute
Time Units (https://developer.apple.com/library/content/qa/qa1398/_index.html
<https://developer.apple.com/library/content/qa/qa1398/_index.html>).
—scott
> On Dec 12, 2016, at 9:49 PM, Guy Harris wrote:
>
> On Dec 12, 2016, at 6:11 PM, Scott Deandrea wrote:
>
>> The bus number is 0 based and the port numbers are 1 based.
>
> So if there are fewer than 6 ports in the route, the unused nibbles will be
> zero.
>
>> I decided to implement isochronous transfers today and changed the structure
>> slightly:
>> struct
>> {
>> // Control information
>> uint32_t frameHeaderLength; // 28
>>
>> // Frame information
>> uint32_t frameLength; // Amount of data sent/received this frame
>> uint32_t frameStatus;// IOReturn result of the I/O this frame
>> uint64_t frameNumber; // Frame number on which this was scheduled/executed
>> by the controller
>> uint64_t ioTimestamp; // Time in which the frame completed
>> } __attribute__((packed, aligned (sizeof(uint32_t;
>>
>> Therefore, the isochronous format for a request with type
>> kAppleUSBHostPacketFilterRequestComplete is as follows:
>> Link Header
>> padding, if required to force 4-byte alignment
>> Isochronous Frame[0] Header (frameHeaderLength bytes in length)
>> Isochronous Frame[0] Data (frameLength bytes)
>> …
>> padding, if required to force 4-byte alignment
>> Isochronous Frame[linkHeader.ioFrameCount - 1] Header (aligned to 4 bytes,
>> frameHeaderLength bytes in length)
>> Isochronous Frame[linkHeader.ioFrameCount - 1] Data (frameLength bytes)
>
> So the headers and data are interleaved; the padding would presumably be
> needed only if frameLength isn't a multiple of 4.
>
> Is this the final version, or should the format of a payload of a packet for
> an isochronous endpoint be left unspecified for now until it's finalized?
>
> Also, what are the formats of the time stamps? What are the units, and is
> there a specified origin or are they just relative to an unspecified base
> time?
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] DLT_ request
Hi Guy,
Any update on allocating a DLT value? Perhaps #define DLT_MACOS_USB 266?
—scott
> On Dec 13, 2016, at 8:38 AM, Scott Deandrea wrote:
>
> Yes, if there are fewer tiers of hubs, the unused nibbles will be zero.
>
> Correct, the padding is only added if frameLength isn’t a multiple of 4.
> This should be considered the final version. The timestamps are in Mach
> Absolute Time Units
> (https://developer.apple.com/library/content/qa/qa1398/_index.html
> <https://developer.apple.com/library/content/qa/qa1398/_index.html>).
>
> —scott
>
>> On Dec 12, 2016, at 9:49 PM, Guy Harris > <mailto:g...@alum.mit.edu>> wrote:
>>
>> On Dec 12, 2016, at 6:11 PM, Scott Deandrea > <mailto:sdeand...@apple.com>> wrote:
>>
>>> The bus number is 0 based and the port numbers are 1 based.
>>
>> So if there are fewer than 6 ports in the route, the unused nibbles will be
>> zero.
>>
>>> I decided to implement isochronous transfers today and changed the
>>> structure slightly:
>>> struct
>>> {
>>> // Control information
>>> uint32_t frameHeaderLength; // 28
>>>
>>> // Frame information
>>> uint32_t frameLength; // Amount of data sent/received this frame
>>> uint32_t frameStatus;// IOReturn result of the I/O this frame
>>> uint64_t frameNumber; // Frame number on which this was
>>> scheduled/executed by the controller
>>> uint64_t ioTimestamp; // Time in which the frame completed
>>> } __attribute__((packed, aligned (sizeof(uint32_t;
>>>
>>> Therefore, the isochronous format for a request with type
>>> kAppleUSBHostPacketFilterRequestComplete is as follows:
>>> Link Header
>>> padding, if required to force 4-byte alignment
>>> Isochronous Frame[0] Header (frameHeaderLength bytes in length)
>>> Isochronous Frame[0] Data (frameLength bytes)
>>> …
>>> padding, if required to force 4-byte alignment
>>> Isochronous Frame[linkHeader.ioFrameCount - 1] Header (aligned to 4 bytes,
>>> frameHeaderLength bytes in length)
>>> Isochronous Frame[linkHeader.ioFrameCount - 1] Data (frameLength bytes)
>>
>> So the headers and data are interleaved; the padding would presumably be
>> needed only if frameLength isn't a multiple of 4.
>>
>> Is this the final version, or should the format of a payload of a packet for
>> an isochronous endpoint be left unspecified for now until it's finalized?
>>
>> Also, what are the formats of the time stamps? What are the units, and is
>> there a specified origin or are they just relative to an unspecified base
>> time?
>
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] DLT_ request
The mach absolute time base is different between ARM and x86/x64 though developers won’t have access to packet capture on iOS devices (internally the packet capture is used on iOS devices). The developers that would be using this software capture are familiar with the Mach Absolute Time format as it is the same values returned by the real software stack so I don’t see any need to change the format to nanoseconds. —scott > On Jan 5, 2017, at 8:23 PM, Guy Harris wrote: > > On Dec 13, 2016, at 8:38 AM, Scott Deandrea wrote: > >> The timestamps are in Mach Absolute Time Units >> (https://developer.apple.com/library/content/qa/qa1398/_index.html). > > That says > > This unit is CPU dependent, so you can't just multiply it by a constant > to get a real world value. Rather, you should call a system-provided > conversion function to convert it to a real world value. > > Unfortunately, that would require that the clock rate be provided somewhere > in the capture file. > > However, a quick look at _absolutetime_to_microtime() for x86 (which is > what's used by absolutetime_to_microtime(), which is what's used by > clock_get_calendar_microtime(), which is what's used by microtime(), which is > what's used by the BPF code to time stamp packets) indicates that the units > are "nanoseconds" (and that it's "nanoseconds since the Epoch", for > appropriate values of "since the Epoch" - don't get me started on leap > seconds and POSIX...). > > I don't know whether that's the case on ARM (the Apple TV has an USB port, > after all...), but, if it is, and if Apple's going to continue to maintain > that as the case, then we can just say it's in nanoseconds. > > Otherwise, you might want to convert it to nanoseconds rather than using a > CPU-dependent unit. ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] DLT_ request
The frameNumber is a USB spec defined value that correlates with the
start-of-frame packets frame number as defined in section 8.4.3 Start-of-Frame
Packets.
The ioTimestamp is in the same Mach Absolute Time.
Yes, the way you have described the padding is correct.
—scott
> On Jan 5, 2017, at 8:08 PM, Guy Harris wrote:
>
> On Dec 12, 2016, at 6:11 PM, Scott Deandrea wrote:
>
>> I decided to implement isochronous transfers today and changed the structure
>> slightly:
>> struct
>> {
>> // Control information
>> uint32_t frameHeaderLength; // 28
>>
>> // Frame information
>> uint32_t frameLength; // Amount of data sent/received this frame
>> uint32_t frameStatus;// IOReturn result of the I/O this frame
>> uint64_t frameNumber; // Frame number on which this was scheduled/executed
>> by the controller
>
> Is that a value specified by the USB standard or is it something specified by
> the Darwin USB stack?
>
>> uint64_t ioTimestamp; // Time in which the frame completed
>
> "In which" sounds like a relative time; is it a relative time or an absolute
> time?
>
>> Therefore, the isochronous format for a request with type
>> kAppleUSBHostPacketFilterRequestComplete is as follows:
>> Link Header
>> padding, if required to force 4-byte alignment
>
> Presumably that padding will not be required with the current link header
> format, as that header's length is a multiple of 4 bytes. (If a future
> version isn't a multiple of 4 bytes in length, it would be required.)
>
>> Isochronous Frame[0] Header (frameHeaderLength bytes in length)
>> Isochronous Frame[0] Data (frameLength bytes)
>> …
>> padding, if required to force 4-byte alignment
>> Isochronous Frame[linkHeader.ioFrameCount - 1] Header (aligned to 4 bytes,
>> frameHeaderLength bytes in length)
>> Isochronous Frame[linkHeader.ioFrameCount - 1] Data (frameLength bytes)
>>
>> The isochronous format for a request with type
>> kAppleUSBHostPacketFilterRequestSubmit is similar (no data following the
>> isochronous frame header):
>> Link Header
>> padding, if required to force 4-byte alignment
>> Isochronous Frame[0] Header (frameHeaderLength bytes in length)
>> …
>> padding, if required to force 4-byte alignment
>> Isochronous Frame[linkHeader.ioFrameCount - 1] Header (aligned to 4 bytes,
>> frameHeaderLength bytes in length)
>
> So no padding after the last frame, as it wouldn't be necessary (there's
> nothing to align on a 4-byte boundary).
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] DLT_ request
Yes, that's the behavior I have implemented in Wireshark and our internal tools. --scott > On Jan 5, 2017, at 8:52 PM, Guy Harris wrote: > >> On Jan 5, 2017, at 8:48 PM, Scott Deandrea wrote: >> >> The mach absolute time base is different between ARM and x86/x64 though >> developers won’t have access to packet capture on iOS devices (internally >> the packet capture is used on iOS devices). The developers that would be >> using this software capture are familiar with the Mach Absolute Time format >> as it is the same values returned by the real software stack so I don’t see >> any need to change the format to nanoseconds. > > ...so a Wireshark dissector, or tcpdump printer, for these packets would > presumably just show the time stamps as a raw 64-bit value, without any > interpretation, and leave it up to the person reading the capture to > interpret it. ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] DLT_ request
The ioLength for a submitted request will the the amount of data requested in
the data stage. The ioLength for a completed request will be the actual amount
of data transferred in the data stage. It’s quite similar to any other data
pipe.
Just to note, it may be acceptable for a device to send less than the amount
requested so the values may differ between a submit and a complete. For
example, a usb ethernet driver could always perform reads of MTU sizes but the
actual amount read will usually be less.
—scott
> On Jan 5, 2017, at 9:32 PM, Guy Harris wrote:
>
> On Dec 9, 2016, at 1:37 PM, Scott Deandrea wrote:
>
>> The link-layer header format is as follows:
>> struct
>> {
>> // Control information
>> uint16_t bcdVersion;// version of this structure
>> uint8_t headerLength; // length of this structure
>> uint8_t requestType; // tAppleUSBHostPacketFilterRequestType
>>
>> // Transfer information
>> uint32_t ioLength; // amount of data requested/transferred
>
> So, for a completed request - or a submitted request to the Default Control
> Pipe - does this have any relationship to the number of bytes of payload, or
> is the number of bytes of payload just "the on-the-wire length from the
> pcap/pcapng header minus the link-layer header length"?
>
> I.e., would checking that value against the actual number of bytes of payload
> in the packet, for completed requests and submitted request to the Default
> Control Pipe - be a useful error check, or should we just print that value
> without any interpretation or checking?
>
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] DLT_ request
An interrupt is generated when the frame number rolls over and we use this to increment the upper bits so the frame number can grow beyond 11 bits. This allows software consuming the frame number not to worry about the frame wrap when scheduling isochronous I/O. Yes, time *at* which… —scott > On Jan 5, 2017, at 9:35 PM, Guy Harris wrote: > >> On Jan 5, 2017, at 8:42 PM, Scott Deandrea wrote: >> >> The frameNumber is a USB spec defined value that correlates with the >> start-of-frame packets frame number as defined in section 8.4.3 >> Start-of-Frame Packets. > > So only 11 of the 64 bits are used? > >> The ioTimestamp is in the same Mach Absolute Time. > > So that's "Time *at* which the frame completed", rather than "Time *in* > which..."? > ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] DLT_ request
Correct, for a submitted request on the default control pipe is the maximum amount of data requested. The ioLength for a completed request of non-isochronous pipes will be equal to the amount of data payload transferred on the bus. I’m not following what/how you would check that. The link-layer headers, etc, for all packet types is handled entirely by the USB controller hardware and is not visible to host software in any way. To describe it another way, the host software is only ever aware of the data portion of the transfer as the link layer information is added/stripped by the controller hardware itself. Therefore, any bounds checking should be performed with respect to the captured length. For isochronous pipes, the ioLength for a submitted request is just the length sum of frameHeader.frameLength over all the submitted frames. For a completed request, the ioLength has no meaning on an isochronous pipe, only the frameLength is valid (which will be equal to the amount of data that was transferred). —scott > On Jan 5, 2017, at 10:33 PM, Guy Harris wrote: > >> On Jan 5, 2017, at 9:59 PM, Scott Deandrea wrote: >> >> The ioLength for a submitted request will the the amount of data requested >> in the data stage. > > So, for a submitted request on the Default Control Pipe, the ioLength won't > be 8 for the setup data, it'll be the maximum amount of data requested? > >> The ioLength for a completed request will be the actual amount of data >> transferred in the data stage. > > So, at least for a non-isochronous endpoint, it *should* be equal to the > amount of payload, i.e. on-the-wire length minus link-layer header length? > > Is there any reason to check that? Or should we just trust the on-the-wire > length (and captured length, for bounds checking), when dissecting the > payload? > > What about an isochronous endpoint? I'm guessing it won't be equal to the > total number of bytes of isochronous headers plus frame data plus padding. > >> Just to note, it may be acceptable for a device to send less than the amount >> requested so the values may differ between a submit and a complete. For >> example, a usb ethernet driver could always perform reads of MTU sizes but >> the actual amount read will usually be less. > > So the submitted request will have an ioLength reflecting the MTU, and the > completed request will have an ioLength reflecting the actual packet size. > That's not unexpected. ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] DLT_ request
Yes, you can think of the lower bits the frame number and the upper bits the number of overflows. There isn’t any reason to interpret it as anything other than a 64-bit integer nor to specify the division. —scott > On Jan 5, 2017, at 10:35 PM, Guy Harris wrote: > > On Jan 5, 2017, at 10:11 PM, Scott Deandrea wrote: > >> An interrupt is generated when the frame number rolls over and we use this >> to increment the upper bits so the frame number can grow beyond 11 bits. >> This allows software consuming the frame number not to worry about the frame >> wrap when scheduling isochronous I/O. > > So the lower 11 bits are the frame number and the upper bits are, in effect, > a count of frame number overflows. > > Is there any reason to interpret it as anything other than a 64-bit integer, > or to specify that division in the spec for the format? ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] DLT_ request
Okay, I understand what the check you are referring to now. The USB stack
guarantees that those conditions will not happen by doing exactly as you’ve
suggested with BPF.
The vernacular I generally hear and speak would be something similar to using
either “over USB” or “over the bus”.
For a completed isochronous request the ioLength is set to the sum of
frameHeader.frameLength over all the completed frames (the total amount of data
that was actually transferred for the request). It can/should be displayed but
it only provides a high-level overview of the actual result.
—scott
> On Jan 5, 2017, at 11:40 PM, Guy Harris wrote:
>
> On Jan 5, 2017, at 11:22 PM, Scott Deandrea wrote:
>
>> Correct, for a submitted request on the default control pipe is the maximum
>> amount of data requested. The ioLength for a completed request of
>> non-isochronous pipes will be equal to the amount of data payload
>> transferred on the bus. I’m not following what/how you would check that.
>
> In Wireshark, it'd be
>
> {Dissect link-layer header; when done, "offset" is the offset of the
> first byte past the link-layer header. Save the value of ioLength in a
> variable called "ioLength", and the protocol tree item for it in a variable
> called "ti_ioLength".}
>
> if (tvb_reported_length_remaining(tvb, offset) < ioLength)
> add an expert info to ti_ioLength saying "packet is missing
> data from the transfer"
> else if (tvb_reported_length_remaining(tvb, offset) > ioLength)
> add an expert info to ti_ioLength saying "packet has extra data
> past the end of the transfer"
>
> I.e., if the on-the-wire length of the packet is less than the amount of data
> payload transferred on the bus, note that the packet doesn't have all the
> data transferred (*not* because the packet was arbitrarily cut short by a
> snapshot length - that would be a comparison against
> tvb_captured_length_remaining(tvb, offset), and would be handled by an
> exception being thrown while dissecting the payload - and if the on-the-wire
> length of the packet is greater than the amount of data payload transferred
> on the bus, note that the packet has extra stuff at the end that didn't come
> from the transfer.
>
> However, if the USB stack guarantees that neither of those will happen, i.e.
> if the code that constructs the packet and submits it to BPF sets the
> on-the-wire length in the BPF header (bh_datalen) to the sum of the length of
> the header and the amount of data transferred, then those checks aren't
> necessary.
>
>> The link-layer headers, etc, for all packet types is handled entirely by the
>> USB controller hardware and is not visible to host software in any way. To
>> describe it another way, the host software is only ever aware of the data
>> portion of the transfer as the link layer information is added/stripped by
>> the controller hardware itself. Therefore, any bounds checking should be
>> performed with respect to the captured length.
>
> Note that when I say "link-layer header", I'm referring to the pseudo-header
> you've described, not any link-layer header information transmitted over the
> USB bus.
>
> (Speaking of which, "USB bus" is redundant, but "over the USB" sounds weird;
> do we just live with it the same way we live with "ATM machine" and "PIN
> number"?)
>
>> For isochronous pipes, the ioLength for a submitted request is just the
>> length sum of frameHeader.frameLength over all the submitted frames.
>
> That could, in theory, be checked, but I'm guessing that the code that
> constructs a packet and hands it to BPF explicitly calculates the ioLength as
> the sum of frameHeader.frameLength over all the submitted frames, so a check
> is pointless.
>
>> For a completed request, the ioLength has no meaning on an isochronous pipe,
>> only the frameLength is valid (which will be equal to the amount of data
>> that was transferred).
>
> Is it set to the sum of frameHeader.frameLength over all the submitted
> frames, or is it some random value that shouldn't even be displayed, much
> less checked?
>
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] DLT_ request
That draft looks correct and complete to me.
Thanks for your help,
—scott
> On Jan 6, 2017, at 8:34 AM, Guy Harris wrote:
>
> On Jan 4, 2017, at 4:19 PM, Scott Deandrea wrote:
>
>> Any update on allocating a DLT value? Perhaps #define DLT_MACOS_USB 266?
>
> So here's a draft of the header format:
>
> http://gharris.users.sonic.net/LINKTYPE_USB_DARWIN.html
>
> It won't look exactly like this, as this doesn't have the www.tcpdump.org
> style sheet, but the content should be the same. (Don't worry about the
> links at the top.)
>
> I've picked LINKTYPE_USB_DARWIN/DLT_USB_DARWIN because 1) if the header is
> OS-specific, we've put the OS name at the end (e.g., {LINKTYPE,DLT}_USB_LINUX
> and {LINKTYPE,DLT}_USB_FREEBSD) and 2) the Apple TV models have USB ports,
> too :-).
>
> Does this look correct and complete?
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] DLT_ request
Excellent, thank you for all your help. --scott > On Jan 7, 2017, at 11:49 AM, Guy Harris wrote: > >> On Jan 6, 2017, at 10:22 AM, Scott Deandrea wrote: >> >> That draft looks correct and complete to me. > > OK, I've assigned 266 for LINKTYPE_USB_DARWIN/DLT_USB_DARWIN. The changes > are checked into the master branch of libpcap, so it should appear in a > future 1.9 release of libpcap (which macOS 10.13 Big Sur, iOS 11, tvOS 11, > etc. should pick up, if it comes out in time). > > I've checked the LINKTYPE_USB_DARWIN document and a change to the link-layer > header types page into the tcpdump.org Web site repository; it should show up > in a day or two on the Web site. ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] DLT_ request
Correct, but it will be set to kIOReturnInvalid (0xe001).
--scott
> On Jan 7, 2017, at 6:51 PM, Guy Harris wrote:
>
>> On Dec 12, 2016, at 6:11 PM, Scott Deandrea wrote:
>>
>> I decided to implement isochronous transfers today and changed the structure
>> slightly:
>> struct
>> {
>> // Control information
>> uint32_t frameHeaderLength; // 28
>>
>> // Frame information
>> uint32_t frameLength; // Amount of data sent/received this frame
>> uint32_t frameStatus;// IOReturn result of the I/O this frame
>> uint64_t frameNumber; // Frame number on which this was scheduled/executed
>> by the controller
>> uint64_t ioTimestamp; // Time in which the frame completed
>> } __attribute__((packed, aligned (sizeof(uint32_t;
>>
>> Therefore, the isochronous format for a request with type
>> kAppleUSBHostPacketFilterRequestComplete is as follows:
>> Link Header
>> padding, if required to force 4-byte alignment
>> Isochronous Frame[0] Header (frameHeaderLength bytes in length)
>> Isochronous Frame[0] Data (frameLength bytes)
>> …
>> padding, if required to force 4-byte alignment
>> Isochronous Frame[linkHeader.ioFrameCount - 1] Header (aligned to 4 bytes,
>> frameHeaderLength bytes in length)
>> Isochronous Frame[linkHeader.ioFrameCount - 1] Data (frameLength bytes)
>>
>> The isochronous format for a request with type
>> kAppleUSBHostPacketFilterRequestSubmit is similar (no data following the
>> isochronous frame header):
>> Link Header
>> padding, if required to force 4-byte alignment
>> Isochronous Frame[0] Header (frameHeaderLength bytes in length)
>> …
>> padding, if required to force 4-byte alignment
>> Isochronous Frame[linkHeader.ioFrameCount - 1] Header (aligned to 4 bytes,
>> frameHeaderLength bytes in length)
>
> So, for a "submit" request, the frameStatus fields are presumably not
> meaningful, as they reports the status of the completed transfer, right?
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
