Re: [tcpdump-workers] New official link-layer type request

[Michael Richardson]

Damir Franusic  wrote:
> Hi

> I have read the specs for pcapng but then again I would have have to use 
The
> Simple Packet Block (SPB) or
> An Enhanced Packet Block (EPB) and that would not solve my problem 
because of
> this:

pcapng is explicitely designed to be easily extensible.
So, you'd create whatever blocks you needed.

--
]   Never tell me the odds! | ipv6 mesh networks [
]   Michael Richardson, Sandelman Software Works|IoT architect   [
] m...@sandelman.ca  http://www.sandelman.ca/|   ruby on rails[

___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers

Re: [tcpdump-workers] New official link-layer type request

[Michael Richardson]

Damir Franusic  wrote:
> for Lawful Interception Data which can also use SCTP for transport, 
should I
> use the following naming
> scheme**instead:***draft-dfranusic-**tsvwg-00 *

No, that would make your draft visible to the Transport WG datatracker, and
unless it is your intention to have it adopted there, there is no advantage
to daking it hta tway.

http://socket.hr/draft-dfranusic-elee-00.xml

This URL is really good enough for me.

--
]   Never tell me the odds! | ipv6 mesh networks [
]   Michael Richardson, Sandelman Software Works|IoT architect   [
] m...@sandelman.ca  http://www.sandelman.ca/|   ruby on rails[

___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers

Re: [tcpdump-workers] New official link-layer type request

[Damir Franusic]

Hi 

The final link is this one: 

http://xml2rfc.tools.ietf.org/cgi-bin/xml2rfc.cgi?url=http://socket.hr/draft-dfranusic-opsawg-elee-00.xml&modeAsFormat=html/ascii

..so draft-dfranusic-opsawg-elee-00.xml

Guy has already assigned new DLT and used this link. It seemed more appropriate 
to target a specific group like opsawg. 
-- 
Damir Franusic

http://socket.hr
http://github.com/dfranusic

On May 18, 2019 11:19:30 PM GMT+02:00, Michael Richardson  
wrote:
>Damir Franusic  wrote:
>> for Lawful Interception Data which can also use SCTP for transport,
>should I
>> use the following naming
>> scheme**instead:***draft-dfranusic-**tsvwg-00 *
>
>No, that would make your draft visible to the Transport WG datatracker,
>and
>unless it is your intention to have it adopted there, there is no
>advantage
>to daking it hta tway.
>
>http://socket.hr/draft-dfranusic-elee-00.xml
>
>This URL is really good enough for me.
>
>--
>]   Never tell me the odds! | ipv6 mesh
>networks [
>]   Michael Richardson, Sandelman Software Works|IoT
>architect   [
>] m...@sandelman.ca  http://www.sandelman.ca/|   ruby on
>rails[
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers

Re: [tcpdump-workers] New official link-layer type request

[Damir Franusic]

Hi

I know it's extensible but ELEE is used for different purpose but I get you're 
trying to say.
-- 
Damir Franusic

http://socket.hr
http://github.com/dfranusic

On May 18, 2019 11:18:00 PM GMT+02:00, Michael Richardson  
wrote:
>Damir Franusic  wrote:
>> Hi
>
>> I have read the specs for pcapng but then again I would have have to
>use The
>> Simple Packet Block (SPB) or
>> An Enhanced Packet Block (EPB) and that would not solve my problem
>because of
>> this:
>
>pcapng is explicitely designed to be easily extensible.
>So, you'd create whatever blocks you needed.
>
>--
>]   Never tell me the odds! | ipv6 mesh
>networks [
>]   Michael Richardson, Sandelman Software Works|IoT
>architect   [
>] m...@sandelman.ca  http://www.sandelman.ca/|   ruby on
>rails[
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers

Re: [tcpdump-workers] New official link-layer type request

[Guy Harris]

On May 11, 2019, at 3:42 PM, Michael Richardson  wrote:

> Also, it might be that pcapng would actually be a really good container for
> your work rather than inventing yet-another-TLV.

Are there any law enforcement agencies that *will* accept a pcap file but 
*won't* accept a pcapng file?  *If* that's the case, that would prevent pcapng 
from being used, but if it's *not* the case, that might mean pcapng could be 
used.

If we *do* use pcapng, that would mean that:

1) Wireshark wouldn't be able to read the lawful intercept information 
in the files until support for new block types and options are added to it;

2) tcpdump wouldn't be able to read the lawful intercept information in 
the files until we add full pcapng support (with new APIs) to libpcap, 
including support for the new block types and options, and add support for the 
new APIs, and for the new block types and options, to tcpdump;

3) other programs that currently read pcap files would need to be able 
to read pcapng to read those files at all, and that support for pcapng would 
have to include the new block types and options in order to read the lawful 
intercept information.

To be fair, those programs would *also* have to be modified to handle 
LINKTYPE_ELEE - and programs that can read pcapng would at least be able to 
read the intercepted packets without change, assuming they just ignore unknown 
block and option types (which they should do!).
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers

Re: [tcpdump-workers] New official link-layer type request

[Damir Franusic]

Hi

Df_type is a part of CC configuration set by LEA for that target and I 
made a little mistake not explaining it properly.
This encoding is only relevant for IRI data in which case, Data can be 
either 0x03 ELEE format for IRI which is explained in
3.3.2.1.2.1.2.1. 
IPIRI 
Data Body. In case of CC data, Data part is alwas rawraw packet data 
starting with ETH header(DLT 0x01).  I will fix this (Df_type should be 
ignored in case of CC data), but like I said it's work in progress.



On 5/19/19 12:21 AM, Guy Harris wrote:

On May 12, 2019, at 1:28 PM, Damir Franusic  wrote:


I've tried to be as prompt and as accurate as possible so here is the draft, I 
hope you'll appreciate the effort. I agree
that the initial thing I sent was an abomination. I will work on this draft as 
the project progresses, but for now, it covers
everything implemented so far.

http://xml2rfc.tools.ietf.org/cgi-bin/xml2rfc.cgi?url=http://socket.hr/draft-dfranusic-elee-00.xml&modeAsFormat=html/ascii

Currently, the spec says that in a "Target PDU with CC data", the "Data size" field is the "Size of CC 
data encoded using the value from Df_type field (UINT32 field)" and the "Data" field is the "Raw CC packet 
data".

The "Df_type" field has values:

0x01Libpcap File Format (PCAP)
0x02ASN.1 Basic Encoding Rules (BER)
0x03ELEE Encapsulation

What do those values mean in this context?

For a value of 0x01, does that mean that the "Raw CC packet data" contains a 
pcap record:

https://www.tcpdump.org/manpages/pcap-savefile.5.html

with a time stamp, captured data length, and on-the-network data length, 
followed by packet data?  If so, what indicates the time stamp resolution and 
the link-layer type of the packet data?

Presumably 0x02 means BER-encoded ASN.1 data according to some ETSI 
specification, as per


The format of that delivery if defined by ETSI; they describe everything in 
great detail by using ASN.1 notation which is then encoded using
BER when sent by wire.

What ETSI specification is that?

And what does 0x03 mean?  If it's an "ELEE Encapsulation", it would presumably 
need to be defined by the ELEE spec itself, but it's not currently defined in that spec.


--
Damir Franusic

email: damir.franu...@gmail.com
http://ele2.io/

___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers

Re: [tcpdump-workers] New official link-layer type request

[Guy Harris]

On May 18, 2019, at 3:54 PM, Michael Richardson  wrote:

> Guy Harris  wrote:
>> If we *do* use pcapng, that would mean that:
> 
>> 1) Wireshark wouldn't be able to read the lawful intercept information
>> in the files until support for new block types and options are added to
>> it;
> 
> Is wireshark modular in how it handles pcapng blocks?

Somewhat, although it could probably use more work.

>> 2) tcpdump wouldn't be able to read the lawful intercept information in
>> the files until we add full pcapng support (with new APIs) to libpcap,
>> including support for the new block types and options, and add support
>> for the new APIs, and for the new block types and options, to tcpdump;
> 
> I hope to solve this in 2019/2020.

Definitely.  The sooner, the better; that would allow capturing on Linux, for 
example, to supply direction information for *all* link-layer header types (or, 
at least, for all link-layer header types provided by regular Linux 
interfaces), as well as providing IDBs for all interfaces when capturing on the 
"any" device, so that you could see what interface each packet came in on, even 
if you're reading the file on a machine other than the one on which the capture 
was done.

>> To be fair, those programs would *also* have to be modified to handle
>> LINKTYPE_ELEE - and programs that can read pcapng would at least be
>> able to read the intercepted packets without change, assuming they just
>> ignore unknown block and option types (which they should do!).
> 
> :-)
> My thought is that the regular packets would be in regular blocks, and the
> extra info would be in the extended blocks.  So without extensions, one can
> read the packets and do stuff with them, but not know, for instanse, which
> link they came from, or maybe (I have no idea if this is real meta-info)
> which warant was used to obtain the data.

Exactly.
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers

Re: [tcpdump-workers] New official link-layer type request

[Guy Harris]

On May 12, 2019, at 1:28 PM, Damir Franusic  wrote:

> I've tried to be as prompt and as accurate as possible so here is the draft, 
> I hope you'll appreciate the effort. I agree
> that the initial thing I sent was an abomination. I will work on this draft 
> as the project progresses, but for now, it covers
> everything implemented so far.
> 
> http://xml2rfc.tools.ietf.org/cgi-bin/xml2rfc.cgi?url=http://socket.hr/draft-dfranusic-elee-00.xml&modeAsFormat=html/ascii

Currently, the spec says that in a "Target PDU with CC data", the "Data size" 
field is the "Size of CC data encoded using the value from Df_type field 
(UINT32 field)" and the "Data" field is the "Raw CC packet data".

The "Df_type" field has values:

0x01Libpcap File Format (PCAP)
0x02ASN.1 Basic Encoding Rules (BER)
0x03ELEE Encapsulation

What do those values mean in this context?

For a value of 0x01, does that mean that the "Raw CC packet data" contains a 
pcap record:

https://www.tcpdump.org/manpages/pcap-savefile.5.html

with a time stamp, captured data length, and on-the-network data length, 
followed by packet data?  If so, what indicates the time stamp resolution and 
the link-layer type of the packet data?

Presumably 0x02 means BER-encoded ASN.1 data according to some ETSI 
specification, as per

> The format of that delivery if defined by ETSI; they describe everything in 
> great detail by using ASN.1 notation which is then encoded using
> BER when sent by wire.

What ETSI specification is that?

And what does 0x03 mean?  If it's an "ELEE Encapsulation", it would presumably 
need to be defined by the ELEE spec itself, but it's not currently defined in 
that spec.
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers

Re: [tcpdump-workers] New official link-layer type request

[Damir Franusic]

No I get now what you you're saying. You think that I should rewrite the 
draft to explain custom options in
Enhanced Packet Block, rather than using a new DLT ? 




On 5/19/19 12:53 AM, Guy Harris wrote:

On May 18, 2019, at 3:05 PM, Damir Franusic  wrote:


I know it's extensible but ELEE is used for different purpose

LINKTYPE_ELEE is used for the *same* purpose as pcapng - recording timestamped 
network events, and metadata for those events and for the capture process, in a 
file.

"Target PDUs" with a subtype of "Content of Communication", and that just 
contain raw packet data (as opposed to the ASN.1 stuff I asked about in an earlier message) are 
just pcapng Enhanced Packet Blocks:

the target identifier and sequence number in the Target PDU header 
would be options;

the timestamp would either just be the block's timestamp (and, unlike 
ELEE with its 32-bit Timestamp_sec, would work past Y2.106K);

the "target activity flag for CC data", "handover connection name for CC data delivery", "destination directory for 
CC data delivery used only for file system based connections" (if relevant here), "target aggregation factor for CC data delivery", 
"communication identifier Operator Id", "communication identifier Network Element Id", and "communication identifier 
Number" would also be options.

"Target PDUs" with a subtype of "Content of Communication" that contain that 
ETSI-specified ASN.1 data would be a new block type, using the same options as the new EPB options;

"Target PDUs" with a subtype of "Intercept Related Information (IRI)" would be 
one or more new block types, depending on whether to have a single block type for all of them, 
possibly using options.


--
Damir Franusic

email: damir.franu...@gmail.com
http://ele2.io/

___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers

Re: [tcpdump-workers] New official link-layer type request

[Damir Franusic]

Hi

LEAs SHOULD accept only ASN.1 BER encoded but that is not the case. I 
encountered a case where they wanted us


to convert that ASN.1 back to pcap. And the problem was that IRI is not 
packet data and that's why I would like a new DLT so I could either have 
a pcap file with all ELLE data or pcapng with mixed LinkLayer types with 
some blocks having DLT_ELEE for IRI data.


I am trying to make our product more professional and combine Wireshark 
with ETSI. With this dissector plugin and ELEE DLT I am actually doing 
pretty great.


I guess pcapng could be used by using SectionHeader block with ELEE DLT 
for LI data (IRI and CC((. I am doing my best and will continue do 
improve on that spec as promissed.


I could use pcapng which seems like a good idea nut I would still need 
that new ELEE DLT to set SectionHeader properly for IRI data that 
follows and also for CC data since unfortunately it also contains LI 
specific attributes. But You are right that pcapng could be ok.


I chose pcap since it's older and there's a better change for support 
and I have previously encountered one agency that actually demanded it.



On 5/19/19 12:27 AM, Guy Harris wrote:

On May 11, 2019, at 3:42 PM, Michael Richardson  wrote:


Also, it might be that pcapng would actually be a really good container for
your work rather than inventing yet-another-TLV.

Are there any law enforcement agencies that *will* accept a pcap file but 
*won't* accept a pcapng file?  *If* that's the case, that would prevent pcapng 
from being used, but if it's *not* the case, that might mean pcapng could be 
used.

If we *do* use pcapng, that would mean that:

1) Wireshark wouldn't be able to read the lawful intercept information 
in the files until support for new block types and options are added to it;

2) tcpdump wouldn't be able to read the lawful intercept information in 
the files until we add full pcapng support (with new APIs) to libpcap, 
including support for the new block types and options, and add support for the 
new APIs, and for the new block types and options, to tcpdump;

3) other programs that currently read pcap files would need to be able 
to read pcapng to read those files at all, and that support for pcapng would 
have to include the new block types and options in order to read the lawful 
intercept information.

To be fair, those programs would *also* have to be modified to handle 
LINKTYPE_ELEE - and programs that can read pcapng would at least be able to 
read the intercepted packets without change, assuming they just ignore unknown 
block and option types (which they should do!).


--
Damir Franusic

email: damir.franu...@gmail.com
http://ele2.io/

___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers

Re: [tcpdump-workers] New official link-layer type request

[Guy Harris]

On May 18, 2019, at 3:05 PM, Damir Franusic  wrote:

> I know it's extensible but ELEE is used for different purpose

LINKTYPE_ELEE is used for the *same* purpose as pcapng - recording timestamped 
network events, and metadata for those events and for the capture process, in a 
file.

"Target PDUs" with a subtype of "Content of Communication", and that just 
contain raw packet data (as opposed to the ASN.1 stuff I asked about in an 
earlier message) are just pcapng Enhanced Packet Blocks:

the target identifier and sequence number in the Target PDU header 
would be options;

the timestamp would either just be the block's timestamp (and, unlike 
ELEE with its 32-bit Timestamp_sec, would work past Y2.106K);

the "target activity flag for CC data", "handover connection name for 
CC data delivery", "destination directory for CC data delivery used only for 
file system based connections" (if relevant here), "target aggregation factor 
for CC data delivery", "communication identifier Operator Id", "communication 
identifier Network Element Id", and "communication identifier Number" would 
also be options.

"Target PDUs" with a subtype of "Content of Communication" that contain that 
ETSI-specified ASN.1 data would be a new block type, using the same options as 
the new EPB options;

"Target PDUs" with a subtype of "Intercept Related Information (IRI)" would be 
one or more new block types, depending on whether to have a single block type 
for all of them, possibly using options.
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers

Re: [tcpdump-workers] New official link-layer type request

[Guy Harris]

On May 18, 2019, at 4:26 PM, Damir Franusic  wrote:

> I chose pcap since it's older and there's a better change for support and I 
> have previously encountered one agency that actually demanded it.

That might be a sufficient reason for pcapng not to be the answer - if there 
are law enforcement agencies that will accept only pcap files, pcapng can't be 
used.
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers

Re: [tcpdump-workers] New official link-layer type request

[Damir Franusic]

And does wireshark currently support new block types and custom options in 
EPBs. I would need to access them in dissector plugin, that's what I'm worried 
about. 
-- 
Damir Franusic

http://socket.hr
http://github.com/dfranusic

On May 19, 2019 2:00:19 AM GMT+02:00, Guy Harris  wrote:
>On May 18, 2019, at 4:33 PM, Damir Franusic 
>wrote:
>
>> No I get now what you you're saying. You think that I should rewrite
>the draft to explain custom options in
>> Enhanced Packet Block, rather than using a new DLT ?
>
>No.
>
>I'm suggesting that:
>
>   you use EPBs, with new options, for Content of Communication PDUs;
>
>   you add *new* block types for Intercept Related Information PDUs;
>
>   you do that in a *separate* draft, as you already want a document to
>describe the ELEE protocol that runs atop SCTP.
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers

Re: [tcpdump-workers] New official link-layer type request

[Guy Harris]

On May 18, 2019, at 5:03 PM, Damir Franusic  wrote:

> And does wireshark currently support new block types and custom options in 
> EPBs. I would need to access them in dissector plugin, that's what I'm 
> worried about. 

There are three types of blocks:

1) standard blocks - you must first register them in the pcapng spec 
before you use them (just as you must register new LINKTYPE_/DLT_ values before 
using them), and then Wireshark can be changed to allow plugins for them;

2) "local" blocks, with a block type with the high-order bit set - you 
don't need to register them before using them, but you also can't arrange that 
nobody else use the same block type value;

3) custom blocks, for which your organization needs an IANA-assigned 
Private Enterprise Number - Wireshark currently doesn't support them, so we 
would have to add custom block support.

There are three types of options:

1) standard options - you must first register them in the pcapng spec 
before you use them (just as you must register new LINKTYPE_/DLT_ values before 
using them), and then Wireshark can be changed to allow plugins for them;

2) "local" options, with an option type with the high-order bit set - 
you don't need to register them before using them, but you also can't arrange 
that nobody else use the same block type value;

3) custom options, for which your organization needs an IANA-assigned 
Private Enterprise Number - Wireshark currently doesn't support them, so we 
would have to add custom option support.

Wireshark *does* support adding plugins to the file-reading code to handle 
local blocks and options, and to handle those standard block and option types 
not already handled by Wireshark (we don't support overriding the code to 
handle standard block and option types that *are* handled).

It also supports mapping pcapng block types to "file-type specific event" 
records, and registering plugin dissectors for those.

(Michael, this is the detailed answer to your question "Is wireshark modular in 
how it handles pcapng blocks?")
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers

Re: [tcpdump-workers] New official link-layer type request

[Guy Harris]

On May 18, 2019, at 4:33 PM, Damir Franusic  wrote:

> No I get now what you you're saying. You think that I should rewrite the 
> draft to explain custom options in
> Enhanced Packet Block, rather than using a new DLT ?

No.

I'm suggesting that:

you use EPBs, with new options, for Content of Communication PDUs;

you add *new* block types for Intercept Related Information PDUs;

you do that in a *separate* draft, as you already want a document to 
describe the ELEE protocol that runs atop SCTP.
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers

Re: [tcpdump-workers] New official link-layer type request

[Damir Franusic]

What works great for me now is the custom DLT and regular PCAP in 
Wireshark. The dissector I wrote
allows me to do a search based on that "metadata" and then use WIreshark 
to for example play RTP CC data
for SIP calls. I am not sure if pcapng is fully supported if i decide to 
implement all that using the options part of Extended Packet Block like 
Guy suggested.




On 5/19/19 1:31 AM, Guy Harris wrote:

On May 18, 2019, at 3:54 PM, Michael Richardson  wrote:


Guy Harris  wrote:

If we *do* use pcapng, that would mean that:
1) Wireshark wouldn't be able to read the lawful intercept information
in the files until support for new block types and options are added to
it;

Is wireshark modular in how it handles pcapng blocks?

Somewhat, although it could probably use more work.


2) tcpdump wouldn't be able to read the lawful intercept information in
the files until we add full pcapng support (with new APIs) to libpcap,
including support for the new block types and options, and add support
for the new APIs, and for the new block types and options, to tcpdump;

I hope to solve this in 2019/2020.

Definitely.  The sooner, the better; that would allow capturing on Linux, for example, to 
supply direction information for *all* link-layer header types (or, at least, for all 
link-layer header types provided by regular Linux interfaces), as well as providing IDBs 
for all interfaces when capturing on the "any" device, so that you could see 
what interface each packet came in on, even if you're reading the file on a machine other 
than the one on which the capture was done.


To be fair, those programs would *also* have to be modified to handle
LINKTYPE_ELEE - and programs that can read pcapng would at least be
able to read the intercepted packets without change, assuming they just
ignore unknown block and option types (which they should do!).

:-)
My thought is that the regular packets would be in regular blocks, and the
extra info would be in the extended blocks.  So without extensions, one can
read the packets and do stuff with them, but not know, for instanse, which
link they came from, or maybe (I have no idea if this is real meta-info)
which warant was used to obtain the data.

Exactly.
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


--
Damir Franusic

email: damir.franu...@gmail.com
http://ele2.io/

___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers