[tcpdump-workers] filters do not work in my program
Hi all,
I work on Debian 8, with linux version 3.16.0-4-amd64, libpcap.1.8.1,
gcc-4.9.2
I write a little program in C langage which reads a pcap file, apply
filter, and write the result into a new pcap file.
The problem: all filters do not work (I use capture filters and not display
filters).
the first filter: *tcp port 80 and host 192.168.10.11* do not work whereas the
second: *vlan 254*, *vlan 255*, etc. work fine. My traffic contains vlan
254, tcp, udp, port 80, port 443 and many ip address including
192.168.10.11. But when I apply the first one, the pcap result file
contains nothing. (I opened it with wireshark).
My program does not print any error except if the filter syntax is not
correct.
At the beginning I thought my filter syntax was wrong (I tried with display
filter, i.e tcp.port==80 and ip.src==192.168.10..1) but pcap_compile does
not accept it.
My questions:
1) According to me, I have to apply *capture filter* because I "capture"
packets from pcap file. Am I wrong ? Have I to use *display filter *instead
?
2) is it correct to compile filter with pcap_t issued from
pcap_open_offline() ? May be I have to do it before dumping, i.e with
pcap_t issued from pcap_open_dead() ?
3) Is there something wrong in my program ?
Here is the skeleton of my program:
#define MAX_FILTER_LEN 2048
#define MAX_SNAPLEN1500
char filter[MAX_FILTER_LEN] = "\0";
int snaplen = MAX_SNAPLEN;
struct bpf_program bpf = { 0 };
pcap_dumper_t *dumper = NULL;
typedef struct param {
pcap_dumper_t *dumper;
pcap_t *pcap_hdlr;
size_t capture_size_max;
int snaplen;
} * param_t;
param_t create_param(pcap_t *pcap_hdlr, pcap_dumper_t * dumper, size_t
capture_size_max, int snaplen)
{
param_t param = (param_t) malloc(sizeof(struct param));
param->pcap_hdlr = pcap_hdlr;
param->dumper = dumper;
param->capture_size_max = capture_size_max;
param->snaplen = snaplen;
return param;
}
void set_snaplen(struct pcap_pkthdr *hdr, int snaplen)
{
hdr->caplen = snaplen;
}
void write_pkts(u_char *user_data, const struct pcap_pkthdr *pkt_hdr, const
u_char *pkt_data)
{
param_t param = (param_t) user_data;
if (pcap_dump_ftell(param->dumper) > param->capture_size_max) {
pcap_breakloop(param->pcap_hdlr);
}
else {
set_snaplen(pkt_hdr, param->snaplen);
pcap_dump((u_char *)(param->dumper), pkt_hdr, pkt_data);
}
}
int
main(int argc, char *argv[])
{
/* ...
* at this stage I use getopt to get filter. I print it, it's OK.
* ...
*/
in_pcap_hdlr = pcap_open_offline(input_file_name, pcap_errbuf);
if (in_pcap_hdlr != NULL) {
if (pcap_compile(in_pcap_hdlr, &bpf, filter, 0, 0) == 0) {
if (pcap_setfilter(in_pcap_hdlr, &bpf) != 0) {
printf("ERROR with pcap_setfilter");
return -1;
}
}
else {
printf("ERROR with pcap_compile");
return -1;
}
int link_layer_type = pcap_datalink(in_pcap_hdlr);
if (link_layer_type != PCAP_ERROR_NOT_ACTIVATED) {
out_pcap_hdlr = pcap_open_dead(link_layer_type, snaplen);
if (out_pcap_hdlr != NULL) {
dumper = pcap_dump_open(out_pcap_hdlr, output_file_name);
if (dumper == NULL) {
printf("ERROR dumper is NULL");
return -1;
}
}
}
}
else {
printf("ERROR with pcap_open_offline");
return -1;
}
/* capture_size_max = 524288000 // 500 MB
* snaplen = 64 // 64 Bytes
*/
param_t param = create_param(in_pcap_hdlr, dumper, capture_size_max,
snaplen);
pcap_loop_ret = pcap_loop(in_pcap_hdlr, -1, write_pkts, (u_char *)
param);
// here I treat pcap_loop_ret
return 0;
}
in_pcap_hdlr = pcap_open_offline(input_file_name, pcap_errbuf);
struct bpf_program bpf = { 0 };
pcap_compile(capture_handles_tab[i], &bpf, filter, 0, 0)
pcap_setfilter(in_pcap_hdlr, &bpf);
Thanks for your help.
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] filters do not work in my program
> the first filter: *tcp port 80 and host 192.168.10.11* do not work whereas the > second: *vlan 254*, *vlan 255*, etc. work fine. My traffic contains vlan > 254, tcp, udp, port 80, port 443 and many ip address including > 192.168.10.11. But when I apply the first one, the pcap result file > contains nothing. (I opened it with wireshark). Try: vlan and tcp port 80 and host 192.168.10.11 Capture filter may needs to know that about the vlan, so it can look at the correct offset in the packet. But "vlan" should suffice, you don't have to specific which vlan it is. A display filter would work too I guess. ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
[tcpdump-workers] Saving packets with libpcap in PCAPNG format
I am working on an application that requires to store packets in PCAPNG format. My understanding is that there isn't support for saving packets in PCAPNG format in the current code base. I have noticed that Apple has created an API in its custom version of libpcap (latest version can be viewed at https://opensource.apple.com/source/libpcap/libpcap-67/libpcap/ and is based on libpcap-1.7.4), and the extension seems to be open source. Is there a plan to merge this to the libpcap at some point? Or is there plan to implement something else? Martin ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Saving packets with libpcap in PCAPNG format
On Dec 6, 2016, at 10:15 AM, Martin Dubuc wrote: > I am working on an application that requires to store packets in PCAPNG > format. My understanding is that there isn't support for saving packets in > PCAPNG format in the current code base. I have noticed that Apple has > created an API in its custom version of libpcap (latest version can be > viewed at https://opensource.apple.com/source/libpcap/libpcap-67/libpcap/ > and is based on libpcap-1.7.4), and the extension seems to be open source. Open source *but* licensed under the Apple Public Source License Version 2.0: https://opensource.apple.com/apsl which says: 1.1 "Applicable Patent Rights" mean: (a) in the case where Apple is the grantor of rights, (i) claims of patents that are now or hereafter acquired, owned by or assigned to Apple and (ii) that cover subject matter contained in the Original Code, but only to the extent necessary to use, reproduce and/or distribute the Original Code without infringement; and (b) in the case where You are the grantor of rights, (i) claims of patents that are now or hereafter acquired, owned by or assigned to You and (ii) that cover subject matter in Your Modifications, taken alone or in combination with Original Code. ... 2. Permitted Uses; Conditions & Restrictions. Subject to the terms and conditions of this License, Apple hereby grants You, effective on the date You accept this License and download the Original Code, a world-wide, royalty-free, non-exclusive license, to the extent of Apple's Applicable Patent Rights and copyrights covering the Original Code, to do the following: ... 2.4 Third Party Rights. You expressly acknowledge and agree that although Apple and each Contributor grants the licenses to their respective portions of the Covered Code set forth herein, no assurances are provided by Apple or any Contributor that the Covered Code does not infringe the patent or other intellectual property rights of any other entity. Apple and each Contributor disclaim any liability to You for claims brought by any other entity based on infringement of intellectual property rights or otherwise. As a condition to exercising the rights and licenses granted hereunder, You hereby assume sole responsibility to secure any other intellectual property rights needed, if any. For example, if a third party patent license is required to allow You to distribute the Covered Code, it is Your responsibility to acquire that license before distributing the Covered Code. 3. Your Grants. In consideration of, and as a condition to, the licenses granted to You under this License, You hereby grant to any person or entity receiving or distributing Covered Code under this License a non-exclusive, royalty-free, perpetual, irrevocable license, under Your Applicable Patent Rights and other intellectual property rights (other than patent) owned or controlled by You, to use, reproduce, display, perform, modify, sublicense, distribute and Externally Deploy Your Modifications of the same scope and extent as Apple's licenses under Sections 2.1 and 2.2 above. ... 5. Limitations on Patent License. Except as expressly stated in Section 2, no other patent rights, express or implied, are granted by Apple herein. Modifications and/or Larger Works may require additional patent licenses from Apple which Apple may grant in its sole discretion. ... 12. Termination. 12.1Termination. This License and the rights granted hereunder will terminate: (a) automatically without notice from Apple if You fail to comply with any term(s) of this License and fail to cure such breach within 30 days of becoming aware of such breach; (b) immediately in the event of the circumstances described in Section 13.5(b); or (c) automatically without notice from Apple if You, at any time during the term of this License, commence an action for patent infringement against Apple; provided that Apple did not first commence an action for patent infringement against You in that instance. I'm not sure whether the patent-related clauses - especially the "Termination" clause - would cause any vendors or distributors who currently include libpcap under its patent-clause-free BSD license not to want to include it if it includes patent clauses of that sort. > Is there a plan to merge this to the libpcap at some point? Or is there > plan to implement something else? My inclination was to implement *some* APIs for reading files (pcapng or pcap, using the same API, so programs can transparently *read* either file type), with the full capabilities of pcapng supported, and for writing pcapng files, with a separate implementation. If we can get away with implementing Apple's API independently, under the same BSD license as is used for the rest of libpcap, and tha
Re: [tcpdump-workers] Saving packets with libpcap in PCAPNG format
Has there been any discussions with folks from Apple that worked on the PCAPNG API to donate there code to tcpdump project? I am sure many (including Apple) would benefit from single source for this code as far as maintenance is concerned. Martin On Tue, Dec 6, 2016 at 1:32 PM, Guy Harris wrote: > On Dec 6, 2016, at 10:15 AM, Martin Dubuc wrote: > > > I am working on an application that requires to store packets in PCAPNG > > format. My understanding is that there isn't support for saving packets > in > > PCAPNG format in the current code base. I have noticed that Apple has > > created an API in its custom version of libpcap (latest version can be > > viewed at https://opensource.apple.com/source/libpcap/libpcap-67/ > libpcap/ > > and is based on libpcap-1.7.4), and the extension seems to be open > source. > > Open source *but* licensed under the Apple Public Source License Version > 2.0: > > https://opensource.apple.com/apsl > > which says: > > 1.1 "Applicable Patent Rights" mean: (a) in the case where > Apple is the grantor of rights, (i) claims of patents that are now or > hereafter acquired, owned by or assigned to Apple and (ii) that cover > subject matter contained in the Original Code, but only to the extent > necessary to use, reproduce and/or distribute the Original Code without > infringement; and (b) in the case where You are the grantor of rights, (i) > claims of patents that are now or hereafter acquired, owned by or assigned > to You and (ii) that cover subject matter in Your Modifications, taken > alone or in combination with Original Code. > > ... > > 2. Permitted Uses; Conditions & Restrictions. Subject to > the terms and conditions of this License, Apple hereby grants You, > effective on the date You accept this License and download the Original > Code, a world-wide, royalty-free, non-exclusive license, to the extent of > Apple's Applicable Patent Rights and copyrights covering the Original Code, > to do the following: > > ... > > 2.4 Third Party Rights. You expressly acknowledge and agree > that although Apple and each Contributor grants the licenses to their > respective portions of the Covered Code set forth herein, no assurances are > provided by Apple or any Contributor that the Covered Code does not > infringe the patent or other intellectual property rights of any other > entity. Apple and each Contributor disclaim any liability to You for claims > brought by any other entity based on infringement of intellectual property > rights or otherwise. As a condition to exercising the rights and licenses > granted hereunder, You hereby assume sole responsibility to secure any > other intellectual property rights needed, if any. For example, if a third > party patent license is required to allow You to distribute the Covered > Code, it is Your responsibility to acquire that license before distributing > the Covered Code. > > 3. Your Grants. In consideration of, and as a condition to, > the licenses granted to You under this License, You hereby grant to any > person or entity receiving or distributing Covered Code under this License > a non-exclusive, royalty-free, perpetual, irrevocable license, under Your > Applicable Patent Rights and other intellectual property rights (other than > patent) owned or controlled by You, to use, reproduce, display, perform, > modify, sublicense, distribute and Externally Deploy Your Modifications of > the same scope and extent as Apple's licenses under Sections 2.1 and 2.2 > above. > > ... > > 5. Limitations on Patent License. Except as expressly > stated in Section 2, no other patent rights, express or implied, are > granted by Apple herein. Modifications and/or Larger Works may require > additional patent licenses from Apple which Apple may grant in its sole > discretion. > > ... > > 12. Termination. > > 12.1Termination. This License and the rights granted > hereunder will terminate: > > (a) automatically without notice from Apple if You fail to > comply with any term(s) of this License and fail to cure such breach within > 30 days of becoming aware of such breach; > (b) immediately in the event of the circumstances described in > Section 13.5(b); or > (c) automatically without notice from Apple if You, at any > time during the term of this License, commence an action for patent > infringement against Apple; provided that Apple did not first commence an > action for patent infringement against You in that instance. > > I'm not sure whether the patent-related clauses - especially the > "Termination" clause - would cause any vendors or distributors who > currently include libpcap under its patent-clause-free BSD license not to > want to include it if it includes patent clauses of that sort. > > > Is there a plan to merge this to the libpcap at some point? Or is there > > plan to impl
Re: [tcpdump-workers] Saving packets with libpcap in PCAPNG format
On Dec 6, 2016, at 11:05 AM, Martin Dubuc wrote: > Has there been any discussions with folks from Apple that worked on the > PCAPNG API to donate there code to tcpdump project? I am sure many (including > Apple) would benefit from single source for this code as far as maintenance > is concerned. This was brought up on tcpdump-worker with somebody from Apple back in 2014: http://seclists.org/tcpdump/2014/q3/4 http://seclists.org/tcpdump/2014/q3/7 http://seclists.org/tcpdump/2014/q3/40 but nothing happened after that. Vincent? If you're still there, has anything happened about releasing Apple's libpcap changes under the BSD license rather than under the APSL? > Martin > > On Tue, Dec 6, 2016 at 1:32 PM, Guy Harris wrote: > On Dec 6, 2016, at 10:15 AM, Martin Dubuc wrote: > > > I am working on an application that requires to store packets in PCAPNG > > format. My understanding is that there isn't support for saving packets in > > PCAPNG format in the current code base. I have noticed that Apple has > > created an API in its custom version of libpcap (latest version can be > > viewed at https://opensource.apple.com/source/libpcap/libpcap-67/libpcap/ > > and is based on libpcap-1.7.4), and the extension seems to be open source. > > Open source *but* licensed under the Apple Public Source License Version 2.0: > > https://opensource.apple.com/apsl > > which says: > > 1.1 "Applicable Patent Rights" mean: (a) in the case where > Apple is the grantor of rights, (i) claims of patents that are now or > hereafter acquired, owned by or assigned to Apple and (ii) that cover subject > matter contained in the Original Code, but only to the extent necessary to > use, reproduce and/or distribute the Original Code without infringement; and > (b) in the case where You are the grantor of rights, (i) claims of patents > that are now or hereafter acquired, owned by or assigned to You and (ii) that > cover subject matter in Your Modifications, taken alone or in combination > with Original Code. > > ... > > 2. Permitted Uses; Conditions & Restrictions. Subject to the > terms and conditions of this License, Apple hereby grants You, effective on > the date You accept this License and download the Original Code, a > world-wide, royalty-free, non-exclusive license, to the extent of Apple's > Applicable Patent Rights and copyrights covering the Original Code, to do the > following: > > ... > > 2.4 Third Party Rights. You expressly acknowledge and agree that > although Apple and each Contributor grants the licenses to their respective > portions of the Covered Code set forth herein, no assurances are provided by > Apple or any Contributor that the Covered Code does not infringe the patent > or other intellectual property rights of any other entity. Apple and each > Contributor disclaim any liability to You for claims brought by any other > entity based on infringement of intellectual property rights or otherwise. As > a condition to exercising the rights and licenses granted hereunder, You > hereby assume sole responsibility to secure any other intellectual property > rights needed, if any. For example, if a third party patent license is > required to allow You to distribute the Covered Code, it is Your > responsibility to acquire that license before distributing the Covered Code. > > 3. Your Grants. In consideration of, and as a condition to, the > licenses granted to You under this License, You hereby grant to any person or > entity receiving or distributing Covered Code under this License a > non-exclusive, royalty-free, perpetual, irrevocable license, under Your > Applicable Patent Rights and other intellectual property rights (other than > patent) owned or controlled by You, to use, reproduce, display, perform, > modify, sublicense, distribute and Externally Deploy Your Modifications of > the same scope and extent as Apple's licenses under Sections 2.1 and 2.2 > above. > > ... > > 5. Limitations on Patent License. Except as expressly stated > in Section 2, no other patent rights, express or implied, are granted by > Apple herein. Modifications and/or Larger Works may require additional > patent licenses from Apple which Apple may grant in its sole discretion. > > ... > > 12. Termination. > > 12.1Termination. This License and the rights granted hereunder > will terminate: > > (a) automatically without notice from Apple if You fail to comply > with any term(s) of this License and fail to cure such breach within 30 days > of becoming aware of such breach; > (b) immediately in the event of the circumstances described in > Section 13.5(b); or > (c) automatically without notice from Apple if You, at any time > during the term of this License, commence an action for patent i
Re: [tcpdump-workers] filters do not work in my program
On Nov 30, 2016, at 4:14 AM, ikuzar RABE wrote: > I work on Debian 8, with linux version 3.16.0-4-amd64, libpcap.1.8.1, > gcc-4.9.2 > I write a little program in C langage which reads a pcap file, apply > filter, and write the result into a new pcap file. > > The problem: all filters do not work (I use capture filters and not display > filters). By "display filter" you mean "*Wireshark* display filter". Those filters only work in Wireshark and other programs that uses Wireshark's libwireshark; they do *not* work in other programs, as those filters require the full Wireshark dissection engine and filtering code, and that is *not* part of, for example, libpcap. > the first filter: *tcp port 80 and host 192.168.10.11* do not work whereas the > second: *vlan 254*, *vlan 255*, etc. work fine. My traffic contains vlan > 254, tcp, udp, port 80, port 443 and many ip address including > 192.168.10.11. But when I apply the first one, the pcap result file > contains nothing. (I opened it with wireshark). If the traffic is in a VLAN, "tcp port 80 and host 192.168.10.11" won't work, as the filter will, on Ethernet, look at the Ethernet type field at an offset of 12 from the beginning of the packet, see 0x8100, and conclude that it's not an IP packet because that's not 0x0800 or 0x86dd, and therefore conclude that it's not TCP, either. You'd need to do "vlan and tcp port 80 and host 192.168.10.11" to see port 80 traffic to/from 192.168.10.11 within a VLAN. To match both VLAN and non-VLAN traffic, you'd have to do (tcp port 80 and host 192.168.10.11) or (vlan and tcp port 80 and host 192.168.10.11) > My program does not print any error except if the filter syntax is not > correct. > At the beginning I thought my filter syntax was wrong (I tried with display > filter, i.e tcp.port==80 and ip.src==192.168.10..1) but pcap_compile does > not accept it. That's because wireshark display filters are not handled by libpcap at all. Only libpcap's capture filters work. > My questions: > 1) According to me, I have to apply *capture filter* because I "capture" > packets from pcap file. Am I wrong ? Have I to use *display filter *instead > ? The term "capture filter" is a Wireshark term, because Wireshark uses libpcap/WinPcap to capture traffic, and thus uses libpcap filters when capturing traffic, but uses a different filter syntax for filtering displayed traffic (and other purposes). Libpcap's filters are *NOT* used only when capturing traffic with other programs; libpcap supports them when reading files, and those are the filters used when reading files in tcpdump. > 2) is it correct to compile filter with pcap_t issued from > pcap_open_offline() ? Yes, it is correct. > May be I have to do it before dumping, i.e with > pcap_t issued from pcap_open_dead() ? The filter acts on packets that you're capturing or reading, so, when you compile it, you should use the pcap_t for the device from which you're capturing or the file from which you're reading, which, in this case, is the pcap_t from pcap_open_offline(). ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
[tcpdump-workers] Ringbuf pcap reading and "bogus savefile header" error
Hi all, I have a problem about reading circular ringbuf pcap records. There is a pcap file which stores last X seconds of packets. And with each X seconds of a period, a new pcap file is created. I can successfully read the initial pcap file for X seconds with "tail -n+o -F | tcpdump -r - -nn". But when the pcap file is rotated, the new pcap file causes to "bogus savefile header" from libpcap. I think that the new pcap file's header section is being tried to parse as packet data. How can I adapt the libpcap for my needs? I kindly want to take your opinions. Best regards, Tugrul, ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Ringbuf pcap reading and "bogus savefile header" error
On Dec 6, 2016, at 10:12 PM, Tugrul Erdogan wrote: > There is a pcap file which stores last X seconds of packets. And with each > X seconds of a period, a new pcap file is created. > > I can successfully read the initial pcap file for X seconds with "tail -n+o > -F | tcpdump -r - -nn". To quote the Linux man page for tail: -n, --lines=K output the last K lines, instead of the last 10; or use -n +K to output lines starting with the Kth The word "lines" appears in that text. Pcap files do not have lines, so any program that processes a pcap file as if it had lines in it will almost certainly do something wrong with the file. tail -n+o or, if this is what you really meant: tail -n+0 processes the file it's reading as if it has lines in it, so it will almost certainly do something wrong with the file You could *try* doing tail -F | tcpdump -r - -nn but I'm not sure even *that* is guaranteed to treat the file as if it were a binary file - which is exactly what a pcap file is. ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
