[tcpdump-workers] Link-Layer Header Type request for Linux Kernel Messages
Hello, I would like to request a new link-layer header type value: LINKTYPE_DEV_KMSG_LINUX DLT_DEV_KMSG_LINUX and LINKTYPE_KLOG_LINUX DLT_KLOG_LINUX Linux Kernel Message can be captured on Linux by /dev/kmsg and klogctl. Kernel logs can be useful for analysis Linux and network(etc.) behaviour. Test patch for libpcap for /dev/kmsg is prepared, so you can test it: https://github.com/MichalLabedzki/libpcap/commit/c671673753bba413fe3fc839425162d682289bec (works kernel >= 3.5 and /dev/kmsg, patch need some fixes to check that) Capture format specification is available at: https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/tree/Documentation/ABI/testing/dev-kmsg Linux logs can be dissected by Wireshark to improve readability/filtering. There is also klogctl (http://linux.die.net/man/3/klogctl) and /proc/kmsg - but there is different format. So another Linktype can be add. Pozdrawiam / Best regards - Michał Łabędzki, Software Engineer Tieto Corporation Product Engineering Services http://www.tieto.com / http://www.tieto.pl --- ASCII: Michal Labedzki e-mail: michal.labed...@tieto.com location: Swobodna 1 Street, 50-088 Wrocław, Poland room: 5.01 (desk next to 5.08) --- Please note: The information contained in this message may be legally privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any unauthorised use, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to the message and deleting it from your computer. Thank You. --- Please consider the environment before printing this e-mail. --- Tieto Poland spółka z ograniczoną odpowiedzialnością z siedzibą w Szczecinie, ul. Malczewskiego 26. Zarejestrowana w Sądzie Rejonowym Szczecin-Centrum w Szczecinie, XIII Wydział Gospodarczy Krajowego Rejestru Sądowego pod numerem 124858. NIP: 8542085557. REGON: 812023656. Kapitał zakładowy: 4 271500 PLN ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] moves to github
> Denis> I tried closing a couple of issues, but ran into a > Denis> question. Do existing tags in GitHub issues (v3.5, v3.6, > Denis> ... v4.3) mean "opened against version N.N" or "fixed in > Denis> version N.N"? > > let's take this to mean, opened against version N.N. I have looked through the GH issues once again, added a few tags and started labelling issues with them: bug feature request duplicate fixed in 4.3.0 web-site The last one deserves a special note, it stands for the kind of issues that I believe to belong to the tcpdump-htdocs repository. That is, the old web-site bugs will be labeled with the tag in the tcpdump repository and any new such issues should be opened in tcpdump-htdocs. I have enabled the issues feature for the tcpdump-htdocs repository and disabled the wiki feature for all 3 repositories (also checked that no wiki pages existed). -- Denis Ovsienko ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
[tcpdump-workers] tcpdump vs libpcap : CPU usage shooting high for two simultaneous captures on wireless monitor interfaces
hi everyone, I have written my own data collection tool, for custom needs using libpcap for wireless interfaces(2.4,5 GHz) on a router. I could not find any flag in tcpdump that i can collect only x number of mgmt packets, y number of control packets and the rest data packets. The issue i face is this : When I run two instances of tcpdump on each interface, it has 5-10% cpu usage on the router. When I use my tool written using libpcap: When I run it on one interface (say 2.4 GHz), I get 5-10% CPU usage (output on top), but when I run another instance of the tool on the other interface(5 GHz lets say), both the processes start eating 100% of CPU all the time. Can someone explain this behavior ? I was thinking, if both the process's recv() call was spinning on some shared resource (buffer)/some kernel thread ... but why does that not happen for tcpdump ? I have literally read tcpdump/pcap code before, but I hadn't notice anything special. Can someone help me debug this problem ? - Abhinav ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] tcpdump vs libpcap : CPU usage shooting high for two simultaneous captures on wireless monitor interfaces
On Apr 17, 2013, at 3:57 PM, abhinav narain wrote: > Can someone explain this behavior ? As you're saying "libpcap" rather than WinPcap, I'm assuming this is some flavor of UN*X, such as Linux. Could you build a profiled version of your tool (compile with "-pg") and then run gprof on the tool and the gmon.out file it produces when it exits? That should show you *where* the CPU time is being spent. This means that the program has to eventually exit cleanly - you can't interrupt it with, for example, ^C unless it catches SIGINT and exits, otherwise it'll fail to write out the gmon.out file. This might also be tricky if you cross-compile the tool to run on the router, and the router doesn't have gprof installed - you might have to have a version of gprof capable of analyzing executable files and gmon.out files from the router. ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
[tcpdump-workers] (no subject)
handle = pcap_open_live(dev, BUFSIZ, 0, 0, errbuf);
pcap_compile(handle, &fp, filter_exp, 0, mask) == -1
pcap_setfilter(handle, &fp);
struct pcap_pkthdr pcap_header; // The header that pcap gives
us
const u_char *pcap_packet; // The actual packet
while(1){
n=fork();
if(n==0) { // child process
fd_set rdfds;
int pcap_fd = pcap_get_selectable_fd(pcap_handler);
for(;;){
FD_ZERO(&rdfds);
FD_SET(pcap_fd, &rdfds);
FD_SET(sd_proxy, &rdfds); // here is another fd
select(pcap_fd>sd_proxy?pcap_fd+1:sd_proxy+1,
&rdfds, NULL, NULL, NULL)
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
[tcpdump-workers] parent-child process, selectable file descriptor and pcap
I have a program, part of the source codes are:
handle = pcap_open_live(dev, BUFSIZ, 0, 0, errbuf);
pcap_compile(handle, &fp, filter_exp, 0, mask) == -1
pcap_setfilter(handle, &fp);
struct pcap_pkthdr pcap_header; // The header that pcap gives
us
const u_char *pcap_packet; // The actual packet
while(1){
n=fork();
if(n==0) { // child process
fd_set rdfds;
int pcap_fd = pcap_get_selectable_fd(pcap_handler);
for(;;){
FD_ZERO(&rdfds);
FD_SET(pcap_fd, &rdfds);
FD_SET(sd_proxy, &rdfds); // here is another fd
select(pcap_fd>sd_proxy?pcap_fd+1:sd_proxy+1,
&rdfds, NULL, NULL, NULL)
if(FD_ISSET(pcap_fd, &rdfds)) {
pcap_packet = pcap_next(pcap_handler,
&pcap_header);
if(pcap_packet !=NULL)
printf("capture one ACK packet with
length of %d\n", pcap_header.len);
printf("pcap_packet size is %d, length is
%d\n", sizeof(pcap_packet), pcap_header.len);
n = send(sd_proxy,
pcap_packet+ETHERNET_HDR_LEN, pcap_header.len-ETHERNET_HDR_LEN, 0);
if(n<0){
printf("send error: %s\n",
strerror(errno));
goto END;
} //if
//blabla
}//if
}//for
// the child process will always exit the for loop after some
time
close(pcap_fd);
}//if
// parent process
}//while
for the first fork(), I mean for the first child process, it works fine
but for the second and following fork(), I always get "send error: Bad
address"
then I notice that the pcap_header.len is 0
what is the reason for this?
I think the problem may be:
1) there are parent-child process relationship
2) in each child process I used
int pcap_fd = pcap_get_selectable_fd(pcap_handler);
close(pcap_fd);
can anyone give some suggestions on this? thanks!
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] tcpdump vs libpcap : CPU usage shooting high for two simultaneous captures on wireless monitor interfaces
Thanks for replying on this, as I am actually helpless on this issue for days now. As you're saying "libpcap" rather than WinPcap, I'm assuming this is some > flavor of UN*X, such as Linux. > > You are right, its running 2.6.32 on OpenWrt. > Could you build a profiled version of your tool (compile with "-pg") and > then run gprof on the tool and the gmon.out file it produces when it exits? > That should show you *where* the CPU time is being spent. > > This means that the program has to eventually exit cleanly - you can't > interrupt it with, for example, ^C unless it catches SIGINT and exits, > otherwise it'll fail to write out the gmon.out file. > > This might also be tricky if you cross-compile the tool to run on the > router, and the router doesn't have gprof installed - you might have to > have a version of gprof capable of analyzing executable files and gmon.out > files from the router. This is the trickiest part, as I haven't found profiling tools for OpenWrt. I am curious to know if tcpdump does something interesting to deal with such situation (I dint find anything when I last read the code..couple of months back) Am I right to say that the two processes spin on some shared resource ? - Abhinav ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] tcpdump vs libpcap : CPU usage shooting high for two simultaneous captures on wireless monitor interfaces
On Apr 17, 2013, at 7:11 PM, abhinav narain wrote: > I am curious to know if tcpdump does something > interesting to deal with such situation (I dint find anything > when I last read the code..couple of months back) It does absolutely nothing whatsoever to discover whether there are other versions of tcpdump are running. > Am I right to say that the two processes spin on some shared resource ? Given that I don't know what routines in the programs or libraries they use are running when they're consuming that CPU time, I have absolutely no idea whether you're right or not to say that. That's why I wanted the profiling data - so that we *know* what routines are consuming the CPU time, not just *guessing*. Can you show us the source code to the parts of your program that make libpcap calls? ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] tcpdump vs libpcap : CPU usage shooting high for two simultaneous captures on wireless monitor interfaces
Hi Guy, Thanks for clearing this > It does absolutely nothing whatsoever to discover whether there are other > versions of tcpdump are running. Ok > Am I right to say that the two processes spin on some shared resource ? > > Given that I don't know what routines in the programs or libraries they > use are running when they're consuming that CPU time, I have absolutely no > idea whether you're right or not to say that. That's why I wanted the > profiling data - so that we *know* what routines are consuming the CPU > time, not just *guessing*. > > I understand your point, but unfortunately I am unable to provide you the information. Can you show us the source code to the parts of your program that make > libpcap calls? thanks for taking the pains to have a look at the code. The line has the pcap handle return. https://github.com/abnarain/wifi_dump-tmpfs/blob/master/mac-darktest.c#L782 I have modified the mac80211 to send me radiotap info with extra information, and hence i check to see the radiotap header length to differentiate between the packet and copy the various parts of buffer into a separate table, write it to a file and ship it from the router. The other fancy thing which I am doing is setting a SIGPROCMASK to capture alarms to get to a handler which writes into the file every x minute(and if the table is full) This file only deals with the pcap code. - Abhinav ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] pcap and select()
On Apr 17, 2013, at 6:56 PM, wen lui wrote:
>handle = pcap_open_live(dev, BUFSIZ, 0, 0, errbuf);
>pcap_compile(handle, &fp, filter_exp, 0, mask) == -1
>pcap_setfilter(handle, &fp);
>struct pcap_pkthdr pcap_header; // The header that pcap gives us
>const u_char *pcap_packet; // The actual packet
>
>while(1){
>
> n=fork();
> if(n==0) { // child process
>fd_set rdfds;
>int pcap_fd = pcap_get_selectable_fd(pcap_handler);
>for(;;){
>FD_ZERO(&rdfds);
>FD_SET(pcap_fd, &rdfds);
>FD_SET(sd_proxy, &rdfds); // here is another fd
>
>select(pcap_fd>sd_proxy?pcap_fd+1:sd_proxy+1,&rdfds,
> NULL, NULL, NULL)
Yes, that's at least a start at writing the kind of code needed for
http://stackoverflow.com/questions/16067015/blocking-pcap-and-multiple-i-o-select
You should be checking the return value of select() for errors, and reporting
errors if they occur, just to be sure.
>From the
>struct pcap_pkthdr pcap_header; // The header that pcap gives us
>const u_char *pcap_packet; // The actual packet
I suspect you're using pcap_next() or pcap_next_ex().
If you're going to use them with select(), you have to use them in a very
specific way.
First of all, you need to put the pcap_t in non-blocking mode. After the
pcap_open_live() call, do:
if (pcap_setnonblock(handle, 1, errbuf) == -1) {
fprintf(stderr, "pcap_setnonblock failed: %s\n", errbuf);
exit(2);
}
Then, of course, you check rdfds; if the bit for pcap_fd is set, do:
for (;;) {
switch (pcap_next_ex(handle, &pcap_header,
&pcap_packet)) {
case -1:
/* Error */
fprintf(stderr, "pcap_next_ex failed: %s\n",
pcap_geterr(handle));
exit(2);
break;
case 0:
/* No more packets to read for now */
goto done;
case -2:
/* Somebody called pcap_breakloop() */
goto done;
default:
/* Process the packet */
do whatever you do with pcap_header and
pcap_packet;
break;
}
done:
;
The "case -2" case can be left out if you don't call pcap_breakloop() in your
program.
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
