[tcpdump-workers] How tcpdump determines the "dropped by kernel"?
I have been reading the man pages of tcpdump and I am not sure if my OS will report the relevant info. Since I would not like to research tcpdump code I would like to get some help about it from others. So my kernel would declare on packets that was dropped but still the connection was OK and was not disrupted in any way I can think about. What exactly this "drop by kernel" means? Is it dropped by kernel and was not handled by any application? or it means that the buffers of tcpdump got filled and there-for was dropped by tcpdump? I am not sure I am even asking the right question but this is how it seems to me. In any case I would like to do a very big dump into a storage system on a very loaded system and which I would like to not drop any packet by either the kernel or any other level if possible. In a case there are tuning to the system in couple layers I would like to at least minimize the drops from lots of packets into a small amount of packets. Thanks in Advance, Eliezer ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] How tcpdump determines the "dropped by kernel"?
Hey Guy, Thanks for the detailed response. I am running Linux on couple systems: Gentoo, Ubuntu 10.04+newers, CentOS. On the ubuntu that I am using now: tcpdump version 4.4.0 libpcap version 1.4.0 On the CentOS it's the exact same version output: tcpdump version 4.4.0 libpcap version 1.4.0 For one system I have about 15MB/s and on the others it's much higher and maybe in the Hundreds of MB/s depends on the load and the system size. It can be 8 cores 32GB ram which is in one case. So In a case there is not much ram limitation for the machine I would thing that an option to use more ram for these buffers can be an option. Thanks, Eliezer On 25/11/13 20:07, Guy Harris wrote: On Nov 24, 2013, at 5:04 PM, Eliezer Croitoru wrote: Since I would not like to research tcpdump code I would like to get some help about it from others. So my kernel would declare on packets that was dropped but still the connection was OK and was not disrupted in any way I can think about. What exactly this "drop by kernel" means? Is it dropped by kernel and was not handled by any application? or it means that the buffers of tcpdump got filled and there-for was dropped by tcpdump? It means that: tcpdump uses libpcap to do packet capture; libpcap uses some mechanism or driver in the OS kernel to do packet capture; that mechanism has, for each capture in progress on each network interface, buffers into which copies of packets are placed; if *those* buffers fill up, because tcpdump (or whatever application is capturing) isn't processing the packets fast enough, any packets that arrive while the buffers are full are not copied to a buffer for capturing on that interface. That doesn't mean that the packets aren't delivered to the OS networking stack (or to other captures being done on the same device). In any case I would like to do a very big dump into a storage system on a very loaded system and which I would like to not drop any packet by either the kernel or any other level if possible. In a case there are tuning to the system in couple layers I would like to at least minimize the drops from lots of packets into a small amount of packets. What OS are you capturing on, and what version of libpcap is tcpdump using (run "tcpdump -h" to get the libpcap version)? ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] How tcpdump determines the "dropped by kernel"?
Hey, Yes in high load it can cause some troubles. The solution I could think about was a dedicated machine that would receive all traffic from the replication(HUB-like) port while the machine Ethernet is on promiscuous mode which will then capture all traffic from the network. I do not know exactly how much resources it would take when there is an option for let say "pin" tcpdump to one or two cores while letting all the others handle the rest of the traffic. For a very high load I would need a big buffer or maybe a frame-buffer card that will help to reduce the load on the kernel while allowing less packet drops. Eliezer On 25/11/13 16:23, Prashant Batra (prbatra) wrote: Hi, To add to tcpdump application eating out resources, it would degrade the performance of send/receive path in the kernel. Each packet going out from the kernel and received in would be cloned and then given to tcpdump application. At very high load this would be significant. Regards, Prashant ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] How tcpdump determines the "dropped by kernel"?
Ho ok. On 25/11/13 21:28, Guy Harris wrote: On Nov 25, 2013, at 11:01 AM, Eliezer Croitoru wrote: >I am running Linux on couple systems: Gentoo, Ubuntu 10.04+newers, CentOS. What kernel version? I have one 2.6.32-X in the CentOS. Ubuntu has 3.2+ kernels(3.2,3.4,3.7..) Gentoo is another story but it's similar. >On the ubuntu that I am using now: >tcpdump version 4.4.0 >libpcap version 1.4.0 > >On the CentOS it's the exact same version output: If you're running on a system with a 3.2 or later kernel, then, if you use libpcap built from the current Git trunk, it can use version 3 of the memory-mapped capture mechanism (TPACKET_V3), which makes more efficient use of the capture mechanism's buffers than do earlier versions of that mechanism (TPACKET_V1 and TPACKET_V2), resulting in fewer packet drops. I hope ubuntu have support for these but it seems like 2MB on linux can be increased easily to more then 50MB on this machine since I do have lots of ram free (6GB) is more then is needed for the task from my way of thinking. I will try to test it later. Thanks again, Eliezer >So In a case there is not much ram limitation for the machine I would thing that an option to use more ram for these buffers can be an option. Yes - that's what the -B flag to tcpdump lets you do. (The default is 2MB on Linux.) ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
