[tcpdump-workers] Hardware Timestamping Problem
Hello, (Sorry if this shows up twice, I wasn't registered for the List, when I first sent my request. And sorry if this is the wrong place for this kind of question.) I'm currently working on my Bachelors Thesis. The aim of my project is to accuratly and precisely timestamp Packages(it has to be at least precise in the tenth of microsecond range). I am trying to accomplish hardware timestamping with tcpdump, but there are some problems I have encountered. My Setup: 2 directly connected identical Servers. Linux: Debian 3.16.7 Network Interface: Intel i350-T4 Used tcpdump command: sudo /usr/sbin/tcpdump -i eth4 -s 59 port 3 -x -n -tt -v -j adapter_unsynced --time-stamp-precision=nano -w name.pcap Now, my results in itself make sense and would give me the desired results, but they have a big offset to them. 36 seconds to be exact. The jitter of my results with the -j adapter_unsynced option are about 10 µs, If I use the -j adapter option I get a Jitter of ~100µs( too imprecise for my application), but not the 36 second offset. All clocks are syncronised as far as I can see. The system clocks were both syncronised, and the PHC Clock of the NIC was too. As a test run I changed the system clock of one of the servers and the change showed up on the results while using -j adapter_unsynced. So my question is: What am I doing wrong, and is what I want to do even possible with tcpdump? I'm thankfull for any advice. Greetings, Christian ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Hardware Timestamping Problem
Christian wrote: > My Setup: > 2 directly connected identical Servers. > Linux: Debian 3.16.7 > Network Interface: Intel i350-T4 > Used tcpdump command: > sudo /usr/sbin/tcpdump -i eth4 -s 59 port 3 -x -n -tt -v -j > adapter_unsynced --time-stamp-precision=nano -w name.pcap > Now, my results in itself make sense and would give me the desired results, > but they have a big offset to them. 36 seconds to be exact. 36s between which point and which point? -- ] Never tell me the odds! | ipv6 mesh networks [ ] Michael Richardson, Sandelman Software Works| network architect [ ] m...@sandelman.ca http://www.sandelman.ca/| ruby on rails[ ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Hardware Timestamping Problem
{please keep it on the list for archival purposes}
Christian wrote:
> between sender and receiver
> so from the point when tcpdump grabs the time off of the Sender and to the
> point where tcpdump grabs the time off of the receiver.
So you are running a tcpdump on sender, using the timestamping service from
the hardware, and the same on receiver, and then you send traffic?
How much traffic? Is it being buffered?
How do you correlate traffic?
How are the *hardware* clocks being synchronzied?
AFAIK, no NICs timestamp traffic on outgoing, tcpdump captures before the
hardware queue. A big hardware send buffer would result in significant skew.
--
] Never tell me the odds! | ipv6 mesh networks [
] Michael Richardson, Sandelman Software Works| network architect [
] m...@sandelman.ca http://www.sandelman.ca/| ruby on rails[
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Hardware Timestamping Problem
Yes, one tcpdump entity per server, after Both are started I start a
Package Generator which sends a 900kbs Trace.
There shouldn't be a big buffering effect, if I'm not using the hardware
timestamp options the one way delay is at a few 100 µs, or am I
misunderstanding something?
The 36 seconds are only showing up, when I'm using the -j
adapter_unsynced option.
The goal is to measure the one way delay, so I simply take the timestamp
of the Receiver and subtract the timestamp of the sender.
I'm using ptp4l for the synchronisation.(I have run ptp oder the same
link as the measurement, and on a different link, synchronising the
clocks with phc2sys. Both implementations had the 36s error)
Am 08.06.2016 um 16:24 schrieb Michael Richardson:
{please keep it on the list for archival purposes}
Christian wrote:
> between sender and receiver
> so from the point when tcpdump grabs the time off of the Sender and to
the
> point where tcpdump grabs the time off of the receiver.
So you are running a tcpdump on sender, using the timestamping service from
the hardware, and the same on receiver, and then you send traffic?
How much traffic? Is it being buffered?
How do you correlate traffic?
How are the *hardware* clocks being synchronzied?
AFAIK, no NICs timestamp traffic on outgoing, tcpdump captures before the
hardware queue. A big hardware send buffer would result in significant skew.
--
] Never tell me the odds! | ipv6 mesh networks [
] Michael Richardson, Sandelman Software Works| network architect [
] m...@sandelman.ca http://www.sandelman.ca/| ruby on rails[
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Hardware Timestamping Problem
On Jun 8, 2016, at 5:53 AM, Christian wrote: > Now, my results in itself make sense and would give me the desired results, > but they have a big offset to them. 36 seconds to be exact. So you're saying there's a 36-second offset between which two times? ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Hardware Timestamping Problem
The Timestamp when tcpdump grabs the package off of the receiver is 36 seconds( +/- innaccuracy, here roughly +/- 5-10 µs) after the timestamp when tcpdump grabs the package of the sender. resulting in an alleged One Way Delay of 36 seconds which wouldn't make any sense in that scenario, given that the software timestamping option and the -j adapter option both result in a ~ 100-200µs one way delay Am 08.06.2016 um 22:13 schrieb Guy Harris: On Jun 8, 2016, at 5:53 AM, Christian wrote: Now, my results in itself make sense and would give me the desired results, but they have a big offset to them. 36 seconds to be exact. So you're saying there's a 36-second offset between which two times? ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Hardware Timestamping Problem
On Jun 8, 2016, at 1:29 PM, Christian Rupp wrote: > The Timestamp when tcpdump grabs the package off of the receiver is 36 > seconds( +/- innaccuracy, here roughly +/- 5-10 µs) after the timestamp when > tcpdump grabs the package of the sender. "The" packet in the sense that you have two tcpdumps, one running on each of the servers, and the time stamp for a single given packet captured by the tcpdump on the machine that sent it is about 36 seconds earlier than the time stamp for the *exact same* packet captured by the tcpdump on the machine that received it? ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Hardware Timestamping Problem
Christian Rupp wrote: > The Timestamp when tcpdump grabs the package off of the receiver is 36 > seconds( +/- innaccuracy, here roughly +/- 5-10 µs) after the timestamp when > tcpdump grabs the package of the sender. resulting in an alleged One > Way > Delay of 36 seconds which wouldn't make any sense in that scenario, given > that the software timestamping option and the -j adapter option both result > in a ~ 100-200µs one way delay The timestamp on the sender is long before the hardware. You are saying that the stamp is correct when you do not use the -j option? if that's the case, then it seems like it's the hardware which is mis-stamping. -- ] Never tell me the odds! | ipv6 mesh networks [ ] Michael Richardson, Sandelman Software Works| network architect [ ] m...@sandelman.ca http://www.sandelman.ca/| ruby on rails[ ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Hardware Timestamping Problem
> Am 08.06.2016 um 23:10 schrieb Michael Richardson : > > Christian Rupp wrote: >> The Timestamp when tcpdump grabs the package off of the receiver is 36 >> seconds( +/- innaccuracy, here roughly +/- 5-10 µs) after the timestamp when >> tcpdump grabs the package of the sender. resulting in an alleged One >> Way >> Delay of 36 seconds which wouldn't make any sense in that scenario, given >> that the software timestamping option and the -j adapter option both result >> in a ~ 100-200µs one way delay > > The timestamp on the sender is long before the hardware. > You are saying that the stamp is correct when you do not use the -j option? > > if that's the case, then it seems like it's the hardware which is > mis-stamping. Hardware timestamping of sending/receiving buffer descriptors is done by NIC. Configured by the NIC-driver (timer resolution of NIC timestamping clock). Setup by the user space application via kernel interface(s) [1] Therefore I would try to run a recent Linux kernel and a recent tcpdump. You may even change to another NIC for testing. [1] https://www.kernel.org/doc/Documentation/networking/timestamping.txt ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
