On Thu, 23 Aug 2007, Rick Jones wrote:
> jamal wrote:
> > [TSO already passed - iirc, it has been
> > demostranted to really not add much to throughput (cant improve much
> > over closeness to wire speed) but improve CPU utilization].
>
> In the one gig space sure, but in the 10 Gig space, TSO on/off does make a
> difference for throughput.
Not too much.
TSO enabled:
[EMAIL PROTECTED] ~]# ethtool -k eth2
Offload parameters for eth2:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp segmentation offload: on
[EMAIL PROTECTED] ~]# nuttcp -w10m 192.168.88.16
11813.4375 MB / 10.00 sec = 9906.1644 Mbps 99 %TX 80 %RX
TSO disabled:
[EMAIL PROTECTED] ~]# ethtool -K eth2 tso off
[EMAIL PROTECTED] ~]# ethtool -k eth2
Offload parameters for eth2:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp segmentation offload: off
[EMAIL PROTECTED] ~]# nuttcp -w10m 192.168.88.16
11818.2500 MB / 10.00 sec = 9910.0176 Mbps 100 %TX 78 %RX
Pretty negligible difference it seems.
This is with a 2.6.20.7 kernel, Myricom 10-GigE NICs, and 9000 byte
jumbo frames, in a LAN environment.
For grins, I also did a couple of tests with an MSS of 1460 to
emulate a standard 1500 byte Ethernet MTU.
TSO enabled:
[EMAIL PROTECTED] ~]# ethtool -k eth2
Offload parameters for eth2:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp segmentation offload: on
[EMAIL PROTECTED] ~]# nuttcp -M1460 -w10m 192.168.88.16
5102.8503 MB / 10.06 sec = 4253.9124 Mbps 39 %TX 99 %RX
TSO disabled:
[EMAIL PROTECTED] ~]# ethtool -K eth2 tso off
[EMAIL PROTECTED] ~]# ethtool -k eth2
Offload parameters for eth2:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp segmentation offload: off
[EMAIL PROTECTED] ~]# nuttcp -M1460 -w10m 192.168.88.16
5399.5625 MB / 10.00 sec = 4527.9070 Mbps 99 %TX 76 %RX
Here you can see there is a major difference in the TX CPU utilization
(99 % with TSO disabled versus only 39 % with TSO enabled), although
the TSO disabled case was able to squeeze out a little extra performance
from its extra CPU utilization. Interestingly, with TSO enabled, the
receiver actually consumed more CPU than with TSO disabled, so I guess
the receiver CPU saturation in that case (99 %) was what restricted
its performance somewhat (this was consistent across a few test runs).
-Bill
-
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
