Den sön 3 mars 2019 kl 21:35 skrev Theo de Raadt <[email protected]>:
> Tony, > > Are you out of your mind? You didn't provide even a rough hint about > what your firewall configuration looks like. You recognize that's > pathetic, right? > > > Earlier in the week I could run parallel ping-pong tests through my test > > firewalls > > at 300kpps without any packet loss. I updated to the latest snapshot > today > > and > > start to see packet loss at around 80kpps. > > > > /T > > > > OpenBSD 6.5-beta (GENERIC.MP) #764: Sun Mar 3 10:24:08 MST 2019 > > [email protected]:/usr/src/sys/arch/amd64/compile/GENERIC.MP > > real mem = 34300891136 (32711MB) > > avail mem = 33251393536 (31711MB) > > mpath0 at root > > scsibus0 at mpath0: 256 targets > > mainbus0 at root > > bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xec170 (34 entries) > > bios0: vendor American Megatrends Inc. version "3.0" date 04/24/2015 > > bios0: Supermicro X10SLD > > acpi0 at bios0: rev 2 > > acpi0: sleep states S0 S4 S5 > > acpi0: tables DSDT FACP APIC FPDT FIDT SSDT SSDT MCFG PRAD HPET SSDT SSDT > > SPMI DMAR EINJ ERST HEST BERT > > acpi0: wakeup devices PEGP(S4) PEG0(S4) PEGP(S4) PEG1(S4) PEGP(S4) > PEG2(S4) > > PXSX(S4) RP01(S4) PXSX(S4) RP02(S4) PXSX(S4) RP03(S4) PXSX(S4) RP04(S4) > > PXSX(S4) RP05(S4) [...] > > acpitimer0 at acpi0: 3579545 Hz, 24 bits > > acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat > > cpu0 at mainbus0: apid 0 (boot processor) > > cpu0: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.68 MHz, 06-3c-03 > > cpu0: > > > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN > > cpu0: 256KB 64b/line 8-way L2 cache > > cpu0: smt 0, core 0, package 0 > > mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges > > cpu0: apic clock running at 99MHz > > cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4, IBE > > cpu1 at mainbus0: apid 2 (application processor) > > cpu1: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.01 MHz, 06-3c-03 > > cpu1: > > > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN > > cpu1: 256KB 64b/line 8-way L2 cache > > cpu1: smt 0, core 1, package 0 > > cpu2 at mainbus0: apid 4 (application processor) > > cpu2: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.01 MHz, 06-3c-03 > > cpu2: > > > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN > > cpu2: 256KB 64b/line 8-way L2 cache > > cpu2: smt 0, core 2, package 0 > > cpu3 at mainbus0: apid 6 (application processor) > > cpu3: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.01 MHz, 06-3c-03 > > cpu3: > > > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN > > cpu3: 256KB 64b/line 8-way L2 cache > > cpu3: smt 0, core 3, package 0 > > ioapic0 at mainbus0: apid 8 pa 0xfec00000, version 20, 24 pins > > acpimcfg0 at acpi0 > > acpimcfg0: addr 0xf8000000, bus 0-63 > > acpihpet0 at acpi0: 14318179 Hz > > acpiprt0 at acpi0: bus 0 (PCI0) > > acpiprt1 at acpi0: bus 1 (PEG0) > > acpiprt2 at acpi0: bus 2 (PEG1) > > acpiprt3 at acpi0: bus -1 (PEG2) > > acpiprt4 at acpi0: bus 3 (RP01) > > acpiprt5 at acpi0: bus -1 (RP02) > > acpiprt6 at acpi0: bus -1 (RP03) > > acpiprt7 at acpi0: bus -1 (RP04) > > acpiprt8 at acpi0: bus -1 (RP05) > > acpiprt9 at acpi0: bus -1 (RP06) > > acpiprt10 at acpi0: bus -1 (RP07) > > acpiprt11 at acpi0: bus -1 (RP08) > > acpiec0 at acpi0: not present > > acpicpu0 at acpi0: C1(@1 halt!) > > acpicpu1 at acpi0: C1(@1 halt!) > > acpicpu2 at acpi0: C1(@1 halt!) > > acpicpu3 at acpi0: C1(@1 halt!) > > acpipwrres0 at acpi0: PG00, resource for PEG0 > > acpipwrres1 at acpi0: PG01, resource for PEG1 > > acpipwrres2 at acpi0: PG02, resource for PEG2 > > acpipwrres3 at acpi0: FN00, resource for FAN0 > > acpipwrres4 at acpi0: FN01, resource for FAN1 > > acpipwrres5 at acpi0: FN02, resource for FAN2 > > acpipwrres6 at acpi0: FN03, resource for FAN3 > > acpipwrres7 at acpi0: FN04, resource for FAN4 > > acpitz0 at acpi0: critical temperature is 105 degC > > acpitz1 at acpi0: critical temperature is 105 degC > > acpipci0 at acpi0 PCI0: 0x00000000 0x00000011 0x00000001 > > acpicmos0 at acpi0 > > "IPI0001" at acpi0 not configured > > acpibtn0 at acpi0: SLPB > > acpibtn1 at acpi0: PWRB > > "PNP0C0B" at acpi0 not configured > > "PNP0C0B" at acpi0 not configured > > "PNP0C0B" at acpi0 not configured > > "PNP0C0B" at acpi0 not configured > > "PNP0C0B" at acpi0 not configured > > acpivideo0 at acpi0: GFX0 > > acpivout0 at acpivideo0: DD1F > > ipmi at mainbus0 not configured > > pci0 at mainbus0 bus 0 > > pchb0 at pci0 dev 0 function 0 "Intel Xeon E3-1200 v3 Host" rev 0x06 > > ppb0 at pci0 dev 1 function 0 "Intel Core 4G PCIE" rev 0x06: msi > > pci1 at ppb0 bus 1 > > ix0 at pci1 dev 0 function 0 "Intel 82599" rev 0x01: msi, address > > 0c:c4:7a:59:30:a4 > > ix1 at pci1 dev 0 function 1 "Intel 82599" rev 0x01: msi, address > > 0c:c4:7a:59:30:a5 > > ppb1 at pci0 dev 1 function 1 "Intel Core 4G PCIE" rev 0x06: msi > > pci2 at ppb1 bus 2 > > ixl0 at pci2 dev 0 function 0 "Intel XL710 QSFP+" rev 0x02: port 0, FW > > 5.0.40043 API 1.5, msi, address 0c:c4:7a:5e:f9:c8 > > ixl0: unable to query phy types > > ixl1 at pci2 dev 0 function 1 "Intel XL710 QSFP+" rev 0x02: port 1, FW > > 5.0.40043 API 1.5, msi, address 0c:c4:7a:5e:f9:c9 > > ixl1: unable to query phy types > > "Intel 8 Series MEI" rev 0x04 at pci0 dev 22 function 0 not configured > > "Intel 8 Series MEI" rev 0x04 at pci0 dev 22 function 1 not configured > > ehci0 at pci0 dev 26 function 0 "Intel 8 Series USB" rev 0x05: apic 8 > int 16 > > usb0 at ehci0: USB revision 2.0 > > uhub0 at usb0 configuration 1 interface 0 "Intel EHCI root hub" rev > > 2.00/1.00 addr 1 > > ppb2 at pci0 dev 28 function 0 "Intel 8 Series PCIE" rev 0xd5: msi > > pci3 at ppb2 bus 3 > > ppb3 at pci3 dev 0 function 0 "ASPEED Technology AST1150 PCI" rev 0x03 > > pci4 at ppb3 bus 4 > > vga1 at pci4 dev 0 function 0 "ASPEED Technology AST2000" rev 0x30 > > wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) > > wsdisplay0: screen 1-5 added (80x25, vt100 emulation) > > ehci1 at pci0 dev 29 function 0 "Intel 8 Series USB" rev 0x05: apic 8 > int 22 > > usb1 at ehci1: USB revision 2.0 > > uhub1 at usb1 configuration 1 interface 0 "Intel EHCI root hub" rev > > 2.00/1.00 addr 1 > > pcib0 at pci0 dev 31 function 0 "Intel C224 LPC" rev 0x05 > > ahci0 at pci0 dev 31 function 2 "Intel 8 Series AHCI" rev 0x05: msi, AHCI > > 1.3 > > ahci0: port 0: 6.0Gb/s > > ahci0: port 1: 6.0Gb/s > > scsibus1 at ahci0: 32 targets > > sd0 at scsibus1 targ 0 lun 0: <ATA, INTEL SSDSC2BB12, G201> SCSI3 > 0/direct > > fixed naa.55cd2e404c022db4 > > sd0: 114473MB, 512 bytes/sector, 234441648 sectors, thin > > sd1 at scsibus1 targ 1 lun 0: <ATA, INTEL SSDSC2BB12, G201> SCSI3 > 0/direct > > fixed naa.55cd2e404c022518 > > sd1: 114473MB, 512 bytes/sector, 234441648 sectors, thin > > ichiic0 at pci0 dev 31 function 3 "Intel 8 Series SMBus" rev 0x05: apic 8 > > int 18 > > iic0 at ichiic0 > > sdtemp0 at iic0 addr 0x18: stts2002 > > sdtemp1 at iic0 addr 0x19: stts2002 > > sdtemp2 at iic0 addr 0x1a: stts2002 > > sdtemp3 at iic0 addr 0x1b: stts2002 > > spdmem0 at iic0 addr 0x50: 8GB DDR3 SDRAM ECC PC3-12800 with thermal > sensor > > spdmem1 at iic0 addr 0x51: 8GB DDR3 SDRAM ECC PC3-12800 with thermal > sensor > > spdmem2 at iic0 addr 0x52: 8GB DDR3 SDRAM ECC PC3-12800 with thermal > sensor > > spdmem3 at iic0 addr 0x53: 8GB DDR3 SDRAM ECC PC3-12800 with thermal > sensor > > "Intel 8 Series Thermal" rev 0x05 at pci0 dev 31 function 6 not > configured > > isa0 at pcib0 > > isadma0 at isa0 > > com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo > > com2 at isa0 port 0x3e8/8 irq 5: ns16550a, 16 byte fifo > > pckbc0 at isa0 port 0x60/5 irq 1 irq 12 > > pcppi0 at isa0 port 0x61 > > spkr0 at pcppi0 > > wbsio0 at isa0 port 0x2e/2: NCT6776F rev 0x33 > > lm1 at wbsio0 port 0xa30/8: NCT6776F > > vmm0 at mainbus0: VMX/EPT > > uhub2 at uhub0 port 1 configuration 1 interface 0 "Intel Rate Matching > Hub" > > rev 2.00/0.05 addr 2 > > uhub3 at uhub2 port 3 configuration 1 interface 0 "ATEN International > > product 0x7000" rev 2.00/0.00 addr 3 > > uhidev0 at uhub3 port 1 configuration 1 interface 0 "ATEN International > > product 0x2419" rev 1.10/1.00 addr 4 > > uhidev0: iclass 3/1 > > ukbd0 at uhidev0: 8 variable keys, 6 key codes > > wskbd0 at ukbd0: console keyboard, using wsdisplay0 > > uhidev1 at uhub3 port 1 configuration 1 interface 1 "ATEN International > > product 0x2419" rev 1.10/1.00 addr 4 > > uhidev1: iclass 3/1 > > ums0 at uhidev1: 3 buttons, Z dir > > wsmouse0 at ums0 mux 0 > > uhub4 at uhub1 port 1 configuration 1 interface 0 "Intel Rate Matching > Hub" > > rev 2.00/0.05 addr 2 > > vscsi0 at root > > scsibus2 at vscsi0: 256 targets > > softraid0 at root > > scsibus3 at softraid0: 256 targets > > root on sd0a (705899aa0befef4d.a) swap on sd0b dump on sd0b > I have a firewall test setup consisting of 14 machines. All old E3's, with a mix of ix and em, and some 2x40G ixl(work in progress). Firewalls interfaces are configured as: 2x ix => trunk(lacp) => vlans+carps 14 vlan interfaces and 13 carps (no carp on pfsync vlan interface) set skip on lo and the vlan carrying pfsync. No shaping or other funkiness, just pass/block. As this is a forwarding performance test with very low session rate the rule-set is irrelevant. The test performed is ping-pong of UDP packets across the firewall at the speed of RTT, additional clients are added to increase load. Testing with OpenBSD 6.5-beta (GENERIC.MP) #764: Sun Mar 3 10:24:08 MST 2019 Baseline, single session between testers. OpenBSD 6.5-beta (GENERIC.MP) #764: Sun Mar 3 10:24:08 MST 2019 [email protected]$ route -T2 exec ./phantom -k 1234 -c 25000 10.96.2.21:2222 packet size: 512 packets per client: 25000 number of clients: 1 server address: 10.96.2.21:2222 pps: 4016 total drops: 0 avg rtt: 0.249 ms clients: 1 pps: 4015 total drops: 0 avg rtt: 0.249 ms clients: 1 pps: 3905 total drops: 0 avg rtt: 0.256 ms clients: 1 pps: 4016 total drops: 0 avg rtt: 0.249 ms clients: 1 pps: 4015 total drops: 0 avg rtt: 0.249 ms clients: 1 pps: 3976 total drops: 0 avg rtt: 0.252 ms clients: 1 Runtime: 7.118854881s Packets received: 25000 Packets dropped: 0 [email protected]$ The pps here is packets sent (and received), so firewall pps on trunk is double the above number. A ramp-up test where one client per second is added until packet loss: [email protected]$ route -T2 exec ./phantom -k 1234 -r 1 10.96.2.21:2222 packet size: 512 ramp-up interval: 1 seconds packets per client: 9223372036854775807 number of clients: 9223372036854775807 server address: 10.96.2.21:2222 pps: 4051 total drops: 0 avg rtt: 0.494 ms clients: 2 pps: 5736 total drops: 0 avg rtt: 0.523 ms clients: 3 pps: 8258 total drops: 0 avg rtt: 0.484 ms clients: 4 pps: 10852 total drops: 0 avg rtt: 0.461 ms clients: 5 pps: 13174 total drops: 0 avg rtt: 0.455 ms clients: 6 pps: 15417 total drops: 0 avg rtt: 0.454 ms clients: 7 pps: 17894 total drops: 0 avg rtt: 0.447 ms clients: 8 pps: 20010 total drops: 0 avg rtt: 0.450 ms clients: 9 pps: 22178 total drops: 0 avg rtt: 0.451 ms clients: 10 pps: 24252 total drops: 0 avg rtt: 0.454 ms clients: 11 pps: 26056 total drops: 0 avg rtt: 0.461 ms clients: 12 pps: 28060 total drops: 0 avg rtt: 0.463 ms clients: 13 pps: 30437 total drops: 0 avg rtt: 0.460 ms clients: 14 pps: 32503 total drops: 0 avg rtt: 0.461 ms clients: 15 pps: 33421 total drops: 0 avg rtt: 0.479 ms clients: 16 pps: 35810 total drops: 0 avg rtt: 0.475 ms clients: 17 pps: 37042 total drops: 0 avg rtt: 0.486 ms clients: 18 pps: 38626 total drops: 0 avg rtt: 0.492 ms clients: 19 read failed: read udp 10.96.3.52:43580->10.96.2.21:2222: i/o timeout pps: 38045 total drops: 1 avg rtt: 0.526 ms clients: 20 read failed: read udp 10.96.3.52:3459->10.96.2.21:2222: i/o timeout Runtime: 20.198108441s Packets received: 472256 Packets dropped: 2 [email protected]$ Packet loss starts at around 35-40kpps: [email protected]$ route -T2 exec ./phantom -k 1234 -c 25000 -n 17 10.96.2.21:2222 packet size: 512 packets per client: 25000 number of clients: 17 server address: 10.96.2.21:2222 pps: 35234 total drops: 0 avg rtt: 0.482 ms clients: 17 pps: 37294 total drops: 0 avg rtt: 0.456 ms clients: 17 pps: 36400 total drops: 0 avg rtt: 0.467 ms clients: 17 pps: 37040 total drops: 0 avg rtt: 0.459 ms clients: 17 pps: 37081 total drops: 0 avg rtt: 0.458 ms clients: 17 pps: 35258 total drops: 0 avg rtt: 0.482 ms clients: 17 read failed: read udp 10.96.3.52:45255->10.96.2.21:2222: i/o timeout pps: 36918 total drops: 1 avg rtt: 0.460 ms clients: 17 ^C Runtime: 8.029600463s Packets received: 285855 Packets dropped: 1 [email protected]$ Now testing with OpenBSD 6.5-beta (GENERIC.MP) #760: Thu Feb 28 12:38:40 MST 2019. Single-session base line: [email protected]$ route -T2 exec ./phantom -k 1234 -c 25000 10.96.2.21:2222 packet size: 512 packets per client: 25000 number of clients: 1 server address: 10.96.2.21:2222 pps: 3640 total drops: 0 avg rtt: 0.275 ms clients: 1 pps: 3568 total drops: 0 avg rtt: 0.280 ms clients: 1 pps: 3638 total drops: 0 avg rtt: 0.275 ms clients: 1 pps: 3637 total drops: 0 avg rtt: 0.275 ms clients: 1 pps: 3641 total drops: 0 avg rtt: 0.275 ms clients: 1 pps: 3674 total drops: 0 avg rtt: 0.272 ms clients: 1 Runtime: 7.131570891s Packets received: 25000 Packets dropped: 0 [email protected]$ Ramp-up test: [email protected]$ route -T2 exec ./phantom -k 1234 -r 1 10.96.2.21:2222 [2/1836] packet size: 512 ramp-up interval: 1 seconds packets per client: 9223372036854775807 number of clients: 9223372036854775807 server address: 10.96.2.21:2222 pps: 3827 total drops: 0 avg rtt: 0.523 ms clients: 2 pps: 5686 total drops: 0 avg rtt: 0.528 ms clients: 3 pps: 8278 total drops: 0 avg rtt: 0.483 ms clients: 4 pps: 10807 total drops: 0 avg rtt: 0.463 ms clients: 5 pps: 13402 total drops: 0 avg rtt: 0.448 ms clients: 6 pps: 15878 total drops: 0 avg rtt: 0.441 ms clients: 7 pps: 18432 total drops: 0 avg rtt: 0.434 ms clients: 8 pps: 20948 total drops: 0 avg rtt: 0.430 ms clients: 9 pps: 23393 total drops: 0 avg rtt: 0.427 ms clients: 10 pps: 25905 total drops: 0 avg rtt: 0.425 ms clients: 11 pps: 28186 total drops: 0 avg rtt: 0.426 ms clients: 12 pps: 29716 total drops: 0 avg rtt: 0.437 ms clients: 13 pps: 31801 total drops: 0 avg rtt: 0.440 ms clients: 14 pps: 32901 total drops: 0 avg rtt: 0.456 ms clients: 15 pps: 34493 total drops: 0 avg rtt: 0.464 ms clients: 16 pps: 35223 total drops: 0 avg rtt: 0.483 ms clients: 17 pps: 36824 total drops: 0 avg rtt: 0.489 ms clients: 18 pps: 38655 total drops: 0 avg rtt: 0.492 ms clients: 19 pps: 39625 total drops: 0 avg rtt: 0.505 ms clients: 20 pps: 41467 total drops: 0 avg rtt: 0.506 ms clients: 21 pps: 43342 total drops: 0 avg rtt: 0.508 ms clients: 22 pps: 44565 total drops: 0 avg rtt: 0.516 ms clients: 23 pps: 47022 total drops: 0 avg rtt: 0.510 ms clients: 24 pps: 48825 total drops: 0 avg rtt: 0.512 ms clients: 25 pps: 49426 total drops: 0 avg rtt: 0.526 ms clients: 26 pps: 51010 total drops: 0 avg rtt: 0.529 ms clients: 27 pps: 53227 total drops: 0 avg rtt: 0.526 ms clients: 28 pps: 55427 total drops: 0 avg rtt: 0.523 ms clients: 29 pps: 56294 total drops: 0 avg rtt: 0.533 ms clients: 30 pps: 57202 total drops: 0 avg rtt: 0.542 ms clients: 31 pps: 58672 total drops: 0 avg rtt: 0.545 ms clients: 32 pps: 60757 total drops: 0 avg rtt: 0.543 ms clients: 33 pps: 61619 total drops: 0 avg rtt: 0.552 ms clients: 34 pps: 62094 total drops: 0 avg rtt: 0.564 ms clients: 35 pps: 65039 total drops: 0 avg rtt: 0.554 ms clients: 36 pps: 65980 total drops: 0 avg rtt: 0.561 ms clients: 37 pps: 68124 total drops: 0 avg rtt: 0.558 ms clients: 38 pps: 68712 total drops: 0 avg rtt: 0.568 ms clients: 39 pps: 71362 total drops: 0 avg rtt: 0.561 ms clients: 40 pps: 72715 total drops: 0 avg rtt: 0.564 ms clients: 41 pps: 72362 total drops: 0 avg rtt: 0.580 ms clients: 42 pps: 72946 total drops: 0 avg rtt: 0.589 ms clients: 43 pps: 77915 total drops: 0 avg rtt: 0.565 ms clients: 44 pps: 77999 total drops: 0 avg rtt: 0.577 ms clients: 45 pps: 78544 total drops: 0 avg rtt: 0.586 ms clients: 46 pps: 80823 total drops: 0 avg rtt: 0.582 ms clients: 47 pps: 82208 total drops: 0 avg rtt: 0.584 ms clients: 48 pps: 83035 total drops: 0 avg rtt: 0.590 ms clients: 49 pps: 84476 total drops: 0 avg rtt: 0.592 ms clients: 50 pps: 86276 total drops: 0 avg rtt: 0.591 ms clients: 51 pps: 86409 total drops: 0 avg rtt: 0.602 ms clients: 52 pps: 87482 total drops: 0 avg rtt: 0.606 ms clients: 53 pps: 88760 total drops: 0 avg rtt: 0.608 ms clients: 54 pps: 90796 total drops: 0 avg rtt: 0.606 ms clients: 55 pps: 92476 total drops: 0 avg rtt: 0.606 ms clients: 56 pps: 94324 total drops: 0 avg rtt: 0.604 ms clients: 57 pps: 96194 total drops: 0 avg rtt: 0.603 ms clients: 58 pps: 98696 total drops: 0 avg rtt: 0.598 ms clients: 59 pps: 99291 total drops: 0 avg rtt: 0.604 ms clients: 60 pps: 100693 total drops: 0 avg rtt: 0.606 ms clients: 61 pps: 103156 total drops: 0 avg rtt: 0.601 ms clients: 62 pps: 104484 total drops: 0 avg rtt: 0.603 ms clients: 63 pps: 103819 total drops: 0 avg rtt: 0.616 ms clients: 64 pps: 105292 total drops: 0 avg rtt: 0.617 ms clients: 65 pps: 107878 total drops: 0 avg rtt: 0.612 ms clients: 66 pps: 108857 total drops: 0 avg rtt: 0.615 ms clients: 67 pps: 108004 total drops: 0 avg rtt: 0.630 ms clients: 68 pps: 108584 total drops: 0 avg rtt: 0.635 ms clients: 69 pps: 108634 total drops: 0 avg rtt: 0.644 ms clients: 70 pps: 109416 total drops: 0 avg rtt: 0.649 ms clients: 71 pps: 111989 total drops: 0 avg rtt: 0.643 ms clients: 72 pps: 112612 total drops: 0 avg rtt: 0.648 ms clients: 73 pps: 113058 total drops: 0 avg rtt: 0.655 ms clients: 74 pps: 114198 total drops: 0 avg rtt: 0.657 ms clients: 75 pps: 116640 total drops: 0 avg rtt: 0.652 ms clients: 76 pps: 117816 total drops: 0 avg rtt: 0.654 ms clients: 77 pps: 117980 total drops: 0 avg rtt: 0.661 ms clients: 78 pps: 120047 total drops: 0 avg rtt: 0.658 ms clients: 79 pps: 123420 total drops: 0 avg rtt: 0.648 ms clients: 80 pps: 123739 total drops: 0 avg rtt: 0.655 ms clients: 81 pps: 125930 total drops: 0 avg rtt: 0.651 ms clients: 82 pps: 127521 total drops: 0 avg rtt: 0.651 ms clients: 83 pps: 128499 total drops: 0 avg rtt: 0.654 ms clients: 84 pps: 129330 total drops: 0 avg rtt: 0.657 ms clients: 85 pps: 131785 total drops: 0 avg rtt: 0.653 ms clients: 86 pps: 131340 total drops: 0 avg rtt: 0.662 ms clients: 87 pps: 131660 total drops: 0 avg rtt: 0.668 ms clients: 88 pps: 133302 total drops: 0 avg rtt: 0.668 ms clients: 89 pps: 131994 total drops: 0 avg rtt: 0.682 ms clients: 90 pps: 133166 total drops: 0 avg rtt: 0.683 ms clients: 91 pps: 132691 total drops: 0 avg rtt: 0.693 ms clients: 92 pps: 132280 total drops: 0 avg rtt: 0.703 ms clients: 93 pps: 131950 total drops: 0 avg rtt: 0.712 ms clients: 94 pps: 133879 total drops: 0 avg rtt: 0.710 ms clients: 95 pps: 132591 total drops: 0 avg rtt: 0.724 ms clients: 96 pps: 131854 total drops: 0 avg rtt: 0.736 ms clients: 97 pps: 132559 total drops: 0 avg rtt: 0.739 ms clients: 98 pps: 132294 total drops: 0 avg rtt: 0.748 ms clients: 99 pps: 132859 total drops: 0 avg rtt: 0.753 ms clients: 100 pps: 131105 total drops: 0 avg rtt: 0.770 ms clients: 101 ^C Runtime: 1m41.78992826s Packets received: 8284037 Packets dropped: 0 [email protected]$ We are here beyond the point where the tester cpu's can cause false positives. With the older snapshot I can run 4 concurrent 30 client tests without packet loss, this is more than 300kpps on trunk0 on the firewall. Output from one of the four testers: [email protected]$ route -T2 exec ./phantom -k 1234 -c 0 -n 30 10.96.2.21:2222 packet size: 512 packets per client: 9223372036854775807 number of clients: 30 server address: 10.96.2.21:2222 pps: 36448 total drops: 0 avg rtt: 0.823 ms clients: 30 pps: 40931 total drops: 0 avg rtt: 0.733 ms clients: 30 pps: 41419 total drops: 0 avg rtt: 0.724 ms clients: 30 pps: 41585 total drops: 0 avg rtt: 0.721 ms clients: 30 pps: 41078 total drops: 0 avg rtt: 0.730 ms clients: 30 pps: 41637 total drops: 0 avg rtt: 0.721 ms clients: 30 pps: 41504 total drops: 0 avg rtt: 0.723 ms clients: 30 pps: 41666 total drops: 0 avg rtt: 0.720 ms clients: 30 pps: 41803 total drops: 0 avg rtt: 0.718 ms clients: 30 pps: 42074 total drops: 0 avg rtt: 0.713 ms clients: 30 pps: 40728 total drops: 0 avg rtt: 0.737 ms clients: 30 ^C Runtime: 11.877603832s Packets received: 475599 Packets dropped: 0 [email protected]$ The recent snapshot takes me from over 300kpps clean, to less than 40kpps clean. The test clients runs 6.3, as that release is a lot faster here than 6.4 or snaps. Regards Tony
