On 16.10.2015. 20:30, Mark Kettenis wrote:
>> Date: Fri, 16 Oct 2015 14:13:52 +0200
>> From: Martin Pieuchot <m...@openbsd.org>
>>
>> On 08/10/15(Thu) 20:49, Mark Kettenis wrote:
>>>> Date: Wed, 30 Sep 2015 14:30:11 +0200 (CEST)
>>>> From: Mark Kettenis <mark.kette...@xs4all.nl>
>>>>
>>>> Since people seemed to like my diff for em(4), here is one for ix(4).
>>>> In addition to unlocking the rx completion path, this one also uses
>>>> intr_barrier() and removes the rx mutex that claudio@ put in recently.
>>>>
>>>> I don't have any ix(4) hardware. So the only guarantee is that this
>>>> compiles ;).
>>>
>>> Based on the experience with em(4), here is an updated diff.
>>
>> What's happening to this? Mark did you get any test report?
>
> Got a report from Hrvoje Popovski that this triggers OACTIVE a lot and
> sometimes even gets stuck completely.
Hi all,
i'm confirming what Mark said. I must admit that this traffic is
synthetic and maybe too aggressive for normal scenario.
With if_ix.c rev 1.125 i can't trigger OACTIVE flag, box is stable but i
can trigger
ix1: unable to fill any rx descriptors
ix1: Could not setup receive structures
I'm sorry that i haven't seen this with revision 1.125.
I will try to describe what I'm seeing with Marks patch.
traffic is routed between ix interfaces
sender is connected to ix1 (82599)
receiver is connected to ix0 (82599)
pf=NO
ddb.console=1
kern.pool_debug=1
net.inet.ip.forwarding=1
net.inet.ip.ifq.maxlen=1024
box is freshly rebooted and i'm generating cca 10Mpps
box is unstable and on receiver pps vary from 180kpps to 1.1Mpps (i
never seen 2Mpps on receiver)
# netstat -i 1
em0 in em0 out total in total out
packets errs packets errs colls packets errs packets errs colls
833 0 649 0 0 259125144 0 146427665 0 0
2 0 2 0 0 1035261 0 341645 0 0
1 0 1 0 0 979946 0 177223 0 0
1 0 2 0 0 1451247 0 1287839 0 0
2 0 1 0 0 1095354 0 847450 0 0
3 0 1 0 0 985081 0 180134 0 0
1 0 1 0 0 1034061 0 370778 0 0
2 0 2 0 0 2182566 0 1420698 0 0
1 0 1 0 0 1137561 0 1115412 0 0
2 0 1 0 0 1012692 0 257518 0 0
3 0 1 0 0 965894 0 180484 0 0
3 0 2 0 0 988832 0 179729 0 0
2 0 1 0 0 959809 0 178349 0 0
2 0 1 0 0 1003070 0 179792 0 0
1 0 1 0 0 989400 0 179187 0 0
1 0 1 0 0 1007894 0 182429 0 0
1 0 1 0 0 2211264 0 1455209 0 0
1 0 1 0 0 2161230 0 2122498 0 0
2 0 1 0 0 1136320 0 1078249 0 0
1 0 1 0 0 1046122 0 486958 0 0
2 0 2 0 0 994789 0 177598 0 0
doing ifconfig ix0 down while generating traffic makes box really
unstable and sometimes breaks ssh session.
after ifconfig ix0 down/up while generating traffic i'm getting stable
1Mpps, box is responsive and everything is fine ...
# netstat -i 1
em0 in em0 out total in total out
packets errs packets errs colls packets errs packets errs colls
2348 0 1714 0 0 1102233921 0 589333306 0 0
2 0 3 0 0 1145492 0 1135339 0 0
1 0 1 0 0 1526533 0 1492768 0 0
3 0 3 0 0 1109545 0 1056238 0 0
2 0 2 0 0 1255325 0 1250910 0 0
2 0 2 0 0 1557626 0 1527923 0 0
2 0 2 0 0 1136320 0 1113975 0 0
2 0 2 0 0 1476924 0 1442136 0 0
2 0 2 0 0 2248410 0 2203062 0 0
2 0 2 0 0 2189965 0 2145617 0 0
2 0 2 0 0 1786688 0 1740730 0 0
2 0 2 0 0 2090607 0 2045849 0 0
2 0 2 0 0 2151949 0 2110065 0 0
3 0 4 0 0 1613589 0 1576791 0 0
2 0 2 0 0 1958826 0 1915090 0 0
2 0 2 0 0 1843465 0 1800886 0 0
2 0 2 0 0 1107742 0 1054389 0 0
3 0 2 0 0 1227703 0 1225070 0 0
2 0 2 0 0 1956501 0 1911418 0 0
3 0 2 0 0 2144780 0 2099272 0 0
3 0 2 0 0 2005521 0 1956075 0 0
doing few more times ifconfig ix0 down/up i'm always getting 1Mpps and
box is fine.
doing ifconfig ix0 down doesn't makes box so unstable ...
then reboot again
box is unstable and pps on receiver vary
# netstat -i 1
em0 in em0 out total in total out
packets errs packets errs colls packets errs packets errs colls
227 0 168 0 0 16468894 0 9656566 0 0
1 0 2 0 0 1519605 0 1376563 0 0
1 0 1 0 0 1479038 0 1454045 0 0
1 0 1 0 0 1144993 0 1124738 0 0
1 0 1 0 0 1040484 0 616282 0 0
2 0 2 0 0 942668 0 184136 0 0
1 0 1 0 0 917126 0 183123 0 0
1 0 1 0 0 1789600 0 1425149 0 0
1 0 1 0 0 1862498 0 1770765 0 0
1 0 1 0 0 1258926 0 1236198 0 0
1 0 1 0 0 1113054 0 1062883 0 0
1 0 1 0 0 1234969 0 1231888 0 0
1 0 1 0 0 1047980 0 697784 0 0
1 0 1 0 0 945403 0 212353 0 0
1 0 1 0 0 931217 0 185713 0 0
1 0 1 0 0 1903592 0 1417938 0 0
1 0 1 0 0 1069005 0 842399 0 0
1 0 1 0 0 1249609 0 1244624 0 0
1 0 1 0 0 1092014 0 1017988 0 0
3 0 1 0 0 1281444 0 1275296 0 0
3 0 1 0 0 1379775 0 1358934 0 0
doing ifconfig ix0 down/up few times i can't get stable 1Mpps but i get
ix0: unable to fill any rx descriptors
ix0: Could not setup receive structures
:)
after that box won't forward traffic until sender is stopped and
ifconfig ix0 down/up is done.
and after that i'm getting stable 1Mpps :)
# netstat -i 1
em0 in em0 out total in total out
packets errs packets errs colls packets errs packets errs colls
3301 0 1424 0 0 3568200369 0 3361985794 0 0
1 0 2 0 0 1148724 0 1131996 0 0
1 0 1 0 0 1146342 0 1124934 0 0
1 0 1 0 0 1790239 0 1743420 0 0
1 0 1 0 0 2266762 0 2204738 0 0
1 0 1 0 0 2150845 0 2092527 0 0
3 0 1 0 0 2244556 0 2187319 0 0
1 0 1 0 0 1107617 0 1047243 0 0
1 0 1 0 0 1191683 0 1190898 0 0
1 0 1 0 0 1128128 0 1103238 0 0
1 0 1 0 0 1957784 0 1905944 0 0
1 0 1 0 0 1108918 0 1048791 0 0
2 0 1 0 0 1198041 0 1196940 0 0
1 0 1 0 0 2007404 0 1955561 0 0
1 0 1 0 0 1923409 0 1874560 0 0
1 0 1 0 0 1109004 0 1050817 0 0
2 0 2 0 0 1203946 0 1201867 0 0
1 0 1 0 0 1332793 0 1304595 0 0
1 0 1 0 0 2146685 0 2092369 0 0
1 0 1 0 0 1787378 0 1742300 0 0
1 0 1 0 0 1110338 0 1055909 0 0
i will do all this a hundred times but i'm sure that i will not be able
to reproduce it exactly like now :)
OpenBSD 5.8-current (GENERIC.MP) #4: Tue Oct 20 23:19:22 CEST 2015
r...@ttttt.srce.hr:/usr/src/sys/arch/amd64/compile/GENERIC.MP
RTC BIOS diagnostic error 80<clock_battery>
real mem = 34314592256 (32724MB)
avail mem = 33270497280 (31729MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0x7e67c000 (84 entries)
bios0: vendor IBM version "-[D7E146CUS-1.82]-" date 04/09/2015
bios0: IBM IBM System x3550 M4 Server -[7914T91]-
acpi0 at bios0: rev 2
acpi0: sleep states S0 S5
acpi0: tables DSDT FACP TCPA ERST HEST HPET APIC MCFG OEM0 OEM1 SLIT
SRAT SLIC SSDT SSDT SSDT SSDT DMAR
acpi0: wakeup devices MRP1(S4) DCC0(S4) MRP3(S4) MRP5(S4) EHC2(S5)
PEX0(S5) PEX7(S5) EHC1(S5) IP2P(S3) MRPB(S4) MRPC(S4) MRPD(S4) MRPF(S4)
MRPG(S4) MRPH(S4) MRPI(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpihpet0 at acpi0: 14318179 Hz
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2400.35 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 100MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.1, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2400.00 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2400.00 MHz
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 6 (application processor)
cpu3: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2400.01 MHz
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 0, core 3, package 0
cpu4 at mainbus0: apid 8 (application processor)
cpu4: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2400.00 MHz
cpu4:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT
cpu4: 256KB 64b/line 8-way L2 cache
cpu4: smt 0, core 4, package 0
cpu5 at mainbus0: apid 10 (application processor)
cpu5: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2400.00 MHz
cpu5:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT
cpu5: 256KB 64b/line 8-way L2 cache
cpu5: smt 0, core 5, package 0
cpu6 at mainbus0: apid 32 (application processor)
cpu6: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2400.00 MHz
cpu6:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT
cpu6: 256KB 64b/line 8-way L2 cache
cpu6: smt 0, core 0, package 1
cpu7 at mainbus0: apid 34 (application processor)
cpu7: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2400.00 MHz
cpu7:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT
cpu7: 256KB 64b/line 8-way L2 cache
cpu7: smt 0, core 1, package 1
cpu8 at mainbus0: apid 36 (application processor)
cpu8: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2400.00 MHz
cpu8:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT
cpu8: 256KB 64b/line 8-way L2 cache
cpu8: smt 0, core 2, package 1
cpu9 at mainbus0: apid 38 (application processor)
cpu9: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2400.00 MHz
cpu9:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT
cpu9: 256KB 64b/line 8-way L2 cache
cpu9: smt 0, core 3, package 1
cpu10 at mainbus0: apid 40 (application processor)
cpu10: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2400.01 MHz
cpu10:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT
cpu10: 256KB 64b/line 8-way L2 cache
cpu10: smt 0, core 4, package 1
cpu11 at mainbus0: apid 42 (application processor)
cpu11: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2400.00 MHz
cpu11:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT
cpu11: 256KB 64b/line 8-way L2 cache
cpu11: smt 0, core 5, package 1
ioapic0 at mainbus0: apid 8 pa 0xfec00000, version 20, 24 pins
ioapic1 at mainbus0: apid 9 pa 0xfec01000, version 20, 24 pins
ioapic2 at mainbus0: apid 10 pa 0xfec40000, version 20, 24 pins
acpimcfg0 at acpi0 addr 0x80000000, bus 0-255
acpiprt0 at acpi0: bus 0 (IOH0)
acpiprt1 at acpi0: bus 12 (MRP1)
acpiprt2 at acpi0: bus 17 (DCC0)
acpiprt3 at acpi0: bus 22 (MRP3)
acpiprt4 at acpi0: bus 27 (MRP5)
acpiprt5 at acpi0: bus 6 (PEX0)
acpiprt6 at acpi0: bus 128 (IOH1)
acpiprt7 at acpi0: bus 134 (MRPD)
acpicpu0 at acpi0: C2(500@59 mwait.3@0x10), C1(1000@3 mwait.1)
acpicpu1 at acpi0: C2(500@59 mwait.3@0x10), C1(1000@3 mwait.1)
acpicpu2 at acpi0: C2(500@59 mwait.3@0x10), C1(1000@3 mwait.1)
acpicpu3 at acpi0: C2(500@59 mwait.3@0x10), C1(1000@3 mwait.1)
acpicpu4 at acpi0: C2(500@59 mwait.3@0x10), C1(1000@3 mwait.1)
acpicpu5 at acpi0: C2(500@59 mwait.3@0x10), C1(1000@3 mwait.1)
acpicpu6 at acpi0: C2(500@59 mwait.3@0x10), C1(1000@3 mwait.1)
acpicpu7 at acpi0: C2(500@59 mwait.3@0x10), C1(1000@3 mwait.1)
acpicpu8 at acpi0: C2(500@59 mwait.3@0x10), C1(1000@3 mwait.1)
acpicpu9 at acpi0: C2(500@59 mwait.3@0x10), C1(1000@3 mwait.1)
acpicpu10 at acpi0: C2(500@59 mwait.3@0x10), C1(1000@3 mwait.1)
acpicpu11 at acpi0: C2(500@59 mwait.3@0x10), C1(1000@3 mwait.1)
ipmi at mainbus0 not configured
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel E5 v2 Host" rev 0x04
ppb0 at pci0 dev 1 function 0 "Intel E5 v2 PCIE" rev 0x04: msi
pci1 at ppb0 bus 12
ppb1 at pci0 dev 2 function 0 "Intel E5 v2 PCIE" rev 0x04: msi
pci2 at ppb1 bus 17
ppb2 at pci0 dev 3 function 0 "Intel E5 v2 PCIE" rev 0x04: msi
pci3 at ppb2 bus 22
22:0:0: mem address conflict 0xfff80000/0x80000
22:0:1: mem address conflict 0xfff80000/0x80000
ix0 at pci3 dev 0 function 0 "Intel 82599" rev 0x01: msi, address
a0:36:9f:2e:96:a0
ix1 at pci3 dev 0 function 1 "Intel 82599" rev 0x01: msi, address
a0:36:9f:2e:96:a1
ppb3 at pci0 dev 3 function 2 "Intel E5 v2 PCIE" rev 0x04: msi
pci4 at ppb3 bus 27
mfii0 at pci4 dev 0 function 0 "Symbios Logic MegaRAID SAS2208" rev
0x05: msi
mfii0: "ServeRAID M5110", firmware 23.33.0-0018, 512MB cache
scsibus1 at mfii0: 64 targets
sd0 at scsibus1 targ 0 lun 0: <IBM, ServeRAID M5110, 3.45> SCSI3
0/direct fixed naa.600605b006c3a0b01a582bd010e4c053
sd0: 952720MB, 512 bytes/sector, 1951170560 sectors
scsibus2 at mfii0: 256 targets
"Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 0 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 1 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 2 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 3 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 4 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 5 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 6 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 7 not configured
"Intel E5 v2 Address Map" rev 0x04 at pci0 dev 5 function 0 not configured
"Intel E5 v2 IIO RAS" rev 0x04 at pci0 dev 5 function 2 not configured
ppb4 at pci0 dev 17 function 0 "Intel C600 Virtual PCIE" rev 0x06
pci5 at ppb4 bus 32
ehci0 at pci0 dev 26 function 0 "Intel C600 USB" rev 0x06: apic 8 int 19
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 "Intel EHCI root hub" rev 2.00/1.00 addr 1
ppb5 at pci0 dev 28 function 0 "Intel C600 PCIE" rev 0xb6: msi
pci6 at ppb5 bus 6
em0 at pci6 dev 0 function 0 "Intel I350" rev 0x01: msi, address
40:f2:e9:10:56:1c
em1 at pci6 dev 0 function 1 "Intel I350" rev 0x01: msi, address
40:f2:e9:10:56:1d
em2 at pci6 dev 0 function 2 "Intel I350" rev 0x01: msi, address
40:f2:e9:10:56:1e
em3 at pci6 dev 0 function 3 "Intel I350" rev 0x01: msi, address
40:f2:e9:10:56:1f
ppb6 at pci0 dev 28 function 7 "Intel C600 PCIE" rev 0xb6: msi
pci7 at ppb6 bus 1
ppb7 at pci7 dev 0 function 0 "Renesas SH7757 PCIE Switch" rev 0x00
pci8 at ppb7 bus 2
ppb8 at pci8 dev 0 function 0 "Renesas SH7757 PCIE Switch" rev 0x00
pci9 at ppb8 bus 3
ppb9 at pci9 dev 0 function 0 "Renesas SH7757 PCIE-PCI" rev 0x00
pci10 at ppb9 bus 4
vga1 at pci10 dev 0 function 0 "Matrox MGA G200eR" rev 0x00
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
ppb10 at pci8 dev 1 function 0 "Renesas SH7757 PCIE Switch" rev 0x00
pci11 at ppb10 bus 5
ehci1 at pci0 dev 29 function 0 "Intel C600 USB" rev 0x06: apic 8 int 23
usb1 at ehci1: USB revision 2.0
uhub1 at usb1 "Intel EHCI root hub" rev 2.00/1.00 addr 1
ppb11 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0xa6
pci12 at ppb11 bus 33
pcib0 at pci0 dev 31 function 0 "Intel C600 LPC" rev 0x06
ichiic0 at pci0 dev 31 function 3 "Intel C600 SMBus" rev 0x06: apic 8 int 11
iic0 at ichiic0
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5 irq 1 irq 12
pckbd0 at pckbc0 (kbd slot)
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
pci13 at mainbus0 bus 128
ppb12 at pci13 dev 0 function 0 "Intel E5 v2 PCIE" rev 0x04
pci14 at ppb12 bus 129
ppb13 at pci13 dev 2 function 0 "Intel E5 v2 PCIE" rev 0x04: msi
pci15 at ppb13 bus 134
"Intel E5 v2 I/OAT" rev 0x04 at pci13 dev 4 function 0 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci13 dev 4 function 1 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci13 dev 4 function 2 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci13 dev 4 function 3 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci13 dev 4 function 4 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci13 dev 4 function 5 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci13 dev 4 function 6 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci13 dev 4 function 7 not configured
"Intel E5 v2 Address Map" rev 0x04 at pci13 dev 5 function 0 not configured
"Intel E5 v2 IIO RAS" rev 0x04 at pci13 dev 5 function 2 not configured
uhub2 at uhub0 port 1 "Intel Rate Matching Hub" rev 2.00/0.00 addr 2
uhub3 at uhub1 port 1 "Intel Rate Matching Hub" rev 2.00/0.00 addr 2
uhub4 at uhub3 port 1 "no manufacturer Gadget USB HUB" rev 2.00/0.00 addr 3
uhub4: device problem, disabling port 1
urndis0 at uhub4 port 5 configuration 2 interface 0 "IBM RNDIS/Ethernet
Gadget" rev 2.00/3.25 addr 4
urndis0: using Vendor, address 42:f2:e9:10:56:1b
vscsi0 at root
scsibus3 at vscsi0: 256 targets
softraid0 at root
scsibus4 at softraid0: 256 targets
root on sd0a (b9e81a0d53d55c77.a) swap on sd0b dump on sd0b
uhidev0 at uhub4 port 1 configuration 1 interface 0 "Avocent
Keyboard/Mouse Function" rev 2.00/0.00 addr 5
uhidev0: iclass 3/1
ukbd0 at uhidev0: 8 variable keys, 6 key codes
wskbd1 at ukbd0 mux 1
wskbd1: connecting to wsdisplay0
uhidev1 at uhub4 port 1 configuration 1 interface 1 "Avocent
Keyboard/Mouse Function" rev 2.00/0.00 addr 5
uhidev1: iclass 3/1
ums0 at uhidev1: 3 buttons, Z dir
wsmouse0 at ums0 mux 0
uhidev2 at uhub4 port 1 configuration 1 interface 2 "Avocent
Keyboard/Mouse Function" rev 2.00/0.00 addr 5
uhidev2: iclass 3/1
ums1 at uhidev2: 3 buttons, Z dir
wsmouse1 at ums1 mux 0
ix0: unable to fill any rx descriptors
ix0: Could not setup receive structures
ix0: unable to fill any rx descriptors
ix0: Could not setup receive structures
