Ack! ^M sends the email. continuing . . .
I came home last night and found my system Hard Locked.
I have a serial console hooked up and used CR~^B to break
into ddb and tried a few things. The previous message
has a log of everything.
Drats! I was hoping that the Hard Locks were gone.
1.) Does anything look out of place in the trace or ps?
2.) What additional commands should I do next time this happens?
This is on a SMP kernel less than a couple of days old.
The dmesg of the system is attached.
--On Saturday, November 23, 2002 12:07 PM -0800 "Joel M. Baldwin" <[EMAIL PROTECTED]> wrote:
I came home last night and found the system Hard Locked. ^ . . . snip . . .
Copyright (c) 1992-2002 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD 5.0-CURRENT #23: Fri Nov 22 07:28:07 PST 2002
[EMAIL PROTECTED]:/disk2/usr.src/sys/i386/compile/testGeneric.smp
Preloaded elf kernel "/boot/kernel.testgeneric.smp/kernel" at 0xc04db000.
Timecounter "i8254" frequency 1193182 Hz
CPU: Pentium II/Pentium II Xeon/Celeron (467.73-MHz 686-class CPU)
Origin = "GenuineIntel" Id = 0x665 Stepping = 5
Features=0x183fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR>
real memory = 268369920 (255 MB)
avail memory = 255553536 (243 MB)
Programming 24 pins in IOAPIC #0
IOAPIC #0 intpin 2 -> irq 0
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
cpu0 (BSP): apic id: 0, version: 0x00040011, at 0xfee00000
cpu1 (AP): apic id: 1, version: 0x00040011, at 0xfee00000
io0 (APIC): apic id: 2, version: 0x00170011, at 0xfec00000
Initializing GEOMetry subsystem
Pentium Pro MTRR support enabled
npx0: <math processor> on motherboard
npx0: INT 16 interface
Using $PIR table, 8 entries at 0xc00fd7e0
pcib0: <Intel 82443BX (440 BX) host to PCI bridge> at pcibus 0 on motherboard
pci0: <PCI bus> on pcib0
IOAPIC #0 intpin 19 -> irq 2
IOAPIC #0 intpin 18 -> irq 5
IOAPIC #0 intpin 17 -> irq 9
IOAPIC #0 intpin 16 -> irq 10
agp0: <Intel 82443BX (440 BX) host to PCI bridge> mem 0xd0000000-0xd3ffffff at device
0.0 on pci0
pcib1: <PCIBIOS PCI-PCI bridge> at device 1.0 on pci0
pci1: <PCI bus> on pcib1
isab0: <PCI-ISA bridge> at device 7.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel PIIX4 ATA33 controller> port 0xf000-0xf00f at device 7.1 on pci0
ata0: at 0x1f0 irq 14 on atapci0
ata1: at 0x170 irq 15 on atapci0
uhci0: <Intel 82371AB/EB (PIIX4) USB controller> port 0xc000-0xc01f irq 2 at device
7.2 on pci0
usb0: <Intel 82371AB/EB (PIIX4) USB controller> on uhci0
usb0: USB revision 1.0
uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 2 ports with 2 removable, self powered
Timecounter "PIIX" frequency 3579545 Hz
pci0: <bridge, PCI-unknown> at device 7.3 (no driver attached)
fxp0: <Intel Pro 10/100B/100+ Ethernet> port 0xc400-0xc43f mem
0xd9000000-0xd90fffff,0xd9101000-0xd9101fff irq 5 at device 11.0 on pci0
fxp0: Ethernet address 00:90:27:a5:f4:f2
inphy0: <i82555 10/100 media interface> on miibus0
inphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
ahc0: <Adaptec 2940 Ultra2 SCSI adapter> port 0xc800-0xc8ff mem 0xd9100000-0xd9100fff
irq 9 at device 13.0 on pci0
aic7890/91: Ultra2 Wide Channel A, SCSI Id=7, 32/253 SCBs
pci0: <display, VGA> at device 15.0 (no driver attached)
rl0: <Accton MPX 5030/5038 10/100BaseTX> port 0xcc00-0xccff mem 0xd9102000-0xd91020ff
irq 2 at device 17.0 on pci0
rl0: Realtek 8139B detected. Warning, this may be unstable in autoselect mode
rl0: Ethernet address: 00:04:e2:20:a5:b2
miibus1: <MII bus> on rl0
rlphy0: <RealTek internal media interface> on miibus1
rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
atapci1: <HighPoint HPT366 ATA66 controller> port
0xd800-0xd8ff,0xd400-0xd403,0xd000-0xd007 irq 5 at device 19.0 on pci0
ata2: at 0xd000 on atapci1
atapci2: <HighPoint HPT366 ATA66 controller> port
0xe400-0xe4ff,0xe000-0xe003,0xdc00-0xdc07 irq 5 at device 19.1 on pci0
ata3: at 0xdc00 on atapci2
orm0: <Option ROMs> at iomem
0xef000-0xeffff,0xd0000-0xd17ff,0xc8000-0xc8fff,0xc0000-0xc7fff on isa0
atkbdc0: <Keyboard controller (i8042)> at port 0x64,0x60 on isa0
atkbd0: <AT Keyboard> flags 0x1 irq 1 on atkbdc0
kbd0 at atkbd0
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: model Generic PS/2 mouse, device ID 0
fdc0: <Enhanced floppy controller (i82077, NE72065 or clone)> at port
0x3f7,0x3f0-0x3f5 irq 6 drq 2 on isa0
fdc0: FIFO enabled, 8 bytes threshold
fd0: <1440-KB 3.5" drive> on fdc0 drive 0
fd1: <1200-KB 5.25" drive> on fdc0 drive 1
pmtimer0 on isa0
ppc0: <Parallel port> at port 0x378-0x37f irq 7 on isa0
ppc0: Generic chipset (EPP/NIBBLE) in COMPATIBLE mode
lpt0: <Printer> on ppbus0
lpt0: Interrupt-driven port
ppi0: <Parallel I/O> on ppbus0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x100>
sio0 at port 0x3f8-0x3ff irq 4 flags 0x30 on isa0
sio0: type 16550A, console
sio1 at port 0x2f8-0x2ff irq 3 on isa0
sio1: type 16550A
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
unknown: <PNP0303> can't assign resources (port)
unknown: <PNP0c02> can't assign resources (memory)
unknown: <PNP0a03> can't assign resources (port)
unknown: <PNP0f13> can't assign resources (irq)
unknown: <PNP0501> can't assign resources (port)
unknown: <PNP0700> can't assign resources (port)
unknown: <PNP0400> can't assign resources (port)
unknown: <PNP0501> can't assign resources (port)
Timecounters tick every 10.000 msec
APIC_IO: Testing 8254 interrupt delivery
APIC_IO: routing 8254 via IOAPIC #0 intpin 2
ipfw2 initialized, divert enabled, rule-based forwarding enabled, default to deny,
logging limited to 100 packets/entry by default
ata1-slave: timeout waiting for interrupt
ata1-slave: ATAPI identify failed
ad0: 42934MB <WDC WD450AA-00BAA0> [87233/16/63] at ata0-master UDMA33
ad1: 19470MB <IBM-DJNA-352030> [39560/16/63] at ata2-master UDMA66
ad2: 19470MB <IBM-DJNA-352030> [39560/16/63] at ata3-master UDMA66
acd0: CD-RW <HP CD-Writer+ 7200> at ata1-master PIO3
Waiting 3 seconds for SCSI devices to settle
da0 at ahc0 bus 0 target 0 lun 0
da0: <QUANTUM VIKING 4.5 SCA 8600> Fixed Direct Access SCSI-2 device
da0: 40.000MB/s transfers (20.000MHz, offset 31, 16bit), Tagged Queueing Enabled
da0: 4345MB (8899737 512 byte sectors: 255H 63S/T 553C)
da2 at ahc0 bus 0 target 5 lun 0
da2: <QUANTUM QM318200TD Q18C> Fixed Direct Access SCSI-3 device
da2: 40.000MB/s transfers (20.000MHz, offset 31, 16bit), Tagged Queueing Enabled
da2: 17366MB (35566480 512 byte sectors: 255H 63S/T 2213C)
da1 at ahc0 bus 0 target 2 lun 0
da1: <IBMRISC DFHSS4W 4242> Fixed Direct Access SCSI-2 device
da1: 20.000MB/s transfers (10.000MHz, offset 15, 16bit), Tagged Queueing Enabled
da1: 4303MB (8813870 512 byte sectors: 255H 63S/T 548C)
SMP: AP CPU #1 Launched!
Mounting root from ufs:/dev/da0s2a
WARNING: / was not properly dismounted
Entropy harvesting: interrupts ethernet point_to_point.
kernel dumps on /dev/da2s1b
vinum: loaded
vinum: reading configuration from /dev/ad2s1e
vinum: updating configuration from /dev/ad1s1e
swapon: adding /dev/da2s1b as swap device
swapon: adding /dev/da1s1b as swap device
swapon: adding /dev/da0s1b as swap device
Starting file system checks:
/dev/da0s2a: INCORRECT BLOCK COUNT I=315521 (2 should be 0) (CORRECTED)
/dev/da0s2a: INCORRECT BLOCK COUNT I=489842 (2 should be 0) (CORRECTED)
/dev/da0s2a: LINK COUNT FILE I=496830 OWNER=mix MODE=100600
/dev/da0s2a: SIZE=171 MTIME=Nov 22 20:20 2002 COUNT 2 SHOULD BE 1 (ADJUSTED)
/dev/da0s2a: UNREF FILE I=544060 OWNER=root MODE=100600
/dev/da0s2a: SIZE=129150 MTIME=Nov 22 10:00 2002 (CLEARED)
/dev/da0s2a: UNREF FILE I=544304 OWNER=root MODE=100640
/dev/da0s2a: SIZE=98836 MTIME=Nov 22 16:00 2002 (CLEARED)
/dev/da0s2a: UNREF FILE I=547784 OWNER=root MODE=100600
/dev/da0s2a: SIZE=28125 MTIME=Nov 22 20:20 2002 (CLEARED)
/dev/da0s2a: UNREF FILE I=548497 OWNER=root MODE=100600
/dev/da0s2a: SIZE=786 MTIME=Nov 22 20:20 2002 (CLEARED)
/dev/da0s2a: UNREF FILE I=548517 OWNER=root MODE=100600
/dev/da0s2a: SIZE=742 MTIME=Nov 22 20:20 2002 (CLEARED)
/dev/da0s2a: LINK COUNT FILE I=548621 OWNER=root MODE=100600
/dev/da0s2a: SIZE=754 MTIME=Nov 22 20:20 2002 COUNT 2 SHOULD BE 1 (ADJUSTED)
/dev/da0s2a: UNREF FILE I=843172 OWNER=nobody MODE=100600
/dev/da0s2a: SIZE=586034 MTIME=Nov 22 20:17 2002 (CLEARED)
/dev/da0s2a: UNREF FILE I=851868 OWNER=nobody MODE=100600
/dev/da0s2a: SIZE=102833 MTIME=Nov 22 20:17 2002 (CLEARED)
/dev/da0s2a: FREE BLK COUNT(S) WRONG IN SUPERBLK (SALVAGED)
/dev/da0s2a: SUMMARY INFORMATION BAD (SALVAGED)
/dev/da0s2a: BLK(S) MISSING IN BIT MAPS (SALVAGED)
/dev/da0s2a: 101479 files, 3398420 used, 813672 free (94968 frags, 89838 blocks, 2.3%
fragmentation)
/dev/da1s2e: DEFER FOR BACKGROUND CHECKING
/dev/ad0s1e: DEFER FOR BACKGROUND CHECKING
/dev/da2s2e: DEFER FOR BACKGROUND CHECKING
/dev/vinum/sset: ZERO LENGTH DIR I=4333568 OWNER=root MODE=40755
/dev/vinum/sset: SIZE=0 MTIME=Nov 22 10:09 2002 (CLEARED)
/dev/vinum/sset: FREE BLK COUNT(S) WRONG IN SUPERBLK (SALVAGED)
/dev/vinum/sset: SUMMARY INFORMATION BAD (SALVAGED)
/dev/vinum/sset: BLK(S) MISSING IN BIT MAPS (SALVAGED)
/dev/vinum/sset: 213106 files, 10871860 used, 8748898 free (2754 frags, 1093268
blocks, 0.0% fragmentation)
WARNING: /disk2 was not properly dismounted
WARNING: /disk2/ftp was not properly dismounted
WARNING: /disk3 was not properly dismounted
Doing initial network setup: hostname.
fxp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
inet 207.173.133.28 netmask 0xfffffff8 broadcast 207.173.133.31
inet6 fe80::290:27ff:fea5:f4f2%fxp0 prefixlen 64 tentative scopeid 0x1
ether 00:90:27:a5:f4:f2
media: Ethernet autoselect (100baseTX <full-duplex>)
status: active
rl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
inet 168.150.177.152 netmask 0xffffff00 broadcast 168.150.177.255
inet6 fe80::204:e2ff:fe20:a5b2%rl0 prefixlen 64 tentative scopeid 0x2
ether 00:04:e2:20:a5:b2
media: Ethernet autoselect (none)
status: no carrier
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
inet 127.0.0.1 netmask 0xff000000
Flushed all rules.
00100 allow ip from any to any via lo0
00200 deny ip from any to 127.0.0.0/8
00300 deny ip from 127.0.0.0/8 to any
00400 deny ip from 192.168.1.0/24 to any in via rl0
00500 deny ip from 168.150.177.152 to any in via fxp0
00600 deny ip from any to 10.0.0.0/8 via rl0
00700 deny ip from any to 172.16.0.0/12 via rl0
00800 deny ip from any to 192.168.0.0/16 via rl0
00900 deny ip from any to 0.0.0.0/8 via rl0
01000 deny ip from any to 169.254.0.0/16 via rl0
01100 deny ip from any to 192.0.2.0/24 via rl0
01200 deny ip from any to 224.0.0.0/4 via rl0
01300 deny ip from any to 240.0.0.0/4 via rl0
01400 allow icmp from any to 168.150.177.1
01500 allow icmp from 168.150.177.1 to any
01600 deny ip from any to any dst-port 3128 via rl0
01700 deny ip from any to any dst-port 3128 via rl0
01800 deny ip from any to any dst-port 135-139 via rl0
01900 allow ip from any to any dst-port 135-139 via fxp0
02000 allow tcp from any to 168.150.177.152 dst-port 2211 setup
02100 allow tcp from any to 168.150.177.152 dst-port 21312-21322 setup
02200 allow tcp from 168.150.177.152 21312-21322 to any setup
02300 allow udp from any to 168.150.177.152 dst-port 2086 via rl0
02400 allow udp from 168.150.177.152 to any dst-port 2086 via rl0
02500 allow tcp from any to 168.150.177.152 dst-port 113 setup
02600 divert 8668 ip from any to any via rl0
02700 deny ip from 10.0.0.0/8 to any via rl0
02800 deny ip from 172.16.0.0/12 to any via rl0
02900 deny ip from 192.168.0.0/16 to any via rl0
03000 deny ip from 0.0.0.0/8 to any via rl0
03100 deny ip from 169.254.0.0/16 to any via rl0
03200 deny ip from 192.0.2.0/24 to any via rl0
03300 deny ip from 224.0.0.0/4 to any via rl0
03400 deny ip from 240.0.0.0/4 to any via rl0
03500 allow udp from any to any via fxp0
03600 allow tcp from any to any established
03700 allow ip from any to any frag
03800 allow tcp from any to any dst-port 54408 setup
03900 allow tcp from any 54408 to any setup
04000 allow tcp from 217.96.41.97 to 168.150.177.152 dst-port 21 setup
04100 allow tcp from any to 168.150.177.152 dst-port 25 setup
04200 allow tcp from any to 168.150.177.152 dst-port 53 setup
04300 allow udp from any to 168.150.177.152 dst-port 53
04400 allow udp from 168.150.177.152 53 to any
04500 allow udp from any 53 to 168.150.177.152
04600 allow tcp from any to 168.150.177.152 dst-port 80 setup
04700 allow tcp from any to 168.150.177.152 dst-port 443 setup
04800 allow tcp from any to 168.150.177.152 dst-port 23 setup
04900 allow tcp from any to 168.150.177.152 dst-port 22 setup
05000 deny log tcp from any to any in via rl0 setup
05100 allow tcp from any to any setup
05200 allow udp from 168.150.177.152 to any dst-port 53 keep-state
05300 allow udp from 192.168.0.0/16 to any dst-port 53 keep-state
05400 allow udp from 168.150.177.152 to any dst-port 123 keep-state
05500 allow udp from 192.168.0.0/16 to any dst-port 123 keep-state
05600 allow udp from any to 192.168.0.0/16 dst-port 123 keep-state
05700 allow udp from any to 217.96.41.97 keep-state
05800 allow udp from 217.96.41.97 to any keep-state
05900 deny log ip from any to any
Firewall rules loaded, starting divert daemons: natdNov 22 22:04:53 natd[265]:
Aliasing to 168.150.177.152, mtu 1500 bytes.
Firewall logging=YES
add net default: gateway 168.150.177.1
Additional routing options: log ICMP redirect=YES IP gateway=YES drop SYN+FIN
packets=YESsysctl: unknown oid 'net.inet.tcp.drop_synfin'
.
Mounting NFS file systems:.
Starting syslogd.
Checking for core dump...
checking for kernel dump on device /dev/da2s1b
mediasize = 271401984
sectorsize = 512
savecore: reboot after panic: from debugger
Checking for available free space
bounds number: 31
Good dump found on device /dev/da2s1b
Architecture: i386
Architecture version: 1
Dump length: 268369920B (255 MB)
Blocksize: 512
Dumptime: Fri Nov 22 21:49:02 2002
Hostname: outel.org
Versionstring: FreeBSD 5.0-CURRENT #23: Fri Nov 22 07:28:07 PST 2002
[EMAIL PROTECTED]:/disk2/usr.src/sys/i386/compile/testGeneric.smp
Panicstring: from debugger
Bounds: 31
savecore: writing core to vmcore.31
1048576
2097152
3145728
4194304
5242880
6291456
7340032
8388608
9437184
10485760
11534336
12582912
13631488
14680064
15728640
16777216
17825792
18874368
19922944
20971520
22020096
23068672
24117248
25165824
26214400
27262976
28311552
29360128
30408704
31457280
32505856
33554432
34603008
35651584
36700160
37748736
38797312
39845888
40894464
41943040
42991616
44040192
45088768
46137344
47185920
48234496
49283072
50331648
51380224
52428800
53477376
54525952
55574528
56623104
57671680
58720256
59768832
60817408
61865984
62914560
63963136
65011712
66060288
67108864
68157440
69206016
70254592
71303168
72351744
73400320
74448896
75497472
76546048
77594624
78643200
79691776
80740352
81788928
82837504
83886080
84934656
85983232
87031808
88080384
89128960
90177536
91226112
92274688
93323264
94371840
95420416
96468992
97517568
98566144
99614720
100663296
101711872
102760448
103809024
104857600
105906176
106954752
108003328
109051904
110100480
111149056
112197632
113246208
114294784
115343360
116391936
117440512
118489088
119537664
120586240
121634816
122683392
123731968
124780544
125829120
126877696
127926272
128974848
130023424
131072000
132120576
133169152
134217728
135266304
136314880
137363456
138412032
139460608
140509184
141557760
142606336
143654912
144703488
145752064
146800640
147849216
148897792
149946368
150994944
152043520
153092096
154140672
155189248
156237824
157286400
158334976
159383552
160432128
161480704
162529280
163577856
164626432
165675008
166723584
167772160
168820736
169869312
170917888
171966464
173015040
174063616
175112192
176160768
177209344
178257920
179306496
180355072
181403648
182452224
183500800
184549376
185597952
186646528
187695104
188743680
189792256
190840832
191889408
192937984
193986560
195035136
196083712
197132288
198180864
199229440
200278016
201326592
202375168
203423744
204472320
205520896
206569472
207618048
208666624
209715200
210763776
211812352
212860928
213909504
214958080
216006656
217055232
218103808
219152384
220200960
221249536
222298112
223346688
224395264
225443840
226492416
227540992
228589568
229638144
230686720
231735296
232783872
233832448
234881024
235929600
236978176
238026752
239075328
240123904
241172480
242221056
243269632
244318208
245366784
246415360
247463936
248512512
249561088
250609664
251658240
252706816
253755392
254803968
255852544
256901120
257949696
258998272
260046848
261095424
262144000
263192576
264241152
265289728
266338304
267386880
268369920dump saved
clearing dump header
checking for kernel dump on device /dev/da1s1b
mediasize = 98671104
sectorsize = 512
magic mismatch on last dump header on /dev/da1s1b
checking for kernel dump on device /dev/da0s1b
mediasize = 98671104
sectorsize = 512
magic mismatch on last dump header on /dev/da0s1b
Starting named.
Starting rpcbind.
ELF ldconfig path: /usr/lib /usr/lib/compat /usr/X11R6/lib /usr/local/lib
a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout /usr/X11R6/lib/aout
/etc/rc: INFO: nfsd depends on mountd, which will be forced to start.
Starting mountd.
NFS on reserved port only=YES
Starting nfsd.
Starting local daemons:.
Updating motd.
Starting usbd.
Configuring syscons: blanktime.
Starting sshd.
Starting sendmail.
Initial i386 initialization:.
Additional ABI support:.
Local package initialization:apache
gnunetd
mysql client
mysqld
netstuff