On 10/12/12 07:34 PM, Bentley, Dain wrote:
Hello Udo, thanks for the reply. Here is the text from fmdump -eV.
> Is there anything I should be looking for?
So BOTH disks spit out ZFS checksum errors like a machine gun,
this can either be a controller/cable problem or a memory problem
(you don't have ECC memory with an ECC supporting processor, e.g.
Xeon ? Otherwise those errors would be reported by fmdump -e).
Or some problem in the OS, those checksum problems haunt me on
my home workstation (also no ECC) when scrubbing the rpool mirror,
although disks and cables are ok and no errors occur when not
scrubbing. But the sheer amount of errors does not look like
that symptom.
'zpool status -v' should show some degradation of the
mirror with the exact checksum count, and also if files or
metadata are affected, 'fmadm faulty' gives components
retired due to errors. If you have data corruption, this
could cause reboots on some occasion.
Hard to say how to hunt this down, other than
checking cables, memory seating, looking for newer HBA or
disk/BIOS firmware, torturing memory with advanced memory testers.
The fact that the machine does reboot let me doubt that this
is a pure ZFS/disk problem, I never saw that, the machine
eventually stalls when those errors hammer the machine
too hard, but it would not cause a reboot. Memory is always
the best bet, but it could also be a motherboard problem.
Maybe the vendor has some hardware checking tool on the
CD in the box or on his website ?
TIME CLASS
Oct 11 2012 06:06:59.370787527 ereport.fs.zfs.data
nvlist version: 0
class = ereport.fs.zfs.data
ena = 0x9ebd5c95f9f00401
detector = (embedded nvlist)
nvlist version: 0
version = 0x0
scheme = zfs
pool = 0x6a8f5a381b7e2f2c
(end detector)
pool = volume0
pool_guid = 0x6a8f5a381b7e2f2c
pool_context = 0
pool_failmode = wait
zio_err = 50
zio_objset = 0x28
zio_object = 0x1
zio_level = 0
zio_blkid = 0x6047da
__ttl = 0x1
__tod = 0x50769a43 0x1619c4c7
Oct 11 2012 06:06:59.370787942 ereport.fs.zfs.checksum
nvlist version: 0
class = ereport.fs.zfs.checksum
ena = 0x9ebd5c95f9f00401
detector = (embedded nvlist)
nvlist version: 0
version = 0x0
scheme = zfs
pool = 0x6a8f5a381b7e2f2c
vdev = 0xdd2eef656bfc1db5
(end detector)
pool = volume0
pool_guid = 0x6a8f5a381b7e2f2c
pool_context = 0
pool_failmode = wait
vdev_guid = 0xdd2eef656bfc1db5
vdev_type = disk
vdev_path = /dev/dsk/c3t0d0s0
vdev_devid = id1,sd@SATA_____HDS725050KLA360_______KRVN03ZAG1JY4D/a
parent_guid = 0xfd55a23a93069d20
parent_type = mirror
zio_err = 50
zio_offset = 0x14893b600
zio_size = 0x4600
zio_objset = 0x28
zio_object = 0x1
zio_level = 0
zio_blkid = 0x6047da
cksum_expected = 0x521c0ebafd7 0x2be41b8abfd1b3 0xfccf5694dcdbaa49
0x3e24575fd2f4aa4d
cksum_actual = 0x521c2ebafd7 0x2be436a2bfd1b3 0xfcd00e26f8dbaa49
0x4161c187aaf4aa4d
cksum_algorithm = fletcher4
__ttl = 0x1
__tod = 0x50769a43 0x1619c666
Oct 11 2012 06:06:59.370788126 ereport.fs.zfs.checksum
nvlist version: 0
class = ereport.fs.zfs.checksum
ena = 0x9ebd5c95f9f00401
detector = (embedded nvlist)
nvlist version: 0
version = 0x0
scheme = zfs
pool = 0x6a8f5a381b7e2f2c
vdev = 0x377ede8d0fb06f7e
(end detector)
pool = volume0
pool_guid = 0x6a8f5a381b7e2f2c
pool_context = 0
pool_failmode = wait
vdev_guid = 0x377ede8d0fb06f7e
vdev_type = disk
vdev_path = /dev/dsk/c3t3d0s0
vdev_devid = id1,sd@SATA_____HDS725050KLA360_______KRVN03ZAG39LKD/a
parent_guid = 0xfd55a23a93069d20
parent_type = mirror
zio_err = 50
zio_offset = 0x14893b600
zio_size = 0x4600
zio_objset = 0x28
zio_object = 0x1
zio_level = 0
zio_blkid = 0x6047da
cksum_expected = 0x521c0ebafd7 0x2be41b8abfd1b3 0xfccf5694dcdbaa49
0x3e24575fd2f4aa4d
cksum_actual = 0x521c2ebafd7 0x2be436a2bfd1b3 0xfcd00e26f8dbaa49
0x4161c187aaf4aa4d
cksum_algorithm = fletcher4
__ttl = 0x1
__tod = 0x50769a43 0x1619c71e
... (1000 more of them in rapid succession)...
>
Oct 12 2012 07:54:53.123217977 ereport.fs.zfs.checksum
nvlist version: 0
class = ereport.fs.zfs.checksum
ena = 0xe644965bba100401
detector = (embedded nvlist)
nvlist version: 0
version = 0x0
scheme = zfs
pool = 0xda1ed4eddd886ca2
vdev = 0x76d6d5ecc9007061
(end detector)
pool = volume0
pool_guid = 0xda1ed4eddd886ca2
pool_context = 0
pool_failmode = wait
vdev_guid = 0x76d6d5ecc9007061
vdev_type = disk
vdev_path = /dev/dsk/c3t0d0s0
vdev_devid = id1,sd@SATA_____HDS725050KLA360_______KRVN03ZAG1JY4D/a
parent_guid = 0xb85bb36665652fa6
parent_type = mirror
zio_err = 50
zio_offset = 0x6414e7e00
zio_size = 0xa400
zio_objset = 0x28
zio_object = 0x1
zio_level = 0
zio_blkid = 0x1c3121
cksum_expected = 0xd682bb3691e 0x110e9a533de6720 0x6bb562c60c3d27a8
0xd15dd859803c65fd
cksum_actual = 0xd682db3691e 0x110e9df4bde6720 0x6bb8ae9ba83d27a8
0xf14a4b22583c65fd
cksum_algorithm = fletcher4
bad_ranges = 0x2fd0 0x2fd8
bad_ranges_min_gap = 0x8
bad_range_sets = 0x1
bad_range_clears = 0x0
bad_set_bits = 0x0 0x0 0x0 0x2 0x0 0x0 0x0 0x0
bad_cleared_bits = 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
__ttl = 0x1
__tod = 0x5078050d 0x7582839
Oct 12 2012 09:31:27.252886984 ereport.fs.zfs.checksum
nvlist version: 0
class = ereport.fs.zfs.checksum
ena = 0x3a9567025fb00801
detector = (embedded nvlist)
nvlist version: 0
version = 0x0
scheme = zfs
pool = 0xda1ed4eddd886ca2
vdev = 0x4dd27d7d2cff5683
(end detector)
pool = volume0
pool_guid = 0xda1ed4eddd886ca2
pool_context = 0
pool_failmode = wait
vdev_guid = 0x4dd27d7d2cff5683
vdev_type = disk
vdev_path = /dev/dsk/c3t3d0s0
vdev_devid = id1,sd@SATA_____HDS725050KLA360_______KRVN03ZAG39LKD/a
parent_guid = 0xb85bb36665652fa6
parent_type = mirror
zio_err = 50
zio_offset = 0x4f8cf3c00
zio_size = 0xca00
zio_objset = 0x28
zio_object = 0x1
zio_level = 0
zio_blkid = 0x110e12
cksum_expected = 0x109aa00f800f 0x1aae29387c03843 0x289b0f3d871cdb32
0xb91c890d879e9fc4
cksum_actual = 0x109aa20f800f 0x1aae2d39fc03843 0x289f125e231cdb32
0xe3fb48d05f9e9fc4
cksum_algorithm = fletcher4
bad_ranges = 0x49d0 0x49d8
bad_ranges_min_gap = 0x8
bad_range_sets = 0x1
bad_range_clears = 0x0
bad_set_bits = 0x0 0x0 0x0 0x2 0x0 0x0 0x0 0x0
bad_cleared_bits = 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
__ttl = 0x1
__tod = 0x50781baf 0xf12bfc8
--
Dr.Udo Grabowski Inst.f.Meteorology a.Climate Research IMK-ASF-SAT
www-imk.fzk.de/asf/sat/grabowski/ www.imk-asf.kit.edu/english/sat.php
KIT - Karlsruhe Institute of Technology http://www.kit.edu
Postfach 3640,76021 Karlsruhe,Germany T:(+49)721 608-26026 F:-926026
_______________________________________________
OpenIndiana-discuss mailing list
[email protected]
http://openindiana.org/mailman/listinfo/openindiana-discuss
