Subject: mdadm: I'm also seeing this since the upgrade to lenny Followup-For: Bug #518834 Package: mdadm Version: 2.6.7.2-1
I'm also seeing this problem (increasing values
in /sys/block/md0/md/mismatch_cnt) on my system. I'm seeing it since the
upgrade to lenny. I upgraded my system from etch.
I use mirrorring on /dev/sda1 and /dev/sdb1 and use LVM on top of that.
Swap is also on there (is that a problem?) as well as root, /var, /home
and most everything else.
/dev/sda is a Maxtor 6V080E0 which does about 55 MB/sec, while /dev/sdb
is a MAXTOR STM380211AS which does about 65 MB/sec (speeds are hdparm -t
values). I'm doing S.M.A.R.T. monitoring and there is nothing wrong with
the drives as far as I can determine.
This is from my syslog archive (note that before the Mar 1 check I was
running etch):
Nov 2 01:32:59 bobo mdadm: RebuildFinished event detected on md device /dev/md0
(probably some logs missing due to incomplete archive)
Feb 1 01:28:27 bobo mdadm: RebuildFinished event detected on md device /dev/md0
Mar 1 01:21:06 bobo mdadm[6692]: RebuildFinished event detected on md device
/dev/md0, component device mismatches found: 3968
Apr 5 01:17:25 bobo mdadm[6692]: RebuildFinished event detected on md device
/dev/md0, component device mismatches found: 7168
Apr 6 11:40:02 bobo mdadm[6692]: RebuildFinished event detected on md device
/dev/md0, component device mismatches found: 7168
Apr 6 14:57:53 bobo mdadm[6692]: RebuildFinished event detected on md device
/dev/md0, component device mismatches found: 7296
Apr 6 20:13:50 bobo mdadm[6734]: RebuildFinished event detected on md device
/dev/md0, component device mismatches found: 7296
Apr 6 21:07:49 bobo mdadm[6734]: RebuildFinished event detected on md device
/dev/md0, component device mismatches found: 7424
Apr 6 21:33:03 bobo mdadm[6734]: RebuildFinished event detected on md device
/dev/md0
Apr 6 23:01:04 bobo mdadm[6734]: RebuildFinished event detected on md device
/dev/md0
Apr 7 09:59:47 bobo mdadm[6734]: RebuildFinished event detected on md device
/dev/md0, component device mismatches found: 128
Apr 7 13:57:41 bobo mdadm[6734]: RebuildFinished event detected on md device
/dev/md0
Apr 7 20:14:15 bobo mdadm[11310]: RebuildFinished event detected on md device
/dev/md0, component device mismatches found: 256
Apr 7 21:37:02 bobo mdadm[11310]: RebuildFinished event detected on md device
/dev/md0, component device mismatches found: 640
Apr 8 10:19:19 bobo mdadm[11310]: RebuildFinished event detected on md device
/dev/md0, component device mismatches found: 384
Before Apr 5 this makes almost one 128 mismatch a day. This is kind of
worrying. I have not experienced (as far as I know) any data loss but
having two disks grow silently out of sync is not good.
I've done some testing and performed a repair after Apr 6 21:07 but as
you can see after that the number still keeps going up. It also goes
down sometimes which is probably due to the problem-block being
re-written.
I have also performed some checks with cmp -lb /dev/sda1 /dev/sdb1 to
confirm that there were actual differences. If this is interesting, I
can provide those files (the last one I did just now was in single-user
mode with only read-only mounted root on the RAID device and the diff
was about 142K).
If you know a way to map these byte offsets to actual logical volumes
inside LVM and maybe even files in the filesystem I could narrow it
down. There always seems to be a difference at the very end of the
device but that is probably the RAID metadata.
If there is any more information that is needed I will do my best to
provide it.
Thanks.
-- Package-specific info:
--- mount output
/dev/mapper/main-root on / type ext3 (rw,noatime,errors=remount-ro)
tmpfs on /lib/init/rw type tmpfs (rw,nosuid,mode=0755)
proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
procbususb on /proc/bus/usb type usbfs (rw)
udev on /dev type tmpfs (rw,mode=0755)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=620)
fusectl on /sys/fs/fuse/connections type fusectl (rw)
/dev/mapper/main-tmp on /tmp type ext3 (rw,nosuid,noatime)
/dev/mapper/main-var on /var type ext3 (rw,noatime)
/dev/mapper/main-home on /home type ext3 (rw,nosuid,nodev,noatime)
/dev/mapper/main-srv on /srv type ext3 (rw,nosuid,nodev,noatime)
/dev/mapper/main-squid on /var/spool/squid type reiserfs
(rw,noexec,nosuid,nodev,noatime)
/dev/mapper/main-netsniff on /var/log/netsniff type ext3
(rw,noexec,nosuid,nodev,noatime)
/dev/sdc1 on /backup type ext3 (rw,nosuid,nodev,noatime)
/dev/sdc2 on /local type jfs (rw,nosuid,noatime)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
nfsd on /proc/fs/nfsd type nfsd (rw)
--- mdadm.conf
# mdadm.conf
#
# Please refer to mdadm.conf(5) for information about this file.
#
# by default, scan all partitions (/proc/partitions) for MD superblocks.
# alternatively, specify devices to scan, using wildcards if desired.
DEVICE partitions
# auto-create devices with Debian standard permissions
CREATE owner=root group=disk mode=0660 auto=yes
# automatically tag new arrays as belonging to the local system
HOMEHOST <system>
# instruct the monitoring daemon where to send mail alerts
MAILADDR root
# definitions of existing MD arrays
ARRAY /dev/md0 level=raid1 num-devices=2
UUID=4f391048:6f5bd779:2f0b4b8d:cf2aba3f
# This file was auto-generated on Thu, 03 May 2007 23:08:25 +0200
# by mkconf $Id: mkconf 261 2006-11-09 13:32:35Z madduck $
--- /proc/mdstat:
Personalities : [raid1]
md0 : active raid1 sda1[0] sdb1[1]
58596992 blocks [2/2] [UU]
unused devices: <none>
--- /proc/partitions:
major minor #blocks name
8 0 80043264 sda
8 1 58597056 sda1
8 2 21438742 sda2
8 16 78150744 sdb
8 17 58597056 sdb1
8 18 19551105 sdb2
8 32 244198584 sdc
8 33 67464936 sdc1
8 34 23358510 sdc2
9 0 58596992 md0
253 0 2097152 dm-0
253 1 1572864 dm-1
253 2 2097152 dm-2
253 3 524288 dm-3
253 4 31457280 dm-4
253 5 2097152 dm-5
253 6 6291456 dm-6
253 7 2097152 dm-7
--- initrd.img-2.6.26-1-amd64:
42132 blocks
etc/mdadm
etc/mdadm/mdadm.conf
sbin/mdadm
scripts/local-top/mdadm
lib/modules/2.6.26-1-amd64/kernel/drivers/md/dm-log.ko
lib/modules/2.6.26-1-amd64/kernel/drivers/md/raid1.ko
lib/modules/2.6.26-1-amd64/kernel/drivers/md/dm-mirror.ko
lib/modules/2.6.26-1-amd64/kernel/drivers/md/raid456.ko
lib/modules/2.6.26-1-amd64/kernel/drivers/md/dm-snapshot.ko
lib/modules/2.6.26-1-amd64/kernel/drivers/md/md-mod.ko
lib/modules/2.6.26-1-amd64/kernel/drivers/md/dm-mod.ko
lib/modules/2.6.26-1-amd64/kernel/drivers/md/raid0.ko
lib/modules/2.6.26-1-amd64/kernel/drivers/md/multipath.ko
lib/modules/2.6.26-1-amd64/kernel/drivers/md/raid10.ko
lib/modules/2.6.26-1-amd64/kernel/drivers/md/linear.ko
--- /proc/modules:
dm_mirror 20608 0 - Live 0xffffffffa0140000
dm_log 13956 1 dm_mirror, Live 0xffffffffa013b000
dm_snapshot 19400 0 - Live 0xffffffffa0135000
dm_mod 58864 20 dm_mirror,dm_log,dm_snapshot, Live 0xffffffffa0125000
raid1 24192 1 - Live 0xffffffffa011e000
md_mod 80164 2 raid1, Live 0xffffffffa0109000
--- /var/log/syslog:
--- volume detail:
/dev/sda1:
Magic : a92b4efc
Version : 00.90.00
UUID : 4f391048:6f5bd779:2f0b4b8d:cf2aba3f
Creation Time : Sun Apr 29 20:01:14 2007
Raid Level : raid1
Used Dev Size : 58596992 (55.88 GiB 60.00 GB)
Array Size : 58596992 (55.88 GiB 60.00 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 0
Update Time : Wed Apr 8 22:48:04 2009
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Checksum : f990d538 - correct
Events : 345934
Number Major Minor RaidDevice State
this 0 8 1 0 active sync /dev/sda1
0 0 8 1 0 active sync /dev/sda1
1 1 8 17 1 active sync /dev/sdb1
--
/dev/sdb1:
Magic : a92b4efc
Version : 00.90.00
UUID : 4f391048:6f5bd779:2f0b4b8d:cf2aba3f
Creation Time : Sun Apr 29 20:01:14 2007
Raid Level : raid1
Used Dev Size : 58596992 (55.88 GiB 60.00 GB)
Array Size : 58596992 (55.88 GiB 60.00 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 0
Update Time : Wed Apr 8 22:48:04 2009
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Checksum : f990d54a - correct
Events : 345934
Number Major Minor RaidDevice State
this 1 8 17 1 active sync /dev/sdb1
0 0 8 1 0 active sync /dev/sda1
1 1 8 17 1 active sync /dev/sdb1
--
--- /proc/cmdline
auto BOOT_IMAGE=Linux ro root=fd00
--- lilo:
root=/dev/mapper/main-root
-- System Information:
Debian Release: 5.0
APT prefers stable
APT policy: (500, 'stable')
Architecture: i386 (x86_64)
Kernel: Linux 2.6.26-1-amd64 (SMP w/2 CPU cores)
Locale: LANG=C, LC_CTYPE=C (charmap=ANSI_X3.4-1968)
Shell: /bin/sh linked to /bin/bash
Versions of packages mdadm depends on:
ii debconf 1.5.24 Debian configuration management sy
ii libc6 2.7-18 GNU C Library: Shared libraries
ii lsb-base 3.2-20 Linux Standard Base 3.2 init scrip
ii makedev 2.3.1-88 creates device files in /dev
ii udev 0.125-7 /dev/ and hotplug management daemo
Versions of packages mdadm recommends:
ii module-init-tools 3.4-1 tools for managing Linux kernel mo
ii postfix [mail-transport-agent 2.5.5-1.1 High-performance mail transport ag
mdadm suggests no packages.
-- debconf information:
* mdadm/autostart: true
* mdadm/initrdstart: all
* mdadm/initrdstart_notinconf: false
mdadm/initrdstart_msg_errexist:
mdadm/initrdstart_msg_intro:
mdadm/initrdstart_msg_errblock:
* mdadm/start_daemon: true
* mdadm/mail_to: root
mdadm/initrdstart_msg_errmd:
mdadm/initrdstart_msg_errconf:
* mdadm/autocheck: true
--
-- arthur - adej...@debian.org - http://people.debian.org/~adejong --
signature.asc
Description: This is a digitally signed message part
