thanks for the quick reply.
On Tue, Jan 06, 2009 at 08:50:28AM +1100, Vincent McIntyre wrote:
Other things I noticed:
- fdisk works ok
# fdisk -l /dev/sdc
Disk /dev/sdc: 1505.9 GB, 1505973239808 bytes
64 heads, 32 sectors/track, 1436208 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Device Boot Start End Blocks Id System
/dev/sdc1 1 1436208 1470676976 83 Linux
Once you use multipath you have to access multipathed devices via
/dev/mapper/<wwid> or via /dev/disk/by-id/. Not doing this is the only
bug here I can see.
ok, then that's the problem. There are no multipathed devices on this
system, so I was not expecting multipath-tools to 'do' anything.
Is there something in the package documentation that would have explained
this point to me? I may have missed it.
Christophe's documentation says 'Last update : Dec 2004', which does
not inspire confidence that it is still even vaguely relevant,
particularly since udev has evolved so much over the interim.
The URL for the FAQ has changed, it should be
http://christophe.varoqui.free.fr/faq.html (patch attached)
- once booted, /dev/mapper is left in a very odd state indeed:
# ls -l /dev/mapper
brw-rw---- 1 root disk 254, 8 2009-01-05 14:35
36000393000007d3901000000fef00a2d
brw-rw---- 1 root disk 254, 9 2009-01-05 14:35
36000393000007d3901000000fef00a2d1
brw-rw---- 1 root disk 254, 7 2009-01-05 14:35
36000393000007da901000000fef00a2d
It seems multipath tools picked up your devices correctly. What did you
expect to happen?
I expected multipath-tools to leave the device alone, or at least
allow the system to mount and be able to access the data as before,
since I had not 'told' the package about these devices.
I had not configured /etc/multipath.conf, that file did not exist.
Nor was there an /etc/default/multipath-tools.
On a system where we _do_ have multipathed storage, the devices are
referred to as /dev/mapper/mpath0p1, /dev/mapper/mpath1p1, etc.
So when I was looking at the /dev/mapper directory I was thinking that
somehow some configuration had not completed, and that the files
named using the WWN would be replaced by something 'friendlier'
(for example the filesystem label, which is what was being used in
the /etc/fstab, or /dev/mapper/mpath0).
The behaviour of 'mount' and 'umount' was also extremely confusing.
The device is not mounted and cannot be umounted but is 'busy'.
Why does 'lsof' not show what is keeping the device 'busy'?
It might help if /dev/sdc1 was not created at all, since it
can't be used to access the device. But I see in [2] that this
is a decision taken by the kernel.
The underlying bug here is probably my lack of understanding.
But at the moment I don't see how to improve my understanding with
the information available in the package. Some suggestions:
* if the package is able (at installation time) to detect devices that
it will start to manage at the next reboot, it would be helpful to
give a debconf warning about this and point people to /usr/share/doc.
I don't think there is such a warning now.
* provide your paragraph in /usr/share/doc:
Once you use multipath you have to access multipathed devices via
/dev/mapper/<wwid> or via /dev/disk/by-id/.
For example, in /etc/fstab one could write:
/dev/mapper/<wwid> /data/foo_1 ext3 defaults 0 0
* it would help to explain what the WWN-named files in /dev/mapper are.
I've had a go at a patch for this and the point above.
* alternatively, or as well, provide a default /etc/multipath.conf that
ignores/blacklists every device on the system, so multipath-tools won't
attempt to manage anything until the system owner takes a positive
action and configures the package.
You may well ask 'wtf did you install the package in the first place?'.
I was having a look-see to see what the package did when it was installed,
before trying it out on a system that does have multipathed storage.
I neglected to uninstall it again and was penalized heavily for this.
You're doing a great job on this package, hope these comments help.
Cheers
Vince
[1] http://christophe.varoqui.free.fr/usage.html
[2] http://christophe.varoqui.free.fr/refbook.html
--- /usr/share/doc/multipath-tools/FAQ 2006-03-14 02:12:08.000000000 +1100
+++ /tmp/FAQ 2009-01-07 09:03:32.000000000 +1100
@@ -1,4 +1,5 @@
-More at http://christophe.varoqui.free.fr/wiki/wakka.php?wiki=FAQ
+More at http://christophe.varoqui.free.fr/faq.html
+See also http://christophe.varoqui.free.fr/usage.html
1. How to set up System-on-multipath ?
======================================
--- /usr/share/doc/multipath-tools/FAQ 2006-03-14 02:12:08.000000000 +1100
+++ /tmp/FAQ 2009-01-07 09:13:05.000000000 +1100
@@ -63,3 +63,22 @@ current/default config like so:
lvm dumpconfig > /etc/lvm/lvm.conf
(tip from Christophe Saout)
+
+4. What are these weird numbers in /dev/mapper?
+===============================================
+Modern Fibre-Channel connected storage devices are identified by
+World-Wide-Numbers. If multipath-tools detects one of these devices,
+it uses that as the default identifier in /dev/mapper.
+
+5. Where did my FC-connected filesystem go?
+===========================================
+If you were previously mounting a FC-connected device and installed
+multipath-tools, you need to change the way you mount it. The device
+must be accessed using the identifier by which device-mapper knows it.
+
+For example if you have in /dev/mapper a file like this:
+ brw-rw---- 1 root disk 254, 8 2009-01-05 14:35 /dev/mapper/36000393000007d3901000000fef00a2d
+
+you can mount the filesystem like this:
+ mount /dev/mapper/36000393000007d3901000000fef00a2d /mnt
+
