Package: multipath-tools
Version: 0.6.1
Multipath-tools creates device files in /dev/mapper for both the block
device and its partitions. The partition device file use the format
/dev/mapper/<mpath-device>-partX, where X is the partition number.
Example:
Configuration details:
Testing on an amd64 system running sid with
multipath-tools/kpartx_0.6.1-3 from experimental. We have 8 FC LUNs
with 4 paths each. The kernel version is 4.6.0-1-amd64.
With the following /etc/multipath.conf file:
defaults {
user_friendly_names "yes"
}
blacklist {
device {
vendor "HP"
product "LOGICAL VOLUME"
}
Here are the devices used (the boot device is blacklisted):
# lsscsi
[0:0:0:0] storage HP P244br 2.52 -
[0:1:0:0] disk HP LOGICAL VOLUME 2.52 /dev/sda
[1:0:0:0] disk 3PARdata VV 3212 /dev/sdj
[1:0:0:1] disk 3PARdata VV 3212 /dev/sdl
[1:0:0:2] disk 3PARdata VV 3212 /dev/sdn
[1:0:0:3] disk 3PARdata VV 3212 /dev/sdq
[1:0:0:4] disk 3PARdata VV 3212 /dev/sdr
[1:0:0:5] disk 3PARdata VV 3212 /dev/sdt
[1:0:0:6] disk 3PARdata VV 3212 /dev/sdv
[1:0:0:7] disk 3PARdata VV 3212 /dev/sdx
[1:0:0:254] enclosu 3PARdata SES 3212 -
[1:0:1:0] disk 3PARdata VV 3212 /dev/sdz
[1:0:1:1] disk 3PARdata VV 3212 /dev/sdaa
[1:0:1:2] disk 3PARdata VV 3212 /dev/sdab
[1:0:1:3] disk 3PARdata VV 3212 /dev/sdac
[1:0:1:4] disk 3PARdata VV 3212 /dev/sdad
[1:0:1:5] disk 3PARdata VV 3212 /dev/sdae
[1:0:1:6] disk 3PARdata VV 3212 /dev/sdaf
[1:0:1:7] disk 3PARdata VV 3212 /dev/sdag
[1:0:1:254] enclosu 3PARdata SES 3212 -
[2:0:0:0] disk 3PARdata VV 3212 /dev/sdb
[2:0:0:1] disk 3PARdata VV 3212 /dev/sdc
[2:0:0:2] disk 3PARdata VV 3212 /dev/sdd
[2:0:0:3] disk 3PARdata VV 3212 /dev/sde
[2:0:0:4] disk 3PARdata VV 3212 /dev/sdf
[2:0:0:5] disk 3PARdata VV 3212 /dev/sdg
[2:0:0:6] disk 3PARdata VV 3212 /dev/sdh
[2:0:0:7] disk 3PARdata VV 3212 /dev/sdi
[2:0:0:254] enclosu 3PARdata SES 3212 -
[2:0:1:0] disk 3PARdata VV 3212 /dev/sdk
[2:0:1:1] disk 3PARdata VV 3212 /dev/sdm
[2:0:1:2] disk 3PARdata VV 3212 /dev/sdo
[2:0:1:3] disk 3PARdata VV 3212 /dev/sdp
[2:0:1:4] disk 3PARdata VV 3212 /dev/sds
[2:0:1:5] disk 3PARdata VV 3212 /dev/sdu
[2:0:1:6] disk 3PARdata VV 3212 /dev/sdw
[2:0:1:7] disk 3PARdata VV 3212 /dev/sdy
[2:0:1:254] enclosu 3PARdata SES 3212 -
And the following multipath -l output:
# multipath -l
mpathe (360002ac00000000000001908000028be) dm-4 3PARdata,VV
size=1.0G features='0' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=0 status=active
|- 1:0:0:4 sdr 65:16 active undef running
|- 1:0:1:4 sdad 65:208 active undef running
|- 2:0:0:4 sdf 8:80 active undef running
`- 2:0:1:4 sds 65:32 active undef running
mpathd (360002ac00000000000001907000028be) dm-3 3PARdata,VV
size=1.0G features='0' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=0 status=active
|- 1:0:0:3 sdq 65:0 active undef running
|- 1:0:1:3 sdac 65:192 active undef running
|- 2:0:0:3 sde 8:64 active undef running
`- 2:0:1:3 sdp 8:240 active undef running
mpathc (360002ac00000000000001906000028be) dm-2 3PARdata,VV
size=1.0G features='0' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=0 status=active
|- 1:0:0:2 sdn 8:208 active undef running
|- 1:0:1:2 sdab 65:176 active undef running
|- 2:0:0:2 sdd 8:48 active undef running
`- 2:0:1:2 sdo 8:224 active undef running
mpathb (360002ac00000000000001905000028be) dm-1 3PARdata,VV
size=1.0G features='0' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=0 status=active
|- 1:0:0:1 sdl 8:176 active undef running
|- 1:0:1:1 sdaa 65:160 active undef running
|- 2:0:0:1 sdc 8:32 active undef running
`- 2:0:1:1 sdm 8:192 active undef running
mpatha (360002ac00000000000001904000028be) dm-0 3PARdata,VV
size=1.0G features='0' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=0 status=active
|- 1:0:0:0 sdj 8:144 active undef running
|- 1:0:1:0 sdz 65:144 active undef running
|- 2:0:0:0 sdb 8:16 active undef running
`- 2:0:1:0 sdk 8:160 active undef running
mpathh (360002ac0000000000000190b000028be) dm-7 3PARdata,VV
size=1.0G features='0' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=0 status=active
|- 1:0:0:7 sdx 65:112 active undef running
|- 1:0:1:7 sdag 66:0 active undef running
|- 2:0:0:7 sdi 8:128 active undef running
`- 2:0:1:7 sdy 65:128 active undef running
mpathg (360002ac0000000000000190a000028be) dm-6 3PARdata,VV
size=1.0G features='0' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=0 status=active
|- 1:0:0:6 sdv 65:80 active undef running
|- 1:0:1:6 sdaf 65:240 active undef running
|- 2:0:0:6 sdh 8:112 active undef running
`- 2:0:1:6 sdw 65:96 active undef running
mpathf (360002ac00000000000001909000028be) dm-5 3PARdata,VV
size=1.0G features='0' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=0 status=active
|- 1:0:0:5 sdt 65:48 active undef running
|- 1:0:1:5 sdae 65:224 active undef running
|- 2:0:0:5 sdg 8:96 active undef running
`- 2:0:1:5 sdu 65:64 active undef running
With the following device files in /dev/mapper:
# ls -l /dev/mapper
total 0
crw------- 1 root root 10, 236 Jun 14 13:28 control
lrwxrwxrwx 1 root root 7 Jun 14 13:53 mpatha -> ../dm-0
lrwxrwxrwx 1 root root 7 Jun 14 13:53 mpathb -> ../dm-1
lrwxrwxrwx 1 root root 7 Jun 14 13:53 mpathc -> ../dm-2
lrwxrwxrwx 1 root root 7 Jun 14 13:53 mpathd -> ../dm-3
lrwxrwxrwx 1 root root 7 Jun 14 13:53 mpathe -> ../dm-4
lrwxrwxrwx 1 root root 7 Jun 14 13:53 mpathf -> ../dm-5
lrwxrwxrwx 1 root root 7 Jun 14 13:53 mpathg -> ../dm-6
lrwxrwxrwx 1 root root 7 Jun 14 13:53 mpathh -> ../dm-7
We now create and remove a partition using fdisk.
# fdisk /dev/mapper/mpatha
Welcome to fdisk (util-linux 2.28).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
Command (m for help): n
Partition number (1-128, default 1):
First sector (34-2097118, default 32768):
Last sector, +sectors or +size{K,M,G,T,P} (32768-2097118, default 2097118):
Created a new partition 1 of type 'Linux filesystem' and of size 1008 MiB.
Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Re-reading the partition table failed.: Invalid argument
The kernel still uses the old table. The new table will be used at the next
reboot or after you run partprobe(8) or kpartx(8).
Resulting in:
# ls -l /dev/mapper/mpatha*
lrwxrwxrwx 1 root root 7 Jun 15 14:58 /dev/mapper/mpatha -> ../dm-0
lrwxrwxrwx 1 root root 7 Jun 15 14:58 /dev/mapper/mpatha-part1 -> ../dm-8
# ls -l /dev/disk/by-id/scsi-mpatha*
lrwxrwxrwx 1 root root 10 Jun 15 15:02 /dev/disk/by-id/scsi-mpatha -> ../../dm-0
lrwxrwxrwx 1 root root 10 Jun 15 15:02 /dev/disk/by-id/scsi-mpatha-part1 ->
../../dm-2
Now when I remove this partition:
# fdisk /dev/mapper/mpatha
Welcome to fdisk (util-linux 2.28).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
Command (m for help): d
Selected partition 1
Partition 1 has been deleted.
Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Re-reading the partition table failed.: Invalid argument
The kernel still uses the old table. The new table will be used at the next
reboot or after you run partprobe(8) or kpartx(8).
The /dev/mapper partition device file is still present:
# ls -l /dev/mapper/mpatha*
lrwxrwxrwx 1 root root 7 Jun 15 15:05 /dev/mapper/mpatha -> ../dm-0
lrwxrwxrwx 1 root root 7 Jun 15 15:02 /dev/mapper/mpatha-part1 -> ../dm-2
# ls -l /dev/disk/by-id/scsi-mpatha
lrwxrwxrwx 1 root root 10 Jun 15 15:13 /dev/disk/by-id/scsi-mpatha -> ../../dm-2
Note that the /dev/disk/by-path file is gone while the /dev/mapper
device file remains.
Here is the output from udevadm monitor while removing the partition:
KERNEL[155.958774] change /devices/virtual/block/dm-2 (block)
UDEV [155.986198] change /devices/virtual/block/dm-2 (block)
and /proc/partitions indicates that the partition is gone:
# cat /proc/partitions | grep -- dm-
254 0 1048576 dm-0
254 1 1048576 dm-1
254 2 1048576 dm-2
254 3 1048576 dm-3
254 4 1048576 dm-4
254 5 1048576 dm-5
254 6 1048576 dm-6
254 7 1048576 dm-7
254 8 1032175 dm-8
This seems to work correctly when using parted, e.g.,
# ls -l /dev/mapper/mpathb*
lrwxrwxrwx 1 root root 7 Jun 15 15:19 /dev/mapper/mpathb -> ../dm-3
lrwxrwxrwx 1 root root 7 Jun 15 15:19 /dev/mapper/mpathb1 -> ../dm-9
# ls -l /dev/disk/by-id/scsi-mpathb*
lrwxrwxrwx 1 root root 10 Jun 15 15:19 /dev/disk/by-id/scsi-mpathb -> ../../dm-3
lrwxrwxrwx 1 root root 10 Jun 15 15:19 /dev/disk/by-id/scsi-mpathb1 ->
../../dm-9
# parted /dev/mapper/mpathb
GNU Parted 3.2
Using /dev/mapper/mpathb
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) rm 1
(parted) quit
Information: You may need to update /etc/fstab.
# ls -l /dev/mapper/mpathb*
lrwxrwxrwx 1 root root 7 Jun 15 15:22 /dev/mapper/mpathb -> ../dm-3
# ls -l /dev/disk/by-id/scsi-mpathb*
lrwxrwxrwx 1 root root 10 Jun 15 15:22 /dev/disk/by-id/scsi-mpathb -> ../../dm-3
Here is the output for udevadm monitor when deleting the parted partition:
KERNEL[735.524037] change /devices/virtual/block/dm-9 (block)
KERNEL[735.524470] change /devices/virtual/block/dm-3 (block)
UDEV [735.540771] change /devices/virtual/block/dm-9 (block)
KERNEL[735.543465] add /devices/virtual/bdi/254:10 (bdi)
KERNEL[735.543534] add /devices/virtual/block/dm-10 (block)
UDEV [735.543727] add /devices/virtual/bdi/254:10 (bdi)
KERNEL[735.543795] remove /devices/virtual/block/dm-10 (block)
UDEV [735.548136] add /devices/virtual/block/dm-10 (block)
UDEV [735.548265] remove /devices/virtual/block/dm-10 (block)
KERNEL[735.556226] remove /devices/virtual/bdi/254:10 (bdi)
UDEV [735.556447] remove /devices/virtual/bdi/254:10 (bdi)
UDEV [735.573117] change /devices/virtual/block/dm-3 (block)
KERNEL[739.475199] change /devices/virtual/block/dm-9 (block)
UDEV [739.517662] change /devices/virtual/block/dm-9 (block)
KERNEL[739.518452] remove /devices/virtual/block/dm-9 (block)
KERNEL[739.518643] remove /devices/virtual/block/dm-9 (block)
UDEV [739.521683] remove /devices/virtual/block/dm-9 (block)
UDEV [739.521786] remove /devices/virtual/block/dm-9 (block)
KERNEL[739.532256] remove /devices/virtual/bdi/254:9 (bdi)
UDEV [739.532847] remove /devices/virtual/bdi/254:9 (bdi)
KERNEL[744.961662] change /devices/virtual/block/dm-3 (block)
UDEV [744.981294] change /devices/virtual/block/dm-3 (block)
Note, we are now getting events for the partition as well as the parent
block device.
Note that when using parted, we get a different partition device file
-- /dev/mapper/mpathb1 than that created with fdisk. This difference
is discussed in bug
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=827308/
I have seen this same issue in earlier versions of multipath-tools.
--
Andrew Patterson
Hewlett-Packard Enterprise
