Multipathing with mdadm on Linux
There are numerous SAN multipathing solutions for Linux. In this post I discuss MD multipathing, which is just a failover solution with manuel re-anable of a failed path. The advantage of using this method is that it is very easy to configure with mdadm
.
- If not installed, install the
mdadm
package. For example:- RHEL/CentOS:
root@centos# yum install mdadm -y
- Ubuntu:
root@ubuntu# apt install mdadm -y
- RHEL/CentOS:
- Enable and start MD multipathing service:
root@linux# chkconfig mdmpd on root@linux# /etc/init.d/mdmpd start
mdmpd
is a daemon used for monitoring multipath (RAID) devices, originally developed by Red Hat as part of themdadm
package, and is usually started at boot time as a service, and afterwards runs as a daemon. - Confirm attached hard drives are detected:
root@linux# ls -l /dev | grep sd brw-rw---- 1 root disk 8, 0 Nov 11 2007 sda brw-rw---- 1 root disk 8, 1 Nov 11 09:14 sda1 brw-rw---- 1 root disk 8, 2 Nov 11 2007 sda2 brw-rw---- 1 root disk 8, 16 Nov 11 2007 sdb brw-rw---- 1 root disk 8, 32 Nov 11 2007 sdc brw-rw---- 1 root disk 8, 48 Nov 11 2007 sdd brw-rw---- 1 root disk 8, 64 Nov 11 2007 sde
From the above we see that our new devices (sdb
-sde
) are available. - Partition the new devices. Starting with sdb:
root@linux# fdisk /dev/sdb
- Press n to create a new partition
- Choose P for primary partition
- Select partition number as 1.
- Select the default values by pressing Enter twice.
- Type t to set partition type
- Enter fd for 'Linux raid auto', press Enter to apply.
- Type w to write changes to disk.
sdc
,sdd
andsde
). - Run
partprobe
to tell the OS to re-read the partition table:root@linux# partprobe
- Use
mdmadm
to bind the multipaths together:root@linux# mdadm --create /dev/md0 --level=multipath --raid-devices=4 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1
For our example our multipath device is/dev/md0
. - Get the UUID of the new multipathing device:
root@linux# mdadm --detail /dev/md0 UUID : b96d3014:3d8ea8c0:14005868:f1e68b27
- Add the multipathing configuration to /etc/mdadm.conf. For example:
DEVICE /dev/sd[bcde]1 ARRAY /dev/md0 uuid=b96d3014:3d8ea8c0:14005868:f1e68b27
In our example configuration:
- For DEVICE, /dev/sd[bcde]1 are our disks, as found and configured in steps #3 and #4,
- For ARRAY, /dev/md0 is our newly created MD-device from step #6,
- and the uuid=<value> is the value produced from the
mdadm
output as found in step #7.
Additional steps
If this is a shared device across multiple hosts, then perform the next nests on each additional host:
- From the original node, copy
/etc/mdadm.conf
onto each additional host. - On the additional hosts:
- Install and start the
mdadm
package (if necessary), - Read in the configuration file
/etc/mdadm.conf
:root@node2# mdadm -As
This will assemble a pre-existing array and scan the config file or/proc/mdstat
for missing information. - Finally, check your configuration is correct:
root@node2# cat /proc/mdstat
- Install and start the