Multipathing with mdadm on Linux

There are numerous SAN multipathing solutions for Linux. In this post I discuss MD multipathing, which is just a failover solution with manuel re-anable of a failed path. The advantage of using this method is that it is very easy to configure with mdadm.

1. If not installed, install the mdadm package. For example:
   • RHEL/CentOS:

     root@centos# yum install mdadm -y

   • Ubuntu:

     root@ubuntu# apt install mdadm -y

2. Enable and start MD multipathing service:

   root@linux# chkconfig mdmpd on
   root@linux# /etc/init.d/mdmpd start

   mdmpd is a daemon used for monitoring multipath (RAID) devices, originally developed by Red Hat as part of the mdadm package, and is usually started at boot time as a service, and afterwards runs as a daemon.
3. Confirm attached hard drives are detected:

   root@linux# ls -l /dev | grep sd
   brw-rw---- 1 root disk      8,   0 Nov 11 2007  sda
   brw-rw---- 1 root disk      8,   1 Nov 11 09:14 sda1
   brw-rw---- 1 root disk      8,   2 Nov 11 2007  sda2
   brw-rw---- 1 root disk      8,  16 Nov 11 2007  sdb  
   brw-rw---- 1 root disk      8,  32 Nov 11 2007  sdc  
   brw-rw---- 1 root disk      8,  48 Nov 11 2007  sdd  
   brw-rw---- 1 root disk      8,  64 Nov 11 2007  sde  

   From the above we see that our new devices (sdb-sde) are available.
4. Partition the new devices. Starting with sdb:

   root@linux# fdisk /dev/sdb

   • Press n to create a new partition
   • Choose P for primary partition
   • Select partition number as 1.
   • Select the default values by pressing Enter twice.
   • Type t to set partition type
   • Enter fd for 'Linux raid auto', press Enter to apply.
   • Type w to write changes to disk.
   Repeat the process for the remaining devices (sdc, sdd and sde).
5. Run partprobe to tell the OS to re-read the partition table:

   root@linux# partprobe

6. Use mdmadm to bind the multipaths together:

   root@linux# mdadm --create /dev/md0 --level=multipath --raid-devices=4 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1

   For our example our multipath device is /dev/md0.
7. Get the UUID of the new multipathing device:

   root@linux# mdadm --detail /dev/md0
   UUID : b96d3014:3d8ea8c0:14005868:f1e68b27

8. Add the multipathing configuration to /etc/mdadm.conf. For example:

   DEVICE /dev/sd[bcde]1
   ARRAY /dev/md0 uuid=b96d3014:3d8ea8c0:14005868:f1e68b27
   

   In our example configuration:
   
   • For DEVICE, /dev/sd[bcde]1 are our disks, as found and configured in steps #3 and #4,
   • For ARRAY, /dev/md0 is our newly created MD-device from step #6,
   • and the uuid=<value> is the value produced from the mdadm output as found in step #7.

Additional steps

If this is a shared device across multiple hosts, then perform the next nests on each additional host:

1. From the original node, copy /etc/mdadm.conf onto each additional host.
2. On the additional hosts:
   1. Install and start the mdadm package (if necessary),
   2. Read in the configuration file /etc/mdadm.conf:

      root@node2# mdadm -As

      This will assemble a pre-existing array and scan the config file or /proc/mdstat for missing information.
   3. Finally, check your configuration is correct:

      root@node2# cat /proc/mdstat