Multipathing with mdadm on Linux

There are numerous SAN multipathing solutions for Linux. In this post I discuss MD multipathing, which is just a failover solution with manuel re-anable of a failed path. The advantage of using this method is that it is very easy to configure with mdadm.

  1. If not installed, install the mdadm package. For example:
    • RHEL/CentOS:
      root@centos# yum install mdadm -y
    • Ubuntu:
      root@ubuntu# apt install mdadm -y
  2. Enable and start MD multipathing service:
    root@linux# chkconfig mdmpd on
    root@linux# /etc/init.d/mdmpd start
    mdmpd is a daemon used for monitoring multipath (RAID) devices, originally developed by Red Hat as part of the mdadm package, and is usually started at boot time as a service, and afterwards runs as a daemon.
  3. Confirm attached hard drives are detected:
    root@linux# ls -l /dev | grep sd
    brw-rw---- 1 root disk      8,   0 Nov 11 2007  sda
    brw-rw---- 1 root disk      8,   1 Nov 11 09:14 sda1
    brw-rw---- 1 root disk      8,   2 Nov 11 2007  sda2
    brw-rw---- 1 root disk      8,  16 Nov 11 2007  sdb  
    brw-rw---- 1 root disk      8,  32 Nov 11 2007  sdc  
    brw-rw---- 1 root disk      8,  48 Nov 11 2007  sdd  
    brw-rw---- 1 root disk      8,  64 Nov 11 2007  sde  
    From the above we see that our new devices (sdb-sde) are available.
  4. Partition the new devices. Starting with sdb:
    root@linux# fdisk /dev/sdb
      • Press n to create a new partition
      • Choose P for primary partition
      • Select partition number as 1.
      • Select the default values by pressing Enter twice.
      • Type t to set partition type
      • Enter fd for 'Linux raid auto', press Enter to apply.
      • Type w to write changes to disk.
    Repeat the process for the remaining devices (sdc, sdd and sde).
  5. Run partprobe to tell the OS to re-read the partition table:
    root@linux# partprobe
  6. Use mdmadm to bind the multipaths together:
    root@linux# mdadm --create /dev/md0 --level=multipath --raid-devices=4 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1
    For our example our multipath device is /dev/md0.
  7. Get the UUID of the new multipathing device:
    root@linux# mdadm --detail /dev/md0
    UUID : b96d3014:3d8ea8c0:14005868:f1e68b27
  8. Add the multipathing configuration to /etc/mdadm.conf. For example:
    DEVICE /dev/sd[bcde]1
    ARRAY /dev/md0 uuid=b96d3014:3d8ea8c0:14005868:f1e68b27
    
    In our example configuration:
    • For DEVICE, /dev/sd[bcde]1 are our disks, as found and configured in steps #3 and #4,
    • For ARRAY, /dev/md0 is our newly created MD-device from step #6,
    • and the uuid=<value> is the value produced from the mdadm output as found in step #7.

Additional steps

If this is a shared device across multiple hosts, then perform the next nests on each additional host:

  1. From the original node, copy /etc/mdadm.conf onto each additional host.
  2. On the additional hosts:
    1. Install and start the mdadm package (if necessary),
    2. Read in the configuration file /etc/mdadm.conf:
      root@node2# mdadm -As
      This will assemble a pre-existing array and scan the config file or /proc/mdstat for missing information.
    3. Finally, check your configuration is correct:
      root@node2# cat /proc/mdstat