Configuring link based IPMP on Solaris
This post describes how to configure link based IPMP interfaces in Solaris. IPMP eliminates single network card failure and it ensures system will be always accessible via network.
In this example we will configure a link based IPMP using network interfaces ce0 & ce1.
- Confirm NICs and status:
# dladm show-dev e1000g0 link: up speed: 1000 Mbps duplex: full ce0 link: unknown speed: 1000 Mbps duplex: full ce1 link: unknown speed: 1000 Mbps duplex: full # ifconfig -a lo0: flags=1000849 mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 e1000g0: flags=1000843 mtu 1500 index 2 inet 192.168.30.41 netmask ffffff00 broadcast 192.168.30.255 ether 8:0:20:93:c9:af ce0: flags=1000842 mtu 1500 index 3 inet 0.0.0.0 netmask 0 ether 0:3:ba:56:b:51 ce1: flags=1000842 mtu 1500 index 4 inet 0.0.0.0 netmask 0 ether 0:3:ba:56:b:52
- Add the required IP address (
/etc/hosts
) and netmask (/etc/netmasks
)# grep 10.1.1.10 /etc/hosts 10.1.1.10 ipmptest # grep 10.1.1 /etc/netmasks 10.1.1.0 255.255.255.0
- Bring up (plumb) the NICs
# ifconfig ce0 plumb # ifconfig ce1 plumb
- Add IP address and IP multipathing
# ifconfig ce0 10.1.1.10 netmask 255.255.255.0 broadcast + group ipmp1 up # ifconfig ce1 group ipmp1 up
- Verify configuration
# dladm show-dev e1000g0 link: up speed: 1000 Mbps duplex: full ce0 link: up speed: 1000 Mbps duplex: full ce1 link: up speed: 1000 Mbps duplex: full # ifconfig -a lo0: flags=1000849 mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 e1000g0: flags=1000843 mtu 1500 index 2 inet 192.168.30.41 netmask ffffff00 broadcast 192.168.30.255 ether 8:0:20:93:c9:af ce0: flags=1000842 mtu 1500 index 3 inet 10.1.1.10 netmask ffffff00 broadcast 10.1.1.255 groupname ipmp1 ether 0:3:ba:56:b:51 ce1: flags=1000843 mtu 1500 index 4 inet 0.0.0.0 netmask ff000000 broadcast 0.255.255.255 groupname ipmp1 ether 0:3:ba:56:b:52
- Verify IPMP failover with the following commands. Basically enable and disable interfaces using the
if_mpadm
command:# ip_mpadm -d ce0 # ifconfig -a # ip_mpadm -r ce0 # ifconfig -a # ip_mpadm -d ce1 # ifconfig -a # ip_mpadm -r ce1 # ifconfig -a
All failovers are logged in/var/adm/messages
. For example:# grep in.mpathd /var/adm/messages Apr 14 09:32:24 ipmptest in.mpathd[3800]: [ID 215189 daemon.error] The link has gone down on ce0 Apr 14 09:32:24 ipmptest in.mpathd[3800]: [ID 594170 daemon.error] NIC failure detected on ce0 of group ipmp1 Apr 14 09:32:24 ipmptest in.mpathd[3800]: [ID 832587 daemon.error] Successfully failed over from NIC ce0 to NIC ce1 Apr 14 09:32:57 ipmptest in.mpathd[3800]: [ID 820239 daemon.error] The link has come up on ce1 Apr 14 09:32:57 ipmptest in.mpathd[3800]: [ID 299542 daemon.error] NIC repair detected on ce0 of group ipmp1 Apr 14 09:32:57 ipmptest in.mpathd[3800]: [ID 620804 daemon.error] Successfully failed back to NIC ce0 Apr 14 09:46:59 ipmptest in.mpathd[3800]: [ID 832587 daemon.error] Successfully failed over from NIC ce1 to NIC ce0 Apr 14 09:47:07 ipmptest in.mpathd[3800]: [ID 620804 daemon.error] Successfully failed back to NIC ce0
- Persistent configuration... To make our IPMP work across system reboots, we create configuration files for both the NICs. For example:
# cat /etc/hostname.ce0 10.1.1.10 netmask + broadcast + group ipmp1 up # cat /etc/hostname.ce1 group ipmp1 up