Why You Should Use IPMP
IPMP provides increased reliability, availability, and network performance for systems with multiple physical
interfaces. Occasionally, a physical interface or the networking hardware attached to that interface
might fail or require maintenance. Traditionally, at that point, the system can no
longer be contacted through any of the IP addresses that are associated with
the failed interface. Additionally, any existing connections to the system using those IP
addresses are disrupted.
By using IPMP, you can configure one or more physical interfaces into an
IP multipathing group, or IPMP group. After configuring IPMP, the system automatically monitors
the interfaces in the IPMP group for failure. If an interface in the
group fails or is removed for maintenance, IPMP automatically migrates, or fails over, the
failed interface's IP addresses. The recipient of these addresses is a functioning interface in
the failed interface's IPMP group. The failover feature of IPMP preserves connectivity and
prevents disruption of any existing connections. Additionally, IPMP improves overall network
performance by automatically spreading out network traffic across the set of interfaces in
the IPMP group. This process is called load spreading.
Solaris IPMP Components
Solaris IPMP involves the following software:
The in.mpathd daemon, which is explained fully in the in.mpathd(1M) man page.
The /etc/default/mpathd configuration file, which is also described in the in.mpathd(1M) man page.
ifconfig options for IPMP configuration, as described in the ifconfig(1M) man page.
Multipathing Daemon, in.mpathd
The in.mpathd daemon detects interface failures, and then implements various procedures for failover
and failback. After in.mpathd detects a failure or a repair, the daemon
sends an ioctl to perform the failover or failback. The ip kernel module,
which implements the ioctl, does the network access failover transparently and automatically.
Note - Do not use Alternate Pathing while using IPMP on the same set of
network interface cards. Likewise, you should not use IPMP while you are using
Alternate Pathing. You can use Alternate Pathing and IPMP at the same time
on different sets of interfaces. For more information about Alternate Pathing, refer to
the Sun Enterprise Server Alternate Pathing 2.3.1 User Guide.
The in.mpathd daemon detects failures and repairs by sending out probes on all
the interfaces that are part of an IPMP group. The in.mpathd daemon also
detects failures and repairs by monitoring the RUNNING flag on each interface in the
group. Refer to the in.mpathd(1M) man page for more information.
IPMP Terminology and Concepts
This section introduces terms and concepts that are used throughout Part V, IPMP.
IP Link
In IPMP terminology, an IP link is a communication facility or medium over which
nodes can communicate at the data-link layer of the Internet protocol suite. Types
of IP links might include simple Ethernets, bridged Ethernets, hubs, or Asynchronous Transfer
Mode (ATM) networks. An IP link can have one or more IPv4 subnet
numbers, and, if applicable, one or more IPv6 subnet prefixes. A subnet number
or prefix cannot be assigned to more than one IP link. In ATM
LANE, an IP link is a single emulated local area network (LAN). With
the Address Resolution Protocol (ARP), the scope of the ARP protocol is a
single IP link.
Note - Other IP-related documents, such as RFC 2460, Internet Protocol, Version 6 (IPv6) Specification, use the term link instead
of IP link. Part VI uses the term IP link to avoid confusion with IEEE 802.
In IEEE 802, link refers to a single wire from an Ethernet network
interface card (NIC) to an Ethernet switch.
Physical Interface
The physical interface provides a system's attachment to an IP link. This attachment is
often implemented as a device driver and a NIC. If a system has
multiple interfaces attached to the same link, you can configure IPMP to perform
failover if one of the interfaces fails. For more information on physical interfaces,
refer to IPMP Interface Configurations.
Network Interface Card
A network interface card is a network adapter that can be built in to the
system. Or, the NIC can be a separate card that serves as
an interface from the system to an IP link. Some NICs can
have multiple physical interfaces. For example, a qfe NIC can have four interfaces, qfe0
through qfe3, and so on.
IPMP Group
An IP multipathing group, or IPMP group, consists of one or more physical
interfaces on the same system that are configured with the same IPMP group
name. All interfaces in the IPMP group must be connected to the same
IP link. The same (non-null) character string IPMP group name identifies all interfaces
in the group. You can place interfaces from NICs of different speeds within
the same IPMP group, as long as the NICs are of the same
type. For example, you can configure the interfaces of 100-megabit Ethernet NICs and
the interfaces of one gigabit Ethernet NICs in the same group. As
another example, suppose you have two 100-megabit Ethernet NICs. You can configure one
of the interfaces down to 10 megabits and still place the two interfaces
into the same IPMP group.
You cannot place two interfaces of different media types into an IPMP group.
For example, you cannot place an ATM interface in the same group
as an Ethernet interface.
Failure Detection and Failover
Failure detection is the process of detecting when an interface or the path from
an interface to an Internet layer device no longer works. IPMP provides systems
with the ability to detect when an interface has failed. IPMP detects the
following types of communication failures:
The transmit or receive path of the interface has failed.
The attachment of the interface to the IP link is down.
The port on the switch does not transmit or receive packets.
The physical interface in an IPMP group is not present at system boot.
After detecting a failure, IPMP begins failover. Failover is the automatic process of
switching the network access from a failed interface to a functioning physical interface
in the same group. Network access includes IPv4 unicast, multicast, and broadcast traffic,
as well as IPv6 unicast and multicast traffic. Failover can only occur when
you have configured more than one interface in the IPMP group. The failover
process ensures uninterrupted access to the network.
Repair Detection and Failback
Repair detection is the process of detecting when a NIC or the path from
a NIC to an Internet layer device starts operating correctly after a
failure. After detecting that a NIC has been repaired, IPMP performs failback, the
process of switching network access back to the repaired interface. Repair detection assumes
that you have enabled failbacks. See Detecting Physical Interface Repairs for more information.
Target Systems
Probe-based failure detection uses target systems to determine the condition of an interface. Each
target system must be attached to the same IP link as the members
of the IPMP group. The in.mpathd daemon on the local system sends
ICMP probe messages to each target system. The probe messages help to determine
the health of each interface in the IPMP group.
For more information about target system use in probe-based failure detection, refer to
Probe-Based Failure Detection.
Outbound Load Spreading
With IPMP configured, outbound network packets are spread across multiple NICs without affecting
the ordering of packets. This process is known as load spreading. As a
result of load spreading, higher throughput is achieved. Load spreading occurs only when
the network traffic is flowing to multiple destinations that use multiple connections.
Dynamic Reconfiguration
Dynamic reconfiguration (DR) is the ability to reconfigure a system while the system is
running, with little or no impact on existing operations. Not all Sun
platforms support DR. Some Sun platforms might only support DR of certain types
of hardware. On platforms that support DR of NIC's, IPMP can be
used to transparently fail over network access, providing uninterrupted network access to the
system.
For more information on how IPMP supports DR, refer to IPMP and Dynamic Reconfiguration.
