Configuration files
For Solaris 10 and earlier/kernel/drv/fp.conf | Fiber channel port configuration file (For Fiber Channel Devices) |
/kernel/drv/mpt.conf | mpt scsi/SAS HBA controller driver configuration file (For SAS Devices) |
/kernel/drv/scsi_vhci.conf | MPXIO operation configuration file (Used to define loadbalancing and failback modes and configure 3rd party devices) |
/kernel/drv/iscsi_conf | iSCSI MPxIO configuration file (For iSCSI devices) |
In case of Solaris 11, files in the directory /kernel/drv/ and /etc/driver/drv are the sample default configuration files delivered with OS (not used by MPxIO). Do not edit these files. The location of the configuration files is /etc/driver/drv/. The 4 configuration files used in case of Solaris 11 are the same, performing the same task. Although the syntax may be different.
/etc/driver/drv/fp.conf
/etc/driver/drv/mpt.conf
/etc/driver/drv/scsi_vhci.conf
/etc/driver/drv/iscsi_conf
Enabling/Disabling MPxIO
The syntax of the command stmsboot used to enable/disable MPxIO is :stmsboot [-D fp|mpt] [-d | -e | -u | -L | -l controller_number] where: -D fp|mpt -> to select the protocol where you want to enable/disable mpxio -e -> to enable STMS on all HBA -d -> to disable STMS on all HBA -u -> to update vfstable and dump config after manual file configuration -l or -L unchanged - see the manpage of stms for a full description.
# stmsboot -e | To enable MPxIO or all devices (globally) |
# stmsboot -D fp -e | To enable MPxIO for fiber channel devices |
# stmsboot -d | To disable MPxIO for all devices (globally) |
How to confirm MPxIO is enabled
To make sure that MPxIO is enabled for a LUN, simply run the format command and check for the /scsi_vhci in the logical path of the LUN.# format 4. c6t60060E800561CF00000061CF00000002d0 /scsi_vhci/ssd@g60060e800561cf00000061cf00000002
Solaris 7,8,9
# egrep "^mpxio-disable=" /kernel/drv/*.conf
/kernel/drv/scsi_vhci.conf:mpxio-disable="no";
# egrep "^mpxio-disable=" /kernel/drv/*.conf
/kernel/drv/fp.conf:mpxio-disable="no";
# egrep "^mpxio-disable=" /etc/driver/drv/*.conf
/etc/driver/drv/fp.conf:mpxio-disable="no";
For Solaris 10 and 11, MPxIO is enabled by default on X86 platforms.
Verifying that the scsi_vhci module is loaded
scsi_vhci is the virtual host controller interconnect driver used to manage path of storage LUN. If you find that the scsi_vhci module is not loaded, then most likely MPxIO is not enabled.Solaris[TM] 7,8,9,10,11 # modinfo -c |grep scsi_vhci 21 1 scsi_vhci LOADED/INSTALLED
Verifying Health of paths
We can check the multiple paths to the storage array LUNs by using mpathadm or luxadm command. The various states for the paths are :ONLINE/primary ONLINE/secondary STANDBY/secondary OFFLINE/* UNAVAILABLE/*
luxadm (supported on FC only) [Works on Solaris 7,8,9,10,11]
# mpathadm list LU /dev/rdsk/c9t60080E50001C0300000006464D8C9044d0s2 Total Path Count: 2 Operational Path Count: 2
# mpathadm show LU /dev/rdsk/c9t60080E50001C0300000006464D8C9044d0s2 Logical Unit: /dev/rdsk/c9t60080E50001C0300000006464D8C9044d0s2 mpath-support: libmpscsi_vhci.so Vendor: SUN Product: LCSM100_F Revision: 0777 Name Type: unknown type Name: 60080e50001c0300000006464d8c9044 Asymmetric: yes Current Load Balance: round-robin Logical Unit Group ID: NA Auto Failback: on Auto Probing: NA Paths: Initiator Port Name: 210000e08b07922c Target Port Name: 203d0080e51c2cc4 Override Path: NA Path State: OK Disabled: no Initiator Port Name: 210000e08b07f32c Target Port Name: 203e0080e51c2cc4 Override Path: NA Path State: OK Disabled: no .........
# luxadm probe Found Fibre Channel device(s): ...... Logical Path:/dev/rdsk/c6t60060E800561CF00000061CF00000002d0s2 Node WWN:203e0080e51c2cc4 Device Type:Disk device ......
# luxadm display 203e0080e51c2cc4 DEVICE PROPERTIES for disk: 203e0080e51c2cc4 Vendor: SUN Product ID: LCSM100_F ------------> this is a 2540 or 2540-M2 array Revision: 0777 Serial Num: SV04720931 Unformatted capacity: 5120.000 MBytes Write Cache: Enabled Read Cache: Enabled Minimum prefetch: 0x3 Maximum prefetch: 0x0 Device Type: Disk device Path(s): /dev/rdsk/c9t60080E50001C0300000006464D8C9044d0s2 /devices/scsi_vhci/ssd@g60080e50001c0300000006464d8c9044:c,raw Controller /devices/pci@1d,700000/SUNW,qlc@1/fp@0,0 Device Address 203d0080e51c2cc4,0 Host controller port WWN 210000e08b07922c Class primary State ONLINE Controller /devices/pci@1c,600000/SUNW,qlc@1/fp@0,0 Device Address 203e0080e51c2cc4,0 Host controller port WWN 210000e08b07f32c Class secondary State ONLINE
Load Balancing policies
MPxIO also provides the load balancing capability to the I/O. There are basically 3 options to configure load balancing :Round Robin | Selects all paths in a round robin way. |
Logical Block | I/Os within the range specified by range_size parameter would go on the same path. Improves performance on some storage arrays which have per path caching of data. |
None | No load balancing – only one path is used. |
load-balance="round-robin";
Auto-failback policy
Auto-failback policy enables the re-enable the failed path after it has recovered from the failure. By default the auto-failback is enabled. There are 2 ways to enable the auto-failback :Non-persistent way
By this method, the auto-failback option does not persist across reboots.
# mpathadm modify mpath-support --autofailback on libmpscsi_vhci.so # mpathadm modify mpath-support -a on libmpscsi_vhci.so
For the changes to persist across reboots set the auto-failback parameter in the configuration file /kernel/drv/scsi_vhci.conf for solaris 10 and earlier or in the /etc/driver/drv/scsi_vhci.conf file for solaris 11. This requires a reconfiguration reboot.
auto-failback="enable";
To confirm that the auto-failback is enabled use :
# mpathadm show mpath-support libmpscsi_vhci.so | grep -i failback Supported Auto Failback Config: 1 Auto Failback: on <--- 0="" current="" failback="" max="" polling="" pre="" rate="">--->
========================================================
bash-3.2# egrep "^mpxio-disable=" /kernel/drv/*.conf
/kernel/drv/fp.conf:mpxio-disable="yes"; => change to no/kernel/drv/iscsi.conf:mpxio-disable="no";
/kernel/drv/mpt.conf:mpxio-disable="yes"; => change to no
/kernel/drv/mpt_sas.conf:mpxio-disable="no";
bash-3.2# mpathadm list lu
/dev/rdsk/c6t20000018620E6681d0s2
Total Path Count: 1
Operational Path Count: 1
/dev/rdsk/c6t20000018620E42AFd0s2
Total Path Count: 1
Operational Path Count: 1
/dev/rdsk/c6t20000018620E400Bd0s2
Total Path Count: 1
Operational Path Count: 1
/dev/rdsk/c6t20000018620E4155d0s2
Total Path Count: 1
Operational Path Count: 1
/scsi_vhci/ses@g5080020000284028
Total Path Count: 1
Operational Path Count: 1
/dev/rdsk/c6t600A0B800018B3CA000052F6534DB0C5d0s2
Total Path Count: 2
Operational Path Count: 2
/dev/rdsk/c6t600A0B800018B3CA000052EF534DB005d0s2
Total Path Count: 2
Operational Path Count: 2
bash-3.2# fcinfo hba-port
HBA Port WWN: 210000144f415c2a
OS Device Name: /dev/cfg/c1
Manufacturer: QLogic Corp.
Model: 2200
Firmware Version: 02.01.145
FCode/BIOS Version: ISP2200 FC-AL Host Adapter Driver: 1.15 04/03/22
Serial Number: not available
Driver Name: qlc
Driver Version: 20120717-4.01
Type: L-port
State: online
Supported Speeds: 1Gb
Current Speed: 1Gb
Node WWN: 200000144f415c2a
HBA Port WWN: 10000000c9c25df0
OS Device Name: /dev/cfg/c4
Manufacturer: Emulex
Model: LP11000-S
Firmware Version: 2.82a4 (B3D2.82A4)
FCode/BIOS Version: Boot:5.02a1 Fcode:1.50a9
Serial Number: 0999BT0-1121000JC2
Driver Name: emlxs
Driver Version: 2.80.8.0 (2012.09.17.15.45)
Type: L-port
State: online
Supported Speeds: 1Gb 2Gb 4Gb
Current Speed: 2Gb
Node WWN: 20000000c9c25df0
HBA Port WWN: 10000000c9c25d6f
OS Device Name: /dev/cfg/c5
Manufacturer: Emulex
Model: LP11000-S
Firmware Version: 2.82a4 (B3D2.82A4)
FCode/BIOS Version: Boot:5.02a1 Fcode:1.50a9
Serial Number: 0999BT0-1121000J8Z
Driver Name: emlxs
Driver Version: 2.80.8.0 (2012.09.17.15.45)
Type: L-port
State: online
Supported Speeds: 1Gb 2Gb 4Gb
Current Speed: 2Gb
Node WWN: 20000000c9c25d6f
bash-3.2# mpathadm list mpath-support
mpath-support: libmpscsi_vhci.so
bash-3.2# mpathadm list initiator-port
Initiator Port: 210000144f415c2a
Initiator Port: 10000000c9c25df0
Initiator Port: 10000000c9c25d6f
Initiator Port: iqn.1986-03.com.sun:01:00144f415c2a.5354dd03,4000002a00ff
bash-3.2# mpathadm show initiator-port 10000000c9c25df0
Initiator Port: 10000000c9c25df0
Transport Type: Fibre Channel
OS Device File: /devices/pci@9,600000/SUNW,emlxs@1/fp@0,0
bash-3.2# mpathadm show initiator-port 10000000c9c25d6f
Initiator Port: 10000000c9c25d6f
Transport Type: Fibre Channel
OS Device File: /devices/pci@9,600000/SUNW,emlxs@2/fp@0,0
bash-3.2# mpathadm show lu /dev/rdsk/c6t600A0B800018B3CA000052F6534DB0C5d0s2
Logical Unit: /dev/rdsk/c6t600A0B800018B3CA000052F6534DB0C5d0s2
mpath-support: libmpscsi_vhci.so
Vendor: SUN
Product: CSM100_R_FC
Revision: 0660
Name Type: unknown type
Name: 600a0b800018b3ca000052f6534db0c5
Asymmetric: yes
Current Load Balance: round-robin
Logical Unit Group ID: NA
Auto Failback: on
Auto Probing: NA
Paths:
Initiator Port Name: 10000000c9c25df0
Target Port Name: 200400a0b81ffbef
Override Path: NA
Path State: OK
Disabled: no
Initiator Port Name: 10000000c9c25d6f
Target Port Name: 200500a0b81ffbef
Override Path: NA
Path State: OK
Disabled: no
Target Port Groups:
ID: 6
Explicit Failover: yes
Access State: standby
Target Ports:
Name: 200400a0b81ffbef
Relative ID: 0
ID: 7
Explicit Failover: yes
Access State: active
Target Ports:
Name: 200500a0b81ffbef
Relative ID: 0
=================================
Test multipath
bash-3.2# fcinfo hba-port
HBA Port WWN: 10000000c9c25df0
OS Device Name: /dev/cfg/c4
Manufacturer: Emulex
Model: LP11000-S
Firmware Version: 2.82a4 (B3D2.82A4)
FCode/BIOS Version: Boot:5.02a1 Fcode:1.50a9
Serial Number: 0999BT0-1121000JC2
Driver Name: emlxs
Driver Version: 2.80.8.0 (2012.09.17.15.45)
Type: L-port
State: online
Supported Speeds: 1Gb 2Gb 4Gb
Current Speed: 2Gb
Node WWN: 20000000c9c25df0
HBA Port WWN: 10000000c9c25d6f
OS Device Name: /dev/cfg/c5
Manufacturer: Emulex
Model: LP11000-S
Firmware Version: 2.82a4 (B3D2.82A4)
FCode/BIOS Version: Boot:5.02a1 Fcode:1.50a9
Serial Number: 0999BT0-1121000J8Z
Driver Name: emlxs
Driver Version: 2.80.8.0 (2012.09.17.15.45)
Type: L-port
State: online
Supported Speeds: 1Gb 2Gb 4Gb
Current Speed: 2Gb
Node WWN: 20000000c9c25d6f
bash-3.2# luxadm -e forcelip /dev/cfg/c4
$ luxadm -e forcelip /dev/cfg/c4
bash-3.2# tail -f /var/adm/messages
[ID 349649 kern.info] [ 5.06C6]emlxs1: NOTICE: 730: Link reset. (Resetting link...)
[ID 349649 kern.info] [ 5.03DD]emlxs1: NOTICE: 710: Link down.
[ID 349649 kern.info] [ 5.0715]emlxs1: NOTICE: 730: Link reset.
[ID 349649 kern.info] [ 5.061B]emlxs1: NOTICE: 720: Link up. (2Gb, loop, initiator)
[ID 349649 kern.info] [ 5.06C6]emlxs0: NOTICE: 730: Link reset. (Resetting link...)
[ID 349649 kern.info] [ 5.03DD]emlxs0: NOTICE: 710: Link down.
[ID 349649 kern.info] [ 5.0715]emlxs0: NOTICE: 730: Link reset.
[ID 349649 kern.info] [ 5.061B]emlxs0: NOTICE: 720: Link up. (2Gb, loop, initiator)