How to add EMC CLARiiON devices to a Solaris server using EMC PowerPath
Article ID: 100037730
Updated On:
Description
Description
This document attempts to explain the essential steps required to present new storage devices controlled by EMC PowerPath.
The following EMC CLARiiON devices will presented to a Solaris server using EMC PowerPath as the multipathing solution.
In this instance the EMC CLARiiON StorageGroup is "rdgv240sol13+14-bootdisks".
How to view the EMC CLARiiON devices associated with StorageGroup " rdgv240sol13+14-bootdisks "
Figure 1.0
Sample output
# /opt/Navisphere/bin/navicli -h 10.12.208.81 storagegroup -list -gname rdgv240sol13+14-bootdisks
Storage Group Name: rdgv240sol13+14-bootdisks
Storage Group UID: 60:CC:49:25:65:41:DF:11:B2:65:00:60:16:01:6D:D5
HBA/SP Pairs:
HBA UID SP Name SPPort
------- ------- ------
10:00:00:01:73:01:BB:EB:20:00:00:01:73:01:BB:EB SP A 1
10:00:00:01:73:01:BB:EB:20:00:00:01:73:01:BB:EB SP B 1
10:00:00:01:73:01:BB:EB:20:00:00:01:73:01:BB:EB SP A 0
10:00:00:01:73:01:BB:EB:20:00:00:01:73:01:BB:EB SP B 0
HLU/ALU Pairs:
HLU Number ALU Number
---------- ----------
5 78
6 79
7 3
Shareable: YES
Note: To avoid confusion it is critical that the EMC PowerPath pseudo device name corresponds to the Veritas Disk access name (DA) when the tpdmode is set to pseudo naming.
Stop eventsource daemon
Due to interoperability issues with EMC PowerPath and the Volume Manager/DMP stack, recommend that the "eventsource" (vxesd) be disabled prior to the presenting of new devices and the removal of devices.
5.1 SP1 will contain a "vxesd" product enhancement allowing EMC PowerPath and Volume Manager/DMP to safely interact, without the current interoperability issues.
# ps -ef | grep vxesd
root 775 1 0 12:22:24 ? 0:00 /sbin/vxesd
root 11898 10965 0 12:48:30 pts/1 0:00 grep vxesd
How to stop the "vxesd" daemon
# vxddladm stop eventsource
# pgrep vxesd
Lun provisioning pre-checks
1.] Ensure Solaris device tree is clean prior to the presentation of any new luns.
# /usr/sbin/devfsadm -Cvc disk
2.] If using the Solaris leadville stack, be sure to confirm that no paths (access points) are reported as "failing" or "unusable"
# cfgadm -alo show_FCP_dev | egrep 'failing|unusable'
Critical: If either state is reported, correct the "cfgadm" output prior to presenting the new devices.
3.] Ensure the EMC PowerPath device database is clean and up-to-date.
# yes | powermt restore dev=all
# /etc/powermt check dev=all class=clariion force
CLARiiON device(s) not found.
# /etc/powercf -q
Note: In this instance no EMC CLARiiON devices are presented to this host.
# /etc/powermt display class=clariion
CLARiiON logical device count=0
==============================================================================
----- Host Bus Adapters --------- ------ I/O Paths ----- ------ Stats ------
### HW Path Summary Total Dead IO/Sec Q-IOs Errors
==============================================================================
Note: Confirmation that DMP has not claimed any EMC CLARiiON related devices.
# vxdmpadm listenclosure all
ENCLR_NAME ENCLR_TYPE ENCLR_SNO STATUS ARRAY_TYPE LUN_COUNT
===================================================================================
disk Disk DISKS CONNECTED Disk 3
pp_emc1 PP_EMC 000290301414 CONNECTED A/A 5
pp_emc0 PP_EMC 000290300822 CONNECTED A/A 11
4.] Ensure VxVM is refreshed and the /etc/vx/disk.info file contents are updated prior to provisioning any new storage.
# vxdisk scandisks
"Data Corruption Protection Activated"
We introduced a new feature in 5.0 MP3 onwards called "Data Corruption Prevention Activity" (DCPA) feature which basically puts a bubble around each DMP device.
The Data Corruption Prevention Activity (DCPA) feature will assist with safeguarding your data.
The data corruption protection feature actually works to prevent corruption and is a proactive message, rather than a warning about specific corruption !
Using the bubble protector technology surrounding all the existing DMP nodenames, the product prevents other paths merging with it from another physical device. This dramatically helps to reduce the chances of data corruption, when the customer fails to perform the correct and essential steps surrounding the complex lun removal and storage provisioning process.
Prior to 5.0 MP3 it is very difficult to conclude if a step during the lun removal process failed or was accidentally missed.
Critical: If you encounter the following messages, please contact Support before preceding.
VxVM vxdisk ERROR V-5-1-14519 Data Corruption Protection Activated - User Corrective Action Needed
VxVM vxdisk INFO V-5-1-14521 To recover, first ensure that the OS device tree is up to date (requires OS specific commands).
Note: The DCPA feature was released in 5.0 MP3 onwards on all the UNIX flavors apart from HP-UX (where it was rebranded as 5.0.1).
# vxddladm assign names
Note: The "vxddladm assign names" command is essential in EMC PowerPath environments, to ensure the EMC PowerPath pseudo device name corresponds to a valid Veritas Disk access name (DA).
EMC CLARiiON Storage commands
# /opt/Navisphere/bin/navicli -h 10.12.208.81 storagegroup -connecthost -o -host rdgv240sol14 -gname rdgv240sol13+14-bootdisks
# /opt/Navisphere/bin/navicli -h 10.12.208.81 storagegroup -list -gname rdgv240sol13+14-bootdisks
Storage Group Name: rdgv240sol13+14-bootdisks
Storage Group UID: 60:CC:49:25:65:41:DF:11:B2:65:00:60:16:01:6D:D5
HBA/SP Pairs:
HBA UID SP Name SPPort
------- ------- ------
10:00:00:01:73:01:BB:EB:20:00:00:01:73:01:BB:EB SP A 1
10:00:00:01:73:01:BB:DF:20:00:00:01:73:01:BB:DF SP A 1
10:00:00:01:73:01:BB:EB:20:00:00:01:73:01:BB:EB SP B 1
10:00:00:01:73:01:BB:DF:20:00:00:01:73:01:BB:DF SP B 1
10:00:00:01:73:01:BB:EB:20:00:00:01:73:01:BB:EB SP A 0
10:00:00:01:73:01:BB:DF:20:00:00:01:73:01:BB:DF SP A 0
10:00:00:01:73:01:BB:EB:20:00:00:01:73:01:BB:EB SP B 0
10:00:00:01:73:01:BB:DF:20:00:00:01:73:01:BB:DF SP B 0
20:00:00:E0:8B:81:29:FA:21:00:00:E0:8B:81:29:FA SP A 0
20:00:00:E0:8B:81:29:FA:21:00:00:E0:8B:81:29:FA SP B 0
20:00:00:E0:8B:81:29:FA:21:00:00:E0:8B:81:29:FA SP A 1
20:00:00:E0:8B:81:29:FA:21:00:00:E0:8B:81:29:FA SP B 1
HLU/ALU Pairs:
HLU Number ALU Number
---------- ----------
5 78
6 79
7 3
Shareable: YES
The "ALU" values reported above can be correlated to the "powermt display dev=all" or "powermt display class=clariion" specifically once presented and EMC PowerPath has been updated.
If using the Solaris leadville stack, run "cfgadm -alo show_FCP_dev" to update the Solaris stack
If this is not performed, EMC PowerPath will not be able to discover the devices in some circumstances.
Sample output
# yes | powermt restore dev=all
# /etc/powermt check dev=all class=clariion force
CLARiiON device(s) not found.
# /etc/powercf -q
# /etc/powermt config
# /etc/powermt save
# /etc/powermt display class=clariion
CLARiiON logical device count=0
==============================================================================
----- Host Bus Adapters --------- ------ I/O Paths ----- ------ Stats ------
### HW Path Summary Total Dead IO/Sec Q-IOs Errors
==============================================================================
Note: As shown above, EMC PowerPath is unable to detect the NEW EMC CLARiiON devices as Solaris has not been updated.
Note: The Solaris device numbers will be c#t#d5,6 and 7 (HLU's) for the newly presented EMC CLARiiON devices.
# cfgadm -alo show_FCP_dev
Ap_Id Type Receptacle Occupant Condition
c1 fc-fabric connected configured unknown
c1::200000017301bbeb unknown connected unconfigured unknown
c1::210000e08b8129fa unknown connected unconfigured unknown
c1::500601603021c60e,0 disk connected configured unknown
c1::500601603021c60e,5 disk connected configured unknown
c1::500601603021c60e,6 disk connected configured unknown
c1::500601603021c60e,7 disk connected configured unknown
c1::500601613021c60e,0 disk connected configured unknown
c1::500601613021c60e,5 disk connected configured unknown
c1::500601613021c60e,6 disk connected configured unknown
c1::500601613021c60e,7 disk connected configured unknown
c1::500601683021c60e,0 disk connected configured unknown
c1::500601683021c60e,5 disk connected configured unknown
c1::500601683021c60e,6 disk connected configured unknown
c1::500601683021c60e,7 disk connected configured unknown
c1::500601693021c60e,0 disk connected configured unknown
c1::500601693021c60e,5 disk connected configured unknown
c1::500601693021c60e,6 disk connected configured unknown
c1::500601693021c60e,7 disk connected configured unknown
c1::5006048c5368e580,0 disk connected configured unknown
c1::5006048c5368e580,16 disk connected configured unknown
c1::5006048c5368e580,17 disk connected configured unknown
c1::5006048c5368e580,18 disk connected configured unknown
c1::5006048c5368e580,19 disk connected configured unknown
c1::5006048c5368e580,20 disk connected configured unknown
c1::5006048c5368e580,26 disk connected configured unknown
c1::5006048c5368e580,27 disk connected configured unknown
c1::5006048c5368e580,210 disk connected configured unknown
c1::5006048c5368e580,211 disk connected configured unknown
c1::5006048c5368e580,212 disk connected configured unknown
c1::5006048c5368e580,213 disk connected configured unknown
c1::5006048c5368e5a0,0 disk connected configured unknown
c1::5006048c5368e5a0,15 disk connected configured unknown
c1::5006048c5368e5a0,196 disk connected configured unknown
c1::5006048c5368e5a0,197 disk connected configured unknown
c1::5006048c5368e5a0,198 disk connected configured unknown
c1::5006048c5368e5a0,199 disk connected configured unknown
c1::5006048c5368e5a0,220 disk connected configured unknown
c1::5006048c5368e5a0,221 disk connected configured unknown
c1::5006048c5368e5a0,228 disk connected configured unknown
c1::5006048c53697980,0 disk connected configured unknown
c1::5006048c53697980,23 disk connected configured unknown
c1::5006048c53697980,32 disk connected configured unknown
c1::5006048c53697980,33 disk connected configured unknown
c1::5006048c53697980,34 disk connected configured unknown
c1::5006048c53697980,215 disk connected configured unknown
c5 fc-fabric connected configured unknown
c5::200000017301bbdf unknown connected unconfigured unknown
c5::200000017301bbeb unknown connected unconfigured unknown
c5::500601603021c60e,0 disk connected configured unknown
c5::500601603021c60e,5 disk connected configured unknown
c5::500601603021c60e,6 disk connected configured unknown
c5::500601603021c60e,7 disk connected configured unknown
c5::500601613021c60e,0 disk connected configured unknown
c5::500601613021c60e,5 disk connected configured unknown
c5::500601613021c60e,6 disk connected configured unknown
c5::500601613021c60e,7 disk connected configured unknown
c5::500601683021c60e,0 disk connected configured unknown
c5::500601683021c60e,5 disk connected configured unknown
c5::500601683021c60e,6 disk connected configured unknown
c5::500601683021c60e,7 disk connected configured unknown
c5::500601693021c60e,0 disk connected configured unknown
c5::500601693021c60e,5 disk connected configured unknown
c5::500601693021c60e,6 disk connected configured unknown
c5::500601693021c60e,7 disk connected configured unknown
Note: The new EMC CLARiiON device paths are shown above in bold-italics
EMC PowerPath related commands
As Solaris is now aware of the EMC CLARiiON devices, the user needs to manually update the EMC PowerPath database.
Note: The following two command "powermt restore and powermt check" are used to assist with EMC PowerPath recovery process of devices for which were not removed correctly in the past.
# yes | powermt restore dev=all
# /etc/powermt check dev=all class=clariion force
Note: The "powercf -q" command will display newly discovered devices and populate the EMC PowerPath database accordingly.
# /etc/powercf -q
Could not find config file entry for:
---------------------------------------
volume ID = 60060160e8341600
---------------------------------------
adding emcpower2
Could not find config file entry for:
---------------------------------------
volume ID = 60060160e8341600
---------------------------------------
adding emcpower4
Could not find config file entry for:
---------------------------------------
volume ID = 60060160e8341600
---------------------------------------
adding emcpower3
# /etc/powermt config
# /etc/powermt save
# powermt display class=clariion dev=all
Pseudo name=emcpower2a
CLARiiON ID=CK200053300424 [rdgv240sol13+14-bootdisks]
Logical device ID=60060160E83416001C81C15E575FDF11 [LUN 3] <<< ALU "3"
state=alive; policy=CLAROpt; priority=0; queued-IOs=0
Owner: default=SP B, current=SP B Array failover mode: 4
==============================================================================
---------------- Host --------------- - Stor - -- I/O Path - -- Stats ---
### HW Path I/O Paths Interf. Mode State Q-IOs Errors
==============================================================================
3073 pci@1e,600000/SUNW,jfca@2/fp@0,0 c1t500601603021C60Ed7s0 SP A0 active alive 0 0
3073 pci@1e,600000/SUNW,jfca@2/fp@0,0 c1t500601613021C60Ed7s0 SP A1 active alive 0 0
3073 pci@1e,600000/SUNW,jfca@2/fp@0,0 c1t500601683021C60Ed7s0 SP B0 active alive 0 0
3073 pci@1e,600000/SUNW,jfca@2/fp@0,0 c1t500601693021C60Ed7s0 SP B1 active alive 0 0
3074 pci@1e,600000/SUNW,qlc@3/fp@0,0 c5t500601603021C60Ed7s0 SP A0 active alive 0 0
3074 pci@1e,600000/SUNW,qlc@3/fp@0,0 c5t500601613021C60Ed7s0 SP A1 active alive 0 0
3074 pci@1e,600000/SUNW,qlc@3/fp@0,0 c5t500601683021C60Ed7s0 SP B0 active alive 0 0
3074 pci@1e,600000/SUNW,qlc@3/fp@0,0 c5t500601693021C60Ed7s0 SP B1 active alive 0 0
Pseudo name=emcpower4a
CLARiiON ID=CK200053300424 [rdgv240sol13+14-bootdisks]
Logical device ID=60060160E83416004A3CB095BD3BDE11 [LUN 78] <<< ALU "78"
state=alive; policy=CLAROpt; priority=0; queued-IOs=0
Owner: default=SP B, current=SP B Array failover mode: 4
==============================================================================
---------------- Host --------------- - Stor - -- I/O Path - -- Stats ---
### HW Path I/O Paths Interf. Mode State Q-IOs Errors
==============================================================================
3073 pci@1e,600000/SUNW,jfca@2/fp@0,0 c1t500601603021C60Ed5s0 SP A0 active alive 0 0
3073 pci@1e,600000/SUNW,jfca@2/fp@0,0 c1t500601613021C60Ed5s0 SP A1 active alive 0 0
3073 pci@1e,600000/SUNW,jfca@2/fp@0,0 c1t500601683021C60Ed5s0 SP B0 active alive 0 0
3073 pci@1e,600000/SUNW,jfca@2/fp@0,0 c1t500601693021C60Ed5s0 SP B1 active alive 0 0
3074 pci@1e,600000/SUNW,qlc@3/fp@0,0 c5t500601603021C60Ed5s0 SP A0 active alive 0 0
3074 pci@1e,600000/SUNW,qlc@3/fp@0,0 c5t500601613021C60Ed5s0 SP A1 active alive 0 0
3074 pci@1e,600000/SUNW,qlc@3/fp@0,0 c5t500601683021C60Ed5s0 SP B0 active alive 0 0
3074 pci@1e,600000/SUNW,qlc@3/fp@0,0 c5t500601693021C60Ed5s0 SP B1 active alive 0 0
Pseudo name=emcpower3a
CLARiiON ID=CK200053300424 [rdgv240sol13+14-bootdisks]
Logical device ID=60060160E8341600F2BC89206340DE11 [LUN 79] <<< ALU "79"
state=alive; policy=CLAROpt; priority=0; queued-IOs=0
Owner: default=SP A, current=SP A Array failover mode: 4
==============================================================================
---------------- Host --------------- - Stor - -- I/O Path - -- Stats ---
### HW Path I/O Paths Interf. Mode State Q-IOs Errors
==============================================================================
3073 pci@1e,600000/SUNW,jfca@2/fp@0,0 c1t500601603021C60Ed6s0 SP A0 active alive 0 0
3073 pci@1e,600000/SUNW,jfca@2/fp@0,0 c1t500601613021C60Ed6s0 SP A1 active alive 0 0
3073 pci@1e,600000/SUNW,jfca@2/fp@0,0 c1t500601683021C60Ed6s0 SP B0 active alive 0 0
3073 pci@1e,600000/SUNW,jfca@2/fp@0,0 c1t500601693021C60Ed6s0 SP B1 active alive 0 0
3074 pci@1e,600000/SUNW,qlc@3/fp@0,0 c5t500601603021C60Ed6s0 SP A0 active alive 0 0
3074 pci@1e,600000/SUNW,qlc@3/fp@0,0 c5t500601613021C60Ed6s0 SP A1 active alive 0 0
3074 pci@1e,600000/SUNW,qlc@3/fp@0,0 c5t500601683021C60Ed6s0 SP B0 active alive 0 0
3074 pci@1e,600000/SUNW,qlc@3/fp@0,0 c5t500601693021C60Ed6s0 SP B1 active alive 0 0
Summary
EMC CLARiiON ALU reference "ALU 3" refers to emcpower2
EMC CLARiiON ALU reference "ALU 78" refers to emcpower4
EMC CLARiiON ALU reference "ALU 79" refers to emcpower3
TPD (Third Party Driver)
As of 5.0 MP3 onwards, VxVM/DMP will report the EMC PowerPath (TPD) EMC CLARiiON controlled devices under the PP_EMC_CLARiiON enclosure type.
# vxdmpadm listenclosure all
ENCLR_NAME ENCLR_TYPE ENCLR_SNO STATUS ARRAY_TYPE LUN_COUNT
===================================================================================
disk Disk DISKS CONNECTED Disk 3
pp_emc1 PP_EMC 000290301414 CONNECTED A/A 5
pp_emc_clariion0 PP_EMC_CLARiiON CK200053300424 CONNECTED A/A 3
pp_emc0 PP_EMC 000290300822 CONNECTED A/A 11
Note: As of 5.0 MP3 onwards, DMP will now report a LUN count per enclosure. In this instance there are "3" disks associated with the EMC CLARiiON enclosure.
# powermt display class=clariion
CLARiiON logical device count=3
==============================================================================
----- Host Bus Adapters --------- ------ I/O Paths ----- ------ Stats ------
### HW Path Summary Total Dead IO/Sec Q-IOs Errors
==============================================================================
3073 pci@1e,600000/SUNW,jfca@2/fp@0,0 optimal 12 0 - 0 0
3074 pci@1e,600000/SUNW,qlc@3/fp@0,0 optimal 12 0 - 0 0
# vxdmpadm getportids enclosure=pp_emc_clariion0
NAME ENCLR-NAME ARRAY-PORT-ID pWWN
===================================================================================
emcpower4c pp_emc_clariion0 - -
emcpower2c pp_emc_clariion0 - -
emcpower3 pp_emc_clariion0 - -
Note: The array-port-id and pWWN content is suppressed due to the EMC PowerPath installation.
Volume Manager disk details
As of 5.0 MP3 onwards, the /etc/vx/disk.info file contents and the "vxdisk list" output can be updated using the "vxddladm assign names" command.
# vxdisk scandisks
# vxddladm assign names
# vxdisk -e -o alldgs list
DEVICE TYPE DISK GROUP STATUS OS_NATIVE_NAME ATTR
disk_0 auto - (51dg) online c2t1d0s2 -
disk_1 auto - (50MP1RP5dg) online c2t2d0s2 -
disk_2 auto c2t0d0 rootdgrp4 online c2t0d0s2 -
emcpower0s2 auto - - online emcpower0c -
emcpower1s2 auto - (datadg) online emcpower1c -
emcpower2s2 auto - - error emcpower2c - <<<< ALU "3"
emcpower3 auto - - error emcpower3 - <<<< ALU "79"
emcpower4s2 auto - - online emcpower4c - <<<< ALU "78"
emcpower5s2 auto - - error emcpower5c -
emcpower6s2 auto - (terrydg) online emcpower6c -
emcpower7s2 auto - - online emcpower7c -
emcpower8s2 auto - (newdg) online emcpower8c -
emcpower9s2 auto - (datadg) online emcpower9c -
emcpower10s2 auto - (datadg) online emcpower10c -
emcpower11s2 auto - (newdg) online emcpower11c -
emcpower12s2 auto - - online emcpower12c -
emcpower13s2 auto - (datadg) online emcpower13c -
emcpower14s2 auto - - online emcpower14c -
emcpower15s2 auto - - online emcpower15c -
emcpower16s2 auto - (datadg) online emcpower16c -
emcpower17s2 auto - (testdg) online emcpower17c -
emcpower18s2 auto - - online emcpower18c -
Figure 2.0
Note: To avoid confusion it is critical that the EMC PowerPath pseudo device name corresponds to the Veritas Disk access name (DA) when the tpdmode is set to pseudo naming.
EMC CLARiiON ALU references to Veritas Disk Access Names
EMC CLARiiON ALU reference "ALU 3" refers to emcpower2, Veritas disk access name emcpower2s2
EMC CLARiiON ALU reference "ALU 78" refers to emcpower4, Veritas disk access name emcpower4s2
EMC CLARiiON ALU reference "ALU 79" refers to emcpower2, Veritas disk access name emcpower3
Note: emcpower3 happens to be an EFI lun, hence the reason why no "s2" extension to appended to the Veritas Disk Access (DA) Name.
Issue/Introduction
