Description

This document attempts to explain the essential steps required to cleanup stale device handles for EMC DMX luns presented to a Solaris server controlled by EMC PowerPath.

 

The following EMC DMX SYMDEV "00E9" device was masked from a Solaris server where EMC PowerPath is used as the multipathing solution.



In this instance the EMC Symmetrix ID(SID) is "000290300822" (SID 822 for short) connected to wwn "200000017301bbeb"

Prior to the lun removal process two disks were controlled by EMC PowerPath.


#  /etc/powermt display class=symm
Symmetrix logical device count=2
==============================================================================
----- Host Bus Adapters ---------  ------ I/O Paths -----  ------ Stats ------
###  HW Path                       Summary   Total   Dead  IO/Sec Q-IOs Errors
==============================================================================
3073 pci@1e,600000/SUNW,jfca@2/fp@0,0  optimal       2      0       -     0      0



The following LUN removal steps were performed:


1.] vxdisk rm emcpower1s2
2.] powermt remove dev=emcpower1c

3.] powercf -q

Could not validate the entry:
---------------------------------------
emcpower1: user ID = 12500000008
---------------------------------------

removing emcpower1


4.] symmaskdb -sid 822 list devs -wwn 200000017301bbeb

Symmetrix ID        : 000290300822

Originator Port wwn : 200000017301bbeb
User-generated Name : /200000017301bbeb

Sym Dev                                               LUN
Name    Dir:P  Physical Device Name     VBUS  TID  SYMM HOST  Attr  Cap(MB)
------  -----  -----------------------  ----  ---  ---- ----  ----  -------
00E9     1C:0  rdmp/emcpower1s2            0    0    12    0           2158


5.] /opt/emc/SYMCLI/V6.5.1/bin/symmask -sid 822 -wwn 200000017301bbeb remove devs 00e9 -dir 1c -p 0


6.] symmask -sid 822 refresh -noprompt

Symmetrix FA directors updated with contents of SymMask Database 000290300822


7.] symcfg -sid 822 discover

Attempting discovery of Symmetrix 000290300822

This operation may take up to a few minutes. Please be patient...


In this instance, the access path for the removed LUN has been marked as "failing", instead of "unusable".


8.] cfgadm -alo show_FCP_dev
Ap_Id                          Type         Receptacle   Occupant     Condition
c1                             fc-fabric    connected    configured   unknown
c1::200000017301bbdf           unknown      connected    unconfigured unknown
c1::210000e08b8129fa           unknown      connected    unconfigured unknown
c1::500601603021c60e,0         disk         connected    configured   unknown
c1::500601613021c60e,0         disk         connected    configured   unknown
c1::500601683021c60e,0         disk         connected    configured   unknown
c1::500601693021c60e,0         disk         connected    configured   unknown
c1::5006048c5368e580,0         disk         connected    configured   unknown
c1::5006048c5368e580,18        disk         connected    configured   failing
c1::5006048c5368e58f,0         disk         connected    configured   unknown
c1::5006048c5368e5a0,0         disk         connected    configured   unknown
c1::5006048c5368e5af,0         disk         connected    configured   unknown
c1::5006048c53697980,0         disk         connected    configured   unknown
c1::5006048c53697980,215       disk         connected    configured   unknown


Steps to convert the cfgadm "failing" state to "unusable"

 

Establish the c#t#d# device path for the cfgadm "failing" LUN.


1.] ls -al /dev/rdsk/*d18s2
lrwxrwxrwx   1 root     root          77 Apr 19  2010 /dev/rdsk/c1t5006048C5368E580d18s2 -> ../../devices/pci@1e,600000/SUNW,jfca@2/fp@0,0/ssd@w5006048c5368e58
0,12:c,raw


Convert the cfgadm "failing" state to "unusable".

2..]  luxadm -e offline /dev/rdsk/c1t5006048C5368E580d18s2
devctl: I/O error


Confirm that VxVM does not have a Veritas disk access name (DA) for the removed LUN.


3.] vxdisk list
DEVICE       TYPE            DISK         GROUP        STATUS
disk_0       auto:none       -            -            online invalid
emcpower0s2  auto:cdsdisk    -            -            online
 

Refresh VxVM disk details.


4.] vxdisk scandisks


Confirm the Veritas disk access name (DA) for the removed LUN does not appear.


5.] vxdisk list
DEVICE       TYPE            DISK         GROUP        STATUS
disk_0       auto:none       -            -            online invalid
emcpower0s2  auto:cdsdisk    -            -            online



Confirm that EMC PowerPath does not report any stale (dead) paths.


6.] powermt display
Symmetrix logical device count=2
CLARiiON logical device count=0
Hitachi logical device count=0
Invista logical device count=0
HP xp logical device count=0
Ess logical device count=0
HP HSx logical device count=0
==============================================================================
----- Host Bus Adapters ---------  ------ I/O Paths -----  ------ Stats ------
###  HW Path                       Summary   Total   Dead  IO/Sec Q-IOs Errors
==============================================================================
3073 pci@1e,600000/SUNW,jfca@2/fp@0,0  degraded      2      1       -     0      1


In this instance, EMC SYMDEV "00E9" has a single "dead" path.


# powermt display dev=all
Symmetrix ID=000290300822
Logical device ID=00E9
state=alive; policy=SymmOpt; priority=0; queued-IOs=0
==============================================================================
---------------- Host ---------------   - Stor -   -- I/O Path -  -- Stats ---
###  HW Path                I/O Paths    Interf.   Mode    State  Q-IOs Errors
==============================================================================
3073 pci@1e,600000/SUNW,jfca@2/fp@0,0 c1t5006048C5368E580d18s0 FA  1cA   active  dead       0      1

Pseudo name=emcpower0a
Symmetrix ID=000290301414
Logical device ID=00C6
state=alive; policy=SymmOpt; priority=0; queued-IOs=0
==============================================================================
---------------- Host ---------------   - Stor -   -- I/O Path -  -- Stats ---
###  HW Path                I/O Paths    Interf.   Mode    State  Q-IOs Errors
==============================================================================
3073 pci@1e,600000/SUNW,jfca@2/fp@0,0 c1t5006048C53697980d215s0 FA  1cA   active  alive      0      0



Restore "dead" paths.


7.]  yes | powermt restore dev=all
Warning: Symmetrix device path c1t5006048C5368E580d18s0 is currently dead.

#  powermt display dev=all
Symmetrix ID=000290300822
Logical device ID=00E9
state=alive; policy=SymmOpt; priority=0; queued-IOs=0
==============================================================================
---------------- Host ---------------   - Stor -   -- I/O Path -  -- Stats ---
###  HW Path                I/O Paths    Interf.   Mode    State  Q-IOs Errors
==============================================================================
3073 pci@1e,600000/SUNW,jfca@2/fp@0,0 c1t5006048C5368E580d18s0 FA  1cA   active  dead       0      1

Pseudo name=emcpower0a
Symmetrix ID=000290301414
Logical device ID=00C6
state=alive; policy=SymmOpt; priority=0; queued-IOs=0
==============================================================================
---------------- Host ---------------   - Stor -   -- I/O Path -  -- Stats ---
###  HW Path                I/O Paths    Interf.   Mode    State  Q-IOs Errors
==============================================================================
3073 pci@1e,600000/SUNW,jfca@2/fp@0,0 c1t5006048C53697980d215s0 FA  1cA   active  alive      0      0



# /etc/powermt check dev=all class=symm force
Warning: Symmetrix device path c1t5006048C5368E580d18s0 is currently dead.

# powermt display dev=all
Pseudo name=emcpower0a
Symmetrix ID=000290301414
Logical device ID=00C6
state=alive; policy=SymmOpt; priority=0; queued-IOs=0
==============================================================================
---------------- Host ---------------   - Stor -   -- I/O Path -  -- Stats ---
###  HW Path                I/O Paths    Interf.   Mode    State  Q-IOs Errors
==============================================================================
3073 pci@1e,600000/SUNW,jfca@2/fp@0,0 c1t5006048C53697980d215s0 FA  1cA   active  alive      0      0


Convert the cfgadm "failing" path to an "unusable" state using luxadm.
 

8.]  luxadm -e offline /dev/rdsk/c1t5006048C5368E580d18s2


# cfgadm -alo show_FCP_dev
Ap_Id                          Type         Receptacle   Occupant     Condition
c1                             fc-fabric    connected    configured   unknown
c1::200000017301bbdf           unknown      connected    unconfigured unknown
c1::210000e08b8129fa           unknown      connected    unconfigured unknown
c1::500601603021c60e,0         disk         connected    configured   unknown
c1::500601613021c60e,0         disk         connected    configured   unknown
c1::500601683021c60e,0         disk         connected    configured   unknown
c1::500601693021c60e,0         disk         connected    configured   unknown
c1::5006048c5368e580,0         disk         connected    configured   unknown
c1::5006048c5368e580,18        disk         connected    configured   unusable
c1::5006048c5368e58f,0         disk         connected    configured   unknown
c1::5006048c5368e5a0,0         disk         connected    configured   unknown
c1::5006048c5368e5af,0         disk         connected    configured   unknown
c1::5006048c53697980,0         disk         connected    configured   unknown
c1::5006048c53697980,215       disk         connected    configured   unknown


Unconfigure the unusable path.


9.]  cfgadm -o unusable_FCP_dev -c unconfigure c1::5006048c5368e580


# cfgadm -alo show_FCP_dev
Ap_Id                          Type         Receptacle   Occupant     Condition
c1                             fc-fabric    connected    configured   unknown
c1::200000017301bbdf           unknown      connected    unconfigured unknown
c1::210000e08b8129fa           unknown      connected    unconfigured unknown
c1::500601603021c60e,0         disk         connected    configured   unknown
c1::500601613021c60e,0         disk         connected    configured   unknown
c1::500601683021c60e,0         disk         connected    configured   unknown
c1::500601693021c60e,0         disk         connected    configured   unknown
c1::5006048c5368e580,0         disk         connected    configured   unknown
c1::5006048c5368e58f,0         disk         connected    configured   unknown
c1::5006048c5368e5a0,0         disk         connected    configured   unknown
c1::5006048c5368e5af,0         disk         connected    configured   unknown
c1::5006048c53697980,0         disk         connected    configured   unknown
c1::5006048c53697980,215       disk         connected    configured   unknown



Refresh the VxVM disk details.


10.] vxdisk scandisks



How to refresh the /etc/vx/disk.info file contents


# more /etc/vx/disk.info
SEAGATE%5FST373307LSUN72G%5FDISKS%5F30353033423959314D320000 c2t1d0 0x4800000 0x2 disk_0 Disk DISKS
EMC%5FSYMMETRIX%5F000290301414%5F140C6330 emcpower0 0x4800008 0x4 emcpower0 PP_EMC 000290301414
EMC%5FSYMMETRIX%5F000290300822%5F60060480000290300822533030304539 emcpower1 0xffffffff 0x4 emcpower1 PP_EMC 000290300822

# vxddladm assign names

# more /etc/vx/disk.info
SEAGATE%5FST373307LSUN72G%5FDISKS%5F30353033423959314D320000 c2t1d0 0x4800000 0x2 disk_0 Disk DISKS
EMC%5FSYMMETRIX%5F000290301414%5F140C6330 emcpower0 0x4800008 0x4 emcpower0 PP_EMC 000290301414

# vxdmpadm listenclosure all
ENCLR_NAME        ENCLR_TYPE     ENCLR_SNO      STATUS       ARRAY_TYPE     LUN_COUNT
=======================================================================================
disk              Disk           DISKS                CONNECTED    Disk        1
pp_emc0           PP_EMC         000290301414         CONNECTED    A/A         1


How to refresh the /etc/vx/array.info file contents


# more /etc/vx/array.info
DISKS
disk
0
Disk
CK200053300424
emc_clariion0
0
EMC_CLARiiON
000290300822
emc0
0
EMC
000290301414
pp_emc0
0
PP_EMC
000290300822
pp_emc1
0
PP_EMC

As of VxVM 5.1 SP1, the /etc/vx/array.info file contents can be refreshed without the need to recycle vxconfigd.


# vxddladm -c assign names


Revised /etc/vx/array.info file details


# more /etc/vx/array.info
DISKS
disk
0
Disk
000290301414
pp_emc0
0
PP_EMC