How to remove EMC CLARiiON devices from a Solaris server using EMC PowerPath
Article ID: 100037729
Updated On:
Description
Description
This document attempts to explain the essential steps required to remove existing storage devices controlled by EMC PowerPath.
The following EMC CLARiiON devices are presented to a Solaris server using EMC PowerPath as the multipathing solution.
In this instance the EMC CLARiiON StorageGroup is "rdgv240sol13+14-bootdisks".
Figure 1.0
How to view the EMC CLARiiON devices associated with StorageGroup " rdgv240sol13+14-bootdisks "
Sample output
# navicli -h 10.12.208.81 storagegroup -list -gname rdgv240sol13+14-bootdisks
Storage Group Name: rdgv240sol13+14-bootdisks
Storage Group UID: 60:CC:49:25:65:41:DF:11:B2:65:00:60:16:01:6D:D5
HBA/SP Pairs:
HBA UID SP Name SPPort
------- ------- ------
10:00:00:01:73:01:BB:EB:20:00:00:01:73:01:BB:EB SP A 1
10:00:00:01:73:01:BB:DF:20:00:00:01:73:01:BB:DF SP A 1
10:00:00:01:73:01:BB:EB:20:00:00:01:73:01:BB:EB SP B 1
10:00:00:01:73:01:BB:DF:20:00:00:01:73:01:BB:DF SP B 1
10:00:00:01:73:01:BB:EB:20:00:00:01:73:01:BB:EB SP A 0
10:00:00:01:73:01:BB:DF:20:00:00:01:73:01:BB:DF SP A 0
10:00:00:01:73:01:BB:EB:20:00:00:01:73:01:BB:EB SP B 0
10:00:00:01:73:01:BB:DF:20:00:00:01:73:01:BB:DF SP B 0
20:00:00:E0:8B:81:29:FA:21:00:00:E0:8B:81:29:FA SP A 0
20:00:00:E0:8B:81:29:FA:21:00:00:E0:8B:81:29:FA SP B 0
20:00:00:E0:8B:81:29:FA:21:00:00:E0:8B:81:29:FA SP A 1
20:00:00:E0:8B:81:29:FA:21:00:00:E0:8B:81:29:FA SP B 1
HLU/ALU Pairs:
HLU Number ALU Number
---------- ----------
5 78
6 79
7 3
Shareable: YES
The "ALU" values reported above can be correlated to the "powermt display dev=all" or "powermt display class=clariion" specifically.
# powermt display class=clariion dev=all
Pseudo name=emcpower3a
CLARiiON ID=CK200053300424 [rdgv240sol13+14-bootdisks]
Logical device ID=60060160E83416001C81C15E575FDF11 [LUN 3] <<< ALU "3"
state=alive; policy=CLAROpt; priority=0; queued-IOs=0
Owner: default=SP B, current=SP B Array failover mode: 4
==============================================================================
---------------- Host --------------- - Stor - -- I/O Path - -- Stats ---
### HW Path I/O Paths Interf. Mode State Q-IOs Errors
==============================================================================
3073 pci@1e,600000/SUNW,jfca@2/fp@0,0 c1t500601603021C60Ed7s0 SP A0 active alive 0 0
3073 pci@1e,600000/SUNW,jfca@2/fp@0,0 c1t500601613021C60Ed7s0 SP A1 active alive 0 0
3073 pci@1e,600000/SUNW,jfca@2/fp@0,0 c1t500601683021C60Ed7s0 SP B0 active alive 0 0
3073 pci@1e,600000/SUNW,jfca@2/fp@0,0 c1t500601693021C60Ed7s0 SP B1 active alive 0 0
3074 pci@1e,600000/SUNW,qlc@3/fp@0,0 c5t500601603021C60Ed7s0 SP A0 active alive 0 0
3074 pci@1e,600000/SUNW,qlc@3/fp@0,0 c5t500601613021C60Ed7s0 SP A1 active alive 0 0
3074 pci@1e,600000/SUNW,qlc@3/fp@0,0 c5t500601683021C60Ed7s0 SP B0 active alive 0 0
3074 pci@1e,600000/SUNW,qlc@3/fp@0,0 c5t500601693021C60Ed7s0 SP B1 active alive 0 0
Pseudo name=emcpower4a
CLARiiON ID=CK200053300424 [rdgv240sol13+14-bootdisks]
Logical device ID=60060160E83416004A3CB095BD3BDE11 [LUN 78] <<< ALU "78"
state=alive; policy=CLAROpt; priority=0; queued-IOs=0
Owner: default=SP B, current=SP B Array failover mode: 4
==============================================================================
---------------- Host --------------- - Stor - -- I/O Path - -- Stats ---
### HW Path I/O Paths Interf. Mode State Q-IOs Errors
==============================================================================
3073 pci@1e,600000/SUNW,jfca@2/fp@0,0 c1t500601603021C60Ed5s0 SP A0 active alive 0 0
3073 pci@1e,600000/SUNW,jfca@2/fp@0,0 c1t500601613021C60Ed5s0 SP A1 active alive 0 0
3073 pci@1e,600000/SUNW,jfca@2/fp@0,0 c1t500601683021C60Ed5s0 SP B0 active alive 0 0
3073 pci@1e,600000/SUNW,jfca@2/fp@0,0 c1t500601693021C60Ed5s0 SP B1 active alive 0 0
3074 pci@1e,600000/SUNW,qlc@3/fp@0,0 c5t500601603021C60Ed5s0 SP A0 active alive 0 0
3074 pci@1e,600000/SUNW,qlc@3/fp@0,0 c5t500601613021C60Ed5s0 SP A1 active alive 0 0
3074 pci@1e,600000/SUNW,qlc@3/fp@0,0 c5t500601683021C60Ed5s0 SP B0 active alive 0 0
3074 pci@1e,600000/SUNW,qlc@3/fp@0,0 c5t500601693021C60Ed5s0 SP B1 active alive 0 0
Pseudo name=emcpower2a
CLARiiON ID=CK200053300424 [rdgv240sol13+14-bootdisks]
Logical device ID=60060160E8341600F2BC89206340DE11 [LUN 79] <<< ALU "79"
state=alive; policy=CLAROpt; priority=0; queued-IOs=0
Owner: default=SP A, current=SP A Array failover mode: 4
==============================================================================
---------------- Host --------------- - Stor - -- I/O Path - -- Stats ---
### HW Path I/O Paths Interf. Mode State Q-IOs Errors
==============================================================================
3073 pci@1e,600000/SUNW,jfca@2/fp@0,0 c1t500601603021C60Ed6s0 SP A0 active alive 0 0
3073 pci@1e,600000/SUNW,jfca@2/fp@0,0 c1t500601613021C60Ed6s0 SP A1 active alive 0 0
3073 pci@1e,600000/SUNW,jfca@2/fp@0,0 c1t500601683021C60Ed6s0 SP B0 active alive 0 0
3073 pci@1e,600000/SUNW,jfca@2/fp@0,0 c1t500601693021C60Ed6s0 SP B1 active alive 0 0
3074 pci@1e,600000/SUNW,qlc@3/fp@0,0 c5t500601603021C60Ed6s0 SP A0 active alive 0 0
3074 pci@1e,600000/SUNW,qlc@3/fp@0,0 c5t500601613021C60Ed6s0 SP A1 active alive 0 0
3074 pci@1e,600000/SUNW,qlc@3/fp@0,0 c5t500601683021C60Ed6s0 SP B0 active alive 0 0
3074 pci@1e,600000/SUNW,qlc@3/fp@0,0 c5t500601693021C60Ed6s0 SP B1 active alive 0 0
Summary
EMC CLARiiON ALU reference "ALU 3" refers to emcpower3
EMC CLARiiON ALU reference "ALU 78" refers to emcpower4
EMC CLARiiON ALU reference "ALU 79" refers to emcpower2
TPD (Third Party Driver)
As of 5.0 MP3 onwards, VxVM/DMP will report the EMC PowerPath (TPD) EMC CLARiiON controlled devices under the PP_EMC_CLARiiON enclosure type.
# vxdmpadm listenclosure all
ENCLR_NAME ENCLR_TYPE ENCLR_SNO STATUS ARRAY_TYPE LUN_COUNT
===================================================================================
disk Disk DISKS CONNECTED Disk 3
pp_emc1 PP_EMC 000290301414 CONNECTED A/A 5
pp_emc_clariion0 PP_EMC_CLARiiON CK200053300424 CONNECTED A/A 3
pp_emc0 PP_EMC 000290300822 CONNECTED A/A 11
Note: As of 5.0 MP3 onwards, DMP will now report a LUN count per enclosure. In this instance there are "3" disks associated with the EMC CLARiiON enclosure.
# powermt display class=clariion
CLARiiON logical device count=3
==============================================================================
----- Host Bus Adapters --------- ------ I/O Paths ----- ------ Stats ------
### HW Path Summary Total Dead IO/Sec Q-IOs Errors
==============================================================================
3073 pci@1e,600000/SUNW,jfca@2/fp@0,0 optimal 12 0 - 0 0
3074 pci@1e,600000/SUNW,qlc@3/fp@0,0 optimal 12 0 - 0 0
# vxdmpadm getportids enclosure=pp_emc_clariion0
NAME ENCLR-NAME ARRAY-PORT-ID pWWN
===================================================================================
emcpower4c pp_emc_clariion0 - -
emcpower2c pp_emc_clariion0 - -
emcpower3 pp_emc_clariion0 - -
Note: The array-port-id and pWWN content is suppressed due to the EMC PowerPath installation.
Volume Manager disk details
As of 5.0 MP3 onwards, the /etc/vx/disk.info file contents and the "vxdisk list" output can be updated using the "vxddladm assign names" command.
# vxddladm assign names
# vxdisk -e -o alldgs list
DEVICE TYPE DISK GROUP STATUS OS_NATIVE_NAME ATTR
disk_0 auto - (51dg) online c2t1d0s2 -
disk_1 auto - (50MP1RP5dg) online c2t2d0s2 -
disk_2 auto c2t0d0 rootdgrp4 online c2t0d0s2 -
emcpower0s2 auto - - online emcpower0c -
emcpower1s2 auto - (datadg) online emcpower1c -
emcpower2s2 auto - (rootdg) online emcpower2c - <<<< ALU "79"
emcpower3 auto - - error emcpower3 - <<<< ALU "3"
emcpower4s2 auto - - online emcpower4c - <<<< ALU "78"
emcpower5s2 auto - - error emcpower5c -
emcpower6s2 auto - (terrydg) online emcpower6c -
emcpower7s2 auto - - online emcpower7c -
emcpower8s2 auto - (newdg) online emcpower8c -
emcpower9s2 auto - (datadg) online emcpower9c -
emcpower10s2 auto - (datadg) online emcpower10c -
emcpower11s2 auto - (newdg) online emcpower11c -
emcpower12s2 auto - - online emcpower12c -
emcpower13s2 auto - (datadg) online emcpower13c -
emcpower14s2 auto - - online emcpower14c -
emcpower15s2 auto - - online emcpower15c -
emcpower16s2 auto - (datadg) online emcpower16c -
emcpower17s2 auto - (testdg) online emcpower17c -
emcpower18s2 auto - - online emcpower18c -
Note: To avoid confusion it is critical that the EMC PowerPath pseudo device name corresponds to the Veritas Disk access name (DA) when the tpdmode is set to pseudo naming.
EMC CLARiiON ALU references to Veritas Disk Access Names
EMC CLARiiON ALU reference "ALU 3" refers to emcpower3, Veritas disk access name emcpower3
EMC CLARiiON ALU reference "ALU 78" refers to emcpower4, Veritas disk access name emcpower4s2
EMC CLARiiON ALU reference "ALU 79" refers to emcpower2, Veritas disk access name emcpower2s2
Steps to unpresent the EMC CLARiiON devices
Stop eventsource daemon
Due to interoperability issues with EMC PowerPath and the Volume Manager/DMP stack, Veritas recommend that the "eventsource" (vxesd) be disabled prior to the presenting of new devices and the removal of devices.
5.1 SP1 will contain a "vxesd" product enhancement allowing EMC PowerPath and Volume Manager/DMP to safely interact, without the current interoperability issues.
# ps -ef | grep vxesd
root 775 1 0 12:22:24 ? 0:00 /sbin/vxesd
root 11898 10965 0 12:48:30 pts/1 0:00 grep vxesd
How to stop the "vxesd" daemon
# vxddladm stop eventsource
# pgrep vxesd
1.] Ensure all corresponding filesystems are unmounted.
2.] Deport related Volume Manager diskgroups related to the EMC CLARiiON storage to be removed.
# vxdg deport
3.] Confirm using "vxdg list" that all required diskgroup(s) were deported
4.] Inform Volume Manager that the corresponding luns are to be removed, by running "vxdisk rm
In this instance, the disk access names are " emcpower2s2 ", " emcpower3 " and " emcpower4s2 "
# vxdisk rm emcpower2s2 emcpower3 emcpower4s2
5.] Inform EMC PowerPath that the EMC pseudo devices are being removed from the host:
# powermt remove dev=emcpower2c
# powermt remove dev=emcpower3c
# powermt remove dev=emcpower4c
Note: It is critical that the EMC Pseudo names are specified separately, otherwise, the EMC "powercf -q" command will fail to delete the removed disk entries from the EMC PowerPath configuration.
# powercf -q
Could not validate the entry:
---------------------------------------
emcpower2: user ID = 11100000010
---------------------------------------
removing emcpower2
Could not validate the entry:
---------------------------------------
emcpower3: user ID = 11100000018
---------------------------------------
removing emcpower3
Could not validate the entry:
---------------------------------------
emcpower4: user ID = 11100000020
---------------------------------------
removing emcpower4
6.] Deactivate and remove the EMC MirrorView SnapView snapshot images using the EMC naviseccli utility.
In this instance, the EMC CLARiiON IP address is "10.12.208.81" and the EMC CLARiiON StorageGroup is "rdgv240sol13+14-bootdisks"
# navicli -h 10.12.208.81 storagegroup -disconnecthost -o -host rdgv240sol14 -gname rdgv240sol13+14-bootdisks
# navicli -h 10.12.208.81 storagegroup -list -gname rdgv240sol13+14-bootdisks
Storage Group Name: rdgv240sol13+14-bootdisks
Storage Group UID: 60:CC:49:25:65:41:DF:11:B2:65:00:60:16:01:6D:D5
HBA/SP Pairs:
HBA UID SP Name SPPort
------- ------- ------
10:00:00:01:73:01:BB:EB:20:00:00:01:73:01:BB:EB SP A 1
10:00:00:01:73:01:BB:EB:20:00:00:01:73:01:BB:EB SP B 1
10:00:00:01:73:01:BB:EB:20:00:00:01:73:01:BB:EB SP A 0
10:00:00:01:73:01:BB:EB:20:00:00:01:73:01:BB:EB SP B 0
HLU/ALU Pairs:
HLU Number ALU Number
---------- ----------
5 78
6 79
7 3
Shareable: YES
6.] Refresh the Solaris leadville stack with the Storage activity events.
# cfgadm -alo show_FCP_dev
Ap_Id Type Receptacle Occupant Condition
c1 fc-fabric connected configured unknown
c1::200000017301bbeb unknown connected unconfigured unknown
c1::210000e08b8129fa unknown connected unconfigured unknown
c1::500601603021c60e,0 disk connected configured unknown
c1::500601603021c60e,5 disk connected configured unusable
c1::500601603021c60e,6 disk connected configured unusable
c1::500601603021c60e,7 disk connected configured unusable
c1::500601613021c60e,0 disk connected configured unknown
c1::500601613021c60e,5 disk connected configured unusable
c1::500601613021c60e,6 disk connected configured unusable
c1::500601613021c60e,7 disk connected configured unusable
c1::500601683021c60e,0 disk connected configured unknown
c1::500601683021c60e,5 disk connected configured unusable
c1::500601683021c60e,6 disk connected configured unusable
c1::500601683021c60e,7 disk connected configured unusable
c1::500601693021c60e,0 disk connected configured unknown
c1::500601693021c60e,5 disk connected configured unusable
c1::500601693021c60e,6 disk connected configured unusable
c1::500601693021c60e,7 disk connected configured unusable
c1::5006048c5368e580,0 disk connected configured unknown
c1::5006048c5368e580,16 disk connected configured unknown
c1::5006048c5368e580,17 disk connected configured unknown
c1::5006048c5368e580,18 disk connected configured unknown
c1::5006048c5368e580,19 disk connected configured unknown
c1::5006048c5368e580,20 disk connected configured unknown
c1::5006048c5368e580,26 disk connected configured unknown
c1::5006048c5368e580,27 disk connected configured unknown
c1::5006048c5368e580,210 disk connected configured unknown
c1::5006048c5368e580,211 disk connected configured unknown
c1::5006048c5368e580,212 disk connected configured unknown
c1::5006048c5368e580,213 disk connected configured unknown
c1::5006048c5368e5a0,0 disk connected configured unknown
c1::5006048c5368e5a0,15 disk connected configured unknown
c1::5006048c5368e5a0,196 disk connected configured unknown
c1::5006048c5368e5a0,197 disk connected configured unknown
c1::5006048c5368e5a0,198 disk connected configured unknown
c1::5006048c5368e5a0,199 disk connected configured unknown
c1::5006048c5368e5a0,220 disk connected configured unknown
c1::5006048c5368e5a0,221 disk connected configured unknown
c1::5006048c5368e5a0,228 disk connected configured unknown
c1::5006048c53697980,0 disk connected configured unknown
c1::5006048c53697980,23 disk connected configured unknown
c1::5006048c53697980,32 disk connected configured unknown
c1::5006048c53697980,33 disk connected configured unknown
c1::5006048c53697980,34 disk connected configured unknown
c1::5006048c53697980,215 disk connected configured unknown
c5 fc-fabric connected configured unknown
c5::200000017301bbdf unknown connected unconfigured unknown
c5::200000017301bbeb unknown connected unconfigured unknown
c5::500601603021c60e,0 disk connected configured unknown
c5::500601603021c60e,5 disk connected configured unusable
c5::500601603021c60e,6 disk connected configured unusable
c5::500601603021c60e,7 disk connected configured unusable
c5::500601613021c60e,0 disk connected configured unknown
c5::500601613021c60e,5 disk connected configured unusable
c5::500601613021c60e,6 disk connected configured unusable
c5::500601613021c60e,7 disk connected configured unusable
c5::500601683021c60e,0 disk connected configured unknown
c5::500601683021c60e,5 disk connected configured unusable
c5::500601683021c60e,6 disk connected configured unusable
c5::500601683021c60e,7 disk connected configured unusable
c5::500601693021c60e,0 disk connected configured unknown
c5::500601693021c60e,5 disk connected configured unusable
c5::500601693021c60e,6 disk connected configured unusable
c5::500601693021c60e,7 disk connected configured unusable
Note: All the access paths for the devices to be removed, must be shown as "unusable" and not failing.
How to unconfigure the devices shown as "unusable"
Sample syntax
# cfgadm -alo show_FCP_dev | grep "unusable" | awk -F"," '{ print $1}' | sort |uniq -c
3 c1::500601603021c60e
3 c1::500601613021c60e
3 c1::500601683021c60e
3 c1::500601693021c60e
3 c5::500601603021c60e
3 c5::500601613021c60e
3 c5::500601683021c60e
3 c5::500601693021c60e
# cfgadm -alo show_FCP_dev | grep "unusable" | awk -F"," '{ print $1}' | sort |uniq | tr -s "\n" " "
c1::500601603021c60e c1::500601613021c60e c1::500601683021c60e c1::500601693021c60e c5::500601603021c60e c5::500601613021c60e c5::500601683021c60e c5::500601693021c60e
# accesspaths=`cfgadm -alo show_FCP_dev | grep "unusable" | awk -F"," '{ print $1}' | sort |uniq | tr -s "\n" " "`
# echo $accesspaths
c1::500601603021c60e c1::500601613021c60e c1::500601683021c60e c1::500601693021c60e c5::500601603021c60e c5::500601613021c60e c5::500601683021c60e c5::500601693021c60e
# /usr/sbin/cfgadm -o unusable_FCP_dev -c unconfigure $accesspaths
7.] Confirm the access paths have been removed from the "cfgadm -alo show_FCP_dev" listing.
# cfgadm -alo show_FCP_dev | grep "unusable" | awk -F"," '{ print $1}' | sort |uniq
# cfgadm -alo show_FCP_dev
Ap_Id Type Receptacle Occupant Condition
c1 fc-fabric connected configured unknown
c1::200000017301bbeb unknown connected unconfigured unknown
c1::210000e08b8129fa unknown connected unconfigured unknown
c1::500601603021c60e,0 disk connected configured unknown
c1::500601613021c60e,0 disk connected configured unknown
c1::500601683021c60e,0 disk connected configured unknown
c1::500601693021c60e,0 disk connected configured unknown
c1::5006048c5368e580,0 disk connected configured unknown
c1::5006048c5368e580,16 disk connected configured unknown
c1::5006048c5368e580,17 disk connected configured unknown
c1::5006048c5368e580,18 disk connected configured unknown
c1::5006048c5368e580,19 disk connected configured unknown
c1::5006048c5368e580,20 disk connected configured unknown
c1::5006048c5368e580,26 disk connected configured unknown
c1::5006048c5368e580,27 disk connected configured unknown
c1::5006048c5368e580,210 disk connected configured unknown
c1::5006048c5368e580,211 disk connected configured unknown
c1::5006048c5368e580,212 disk connected configured unknown
c1::5006048c5368e580,213 disk connected configured unknown
c1::5006048c5368e5a0,0 disk connected configured unknown
c1::5006048c5368e5a0,15 disk connected configured unknown
c1::5006048c5368e5a0,196 disk connected configured unknown
c1::5006048c5368e5a0,197 disk connected configured unknown
c1::5006048c5368e5a0,198 disk connected configured unknown
c1::5006048c5368e5a0,199 disk connected configured unknown
c1::5006048c5368e5a0,220 disk connected configured unknown
c1::5006048c5368e5a0,221 disk connected configured unknown
c1::5006048c5368e5a0,228 disk connected configured unknown
c1::5006048c53697980,0 disk connected configured unknown
c1::5006048c53697980,23 disk connected configured unknown
c1::5006048c53697980,32 disk connected configured unknown
c1::5006048c53697980,33 disk connected configured unknown
c1::5006048c53697980,34 disk connected configured unknown
c1::5006048c53697980,215 disk connected configured unknown
c5 fc-fabric connected configured unknown
c5::200000017301bbdf unknown connected unconfigured unknown
c5::200000017301bbeb unknown connected unconfigured unknown
c5::500601603021c60e,0 disk connected configured unknown
c5::500601613021c60e,0 disk connected configured unknown
c5::500601683021c60e,0 disk connected configured unknown
c5::500601693021c60e,0 disk connected configured unknown
8.] Ensure the stale device entries are removed from /dev/[r]dsk.
#
rdgv240sol14# /usr/sbin/devfsadm -Cvc disk
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601603021C60Ed7s1
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601603021C60Ed7s2
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601603021C60Ed7s3
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601603021C60Ed7s4
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601603021C60Ed7s5
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601603021C60Ed7s6
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601603021C60Ed7
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601603021C60Ed6s0
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601603021C60Ed6s1
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601603021C60Ed7s0
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601603021C60Ed5s7
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601613021C60Ed7s0
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601613021C60Ed5s1
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601613021C60Ed5s2
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601613021C60Ed5s3
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601613021C60Ed5s4
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601613021C60Ed5s5
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601613021C60Ed5s6
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601613021C60Ed5s7
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601613021C60Ed7s5
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601613021C60Ed7s6
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601613021C60Ed7
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601613021C60Ed6s0
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601613021C60Ed6s1
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601613021C60Ed6s2
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601613021C60Ed6s3
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601613021C60Ed6s4
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601613021C60Ed6s5
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601613021C60Ed6s6
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601613021C60Ed6s7
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601613021C60Ed5s0
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601603021C60Ed5s0
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601603021C60Ed5s1
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601603021C60Ed5s2
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601603021C60Ed5s3
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601603021C60Ed5s4
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601603021C60Ed5s5
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601603021C60Ed5s6
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601603021C60Ed5s7
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601613021C60Ed7s0
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601613021C60Ed7s1
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601613021C60Ed7s2
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601613021C60Ed7s3
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601613021C60Ed7s4
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601603021C60Ed6s5
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601603021C60Ed6s6
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601603021C60Ed6s7
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601603021C60Ed6s2
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601603021C60Ed6s3
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601603021C60Ed6s4
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601603021C60Ed6s5
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601603021C60Ed6s6
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601603021C60Ed6s7
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601603021C60Ed7s4
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601603021C60Ed7s5
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601603021C60Ed7s6
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601603021C60Ed7
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601603021C60Ed6s0
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601603021C60Ed6s1
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601603021C60Ed6s2
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601603021C60Ed6s3
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601603021C60Ed6s4
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601603021C60Ed7s3
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601693021C60Ed5s5
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601693021C60Ed5s6
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601693021C60Ed5s7
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601603021C60Ed7s0
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601603021C60Ed7s1
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601603021C60Ed7s2
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601603021C60Ed5s6
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601603021C60Ed5s5
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601613021C60Ed7s1
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601613021C60Ed7s2
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601613021C60Ed7s3
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601613021C60Ed7s4
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601613021C60Ed7s5
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601613021C60Ed7s6
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601613021C60Ed7
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601613021C60Ed6s0
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601603021C60Ed5s4
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601613021C60Ed6s1
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601613021C60Ed6s2
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601613021C60Ed6s3
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601613021C60Ed6s4
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601613021C60Ed6s5
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601603021C60Ed5s3
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601693021C60Ed6s6
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601693021C60Ed6s7
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601693021C60Ed5s0
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601693021C60Ed5s1
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601693021C60Ed5s2
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601693021C60Ed5s3
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601693021C60Ed5s4
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601683021C60Ed6s3
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601693021C60Ed6s5
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601693021C60Ed6s4
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601693021C60Ed6s5
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601693021C60Ed6s6
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601693021C60Ed6s7
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601693021C60Ed5s0
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601693021C60Ed5s1
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601693021C60Ed5s2
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601693021C60Ed5s3
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601693021C60Ed5s4
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601693021C60Ed5s5
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601693021C60Ed5s6
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601693021C60Ed5s7
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601693021C60Ed7s6
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601693021C60Ed7
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601693021C60Ed6s0
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601693021C60Ed6s1
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601693021C60Ed6s2
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601693021C60Ed6s3
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601683021C60Ed6s2
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601603021C60Ed5s2
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601613021C60Ed6s6
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601613021C60Ed6s7
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601613021C60Ed5s0
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601613021C60Ed5s1
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601613021C60Ed5s2
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601613021C60Ed5s3
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601613021C60Ed5s4
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601613021C60Ed5s5
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601613021C60Ed5s6
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601603021C60Ed5s1
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601613021C60Ed5s7
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601693021C60Ed7s3
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601693021C60Ed7s4
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601693021C60Ed7s5
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601693021C60Ed7s6
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601693021C60Ed7
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601693021C60Ed6s0
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601693021C60Ed6s1
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601693021C60Ed6s2
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601693021C60Ed6s3
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601693021C60Ed6s4
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601603021C60Ed5s0
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601683021C60Ed7s0
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601683021C60Ed7s1
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601683021C60Ed7s2
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601683021C60Ed7s3
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601683021C60Ed7s4
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601683021C60Ed7s5
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601683021C60Ed7s6
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601683021C60Ed7
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601683021C60Ed6s0
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601683021C60Ed5s0
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601683021C60Ed5s1
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601683021C60Ed5s2
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601683021C60Ed5s3
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601683021C60Ed5s4
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601683021C60Ed5s5
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601683021C60Ed5s6
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601683021C60Ed5s7
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601693021C60Ed7s0
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601693021C60Ed7s1
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601693021C60Ed7s2
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601683021C60Ed6s5
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601683021C60Ed6s4
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601683021C60Ed6s1
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601693021C60Ed7s0
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601693021C60Ed7s1
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601693021C60Ed7s2
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601693021C60Ed7s3
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601693021C60Ed7s4
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601693021C60Ed7s5
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601683021C60Ed5s2
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601683021C60Ed5s3
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601683021C60Ed5s4
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601683021C60Ed5s5
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601683021C60Ed5s6
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601683021C60Ed5s7
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601683021C60Ed5s0
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601683021C60Ed5s1
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601683021C60Ed6s6
devfsadm[12870]: verbose: removing file: /dev/dsk/c5t500601683021C60Ed6s7
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601683021C60Ed7s2
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601683021C60Ed7s3
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601683021C60Ed7s4
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601683021C60Ed7s5
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601683021C60Ed7s6
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601683021C60Ed7
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601683021C60Ed6s0
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601683021C60Ed6s1
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601683021C60Ed6s2
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601683021C60Ed6s3
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601683021C60Ed6s4
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601683021C60Ed6s5
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601683021C60Ed6s6
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601683021C60Ed6s7
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601683021C60Ed7s0
devfsadm[12870]: verbose: removing file: /dev/dsk/c1t500601683021C60Ed7s1
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601603021C60Ed7s3
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601603021C60Ed7s4
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601603021C60Ed7s5
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601603021C60Ed7s6
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601603021C60Ed7
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601603021C60Ed6s0
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601603021C60Ed6s1
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601603021C60Ed6s2
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601603021C60Ed6s3
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601603021C60Ed7s2
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601603021C60Ed5s7
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601613021C60Ed7s0
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601613021C60Ed6s5
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601613021C60Ed6s6
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601613021C60Ed6s7
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601613021C60Ed5s0
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601613021C60Ed5s1
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601613021C60Ed5s2
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601613021C60Ed5s3
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601613021C60Ed5s4
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601613021C60Ed5s5
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601613021C60Ed5s6
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601613021C60Ed5s7
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601613021C60Ed7s1
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601613021C60Ed7s2
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601613021C60Ed7s3
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601613021C60Ed7s4
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601613021C60Ed7s5
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601613021C60Ed7s6
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601613021C60Ed7
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601613021C60Ed6s0
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601613021C60Ed6s1
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601613021C60Ed6s2
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601613021C60Ed6s3
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601613021C60Ed6s4
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601603021C60Ed6s4
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601603021C60Ed6s5
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601603021C60Ed6s6
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601603021C60Ed6s7
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601603021C60Ed5s0
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601603021C60Ed5s1
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601603021C60Ed5s2
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601603021C60Ed5s3
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601603021C60Ed5s4
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601603021C60Ed5s5
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601603021C60Ed5s6
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601603021C60Ed5s7
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601613021C60Ed7s0
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601603021C60Ed7s6
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601603021C60Ed7
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601603021C60Ed6s0
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601603021C60Ed6s1
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601603021C60Ed6s2
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601603021C60Ed6s3
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601603021C60Ed6s4
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601603021C60Ed6s5
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601603021C60Ed6s6
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601693021C60Ed5s5
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601693021C60Ed5s6
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601693021C60Ed5s7
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601603021C60Ed7s0
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601603021C60Ed7s1
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601603021C60Ed7s2
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601603021C60Ed7s3
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601603021C60Ed7s4
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601603021C60Ed7s5
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601693021C60Ed5s4
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601693021C60Ed6s6
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601693021C60Ed6s7
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601693021C60Ed5s0
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601693021C60Ed5s1
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601693021C60Ed5s2
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601693021C60Ed5s3
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601603021C60Ed5s6
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601603021C60Ed5s5
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601613021C60Ed7s1
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601613021C60Ed7s2
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601613021C60Ed7s3
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601613021C60Ed7s4
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601613021C60Ed7s5
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601613021C60Ed7s6
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601613021C60Ed7
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601613021C60Ed6s0
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601603021C60Ed5s4
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601613021C60Ed6s1
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601613021C60Ed6s2
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601613021C60Ed6s3
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601613021C60Ed6s4
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601613021C60Ed6s5
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601603021C60Ed5s3
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601603021C60Ed5s2
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601613021C60Ed6s6
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601613021C60Ed6s7
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601613021C60Ed5s0
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601613021C60Ed5s1
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601683021C60Ed6s5
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601693021C60Ed6s6
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601693021C60Ed6s7
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601693021C60Ed5s0
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601693021C60Ed5s1
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601693021C60Ed5s2
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601693021C60Ed5s3
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601693021C60Ed5s4
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601693021C60Ed5s5
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601693021C60Ed5s6
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601693021C60Ed5s7
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601603021C60Ed7s0
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601603021C60Ed7s1
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601693021C60Ed7s6
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601693021C60Ed7
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601693021C60Ed6s0
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601693021C60Ed6s1
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601693021C60Ed6s2
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601693021C60Ed6s3
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601693021C60Ed6s4
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601693021C60Ed6s5
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601603021C60Ed5s1
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601613021C60Ed5s2
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601613021C60Ed5s3
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601613021C60Ed5s4
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601613021C60Ed5s5
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601613021C60Ed5s6
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601613021C60Ed5s7
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601683021C60Ed7s0
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601683021C60Ed7s1
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601683021C60Ed7s2
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601603021C60Ed5s0
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601683021C60Ed7s3
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601693021C60Ed7s4
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601693021C60Ed7s5
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601693021C60Ed7s6
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601693021C60Ed7
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601693021C60Ed6s0
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601693021C60Ed6s1
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601693021C60Ed6s2
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601693021C60Ed6s3
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601693021C60Ed6s4
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601693021C60Ed6s5
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601603021C60Ed6s7
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601683021C60Ed7s4
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601683021C60Ed7s5
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601683021C60Ed7s6
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601683021C60Ed7
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601683021C60Ed6s0
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601683021C60Ed6s1
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601683021C60Ed6s2
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601683021C60Ed6s3
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601683021C60Ed6s4
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601683021C60Ed6s7
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601683021C60Ed6s6
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601693021C60Ed7s0
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601693021C60Ed7s1
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601693021C60Ed7s2
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601693021C60Ed7s3
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601693021C60Ed7s4
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601693021C60Ed7s5
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601683021C60Ed5s7
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601693021C60Ed7s0
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601693021C60Ed7s1
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601693021C60Ed7s2
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601693021C60Ed7s3
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601683021C60Ed5s2
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601683021C60Ed5s3
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601683021C60Ed5s4
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601683021C60Ed5s5
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601683021C60Ed5s6
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601683021C60Ed5s7
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601683021C60Ed5s0
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601683021C60Ed5s1
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601683021C60Ed5s0
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601683021C60Ed5s1
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601683021C60Ed5s2
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601683021C60Ed5s3
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601683021C60Ed5s4
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601683021C60Ed5s5
devfsadm[12870]: verbose: removing file: /dev/rdsk/c5t500601683021C60Ed5s6
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601683021C60Ed7s2
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601683021C60Ed7s3
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601683021C60Ed7s4
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601683021C60Ed7s5
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601683021C60Ed7s6
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601683021C60Ed7
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601683021C60Ed6s0
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601683021C60Ed6s1
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601683021C60Ed6s2
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601683021C60Ed6s3
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601683021C60Ed6s4
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601683021C60Ed6s5
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601683021C60Ed6s6
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601683021C60Ed6s7
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601683021C60Ed7s0
devfsadm[12870]: verbose: removing file: /dev/rdsk/c1t500601683021C60Ed7s1
9.] Refresh EMC PowerPath
# powercf -q
# powermt config
# powermt save
# powermt display
Symmetrix logical device count=16
CLARiiON logical device count=0
Hitachi logical device count=0
Invista logical device count=0
HP xp logical device count=0
Ess logical device count=0
HP HSx logical device count=0
==============================================================================
----- Host Bus Adapters --------- ------ I/O Paths ----- ------ Stats ------
### HW Path Summary Total Dead IO/Sec Q-IOs Errors
==============================================================================
3073 pci@1e,600000/SUNW,jfca@2/fp@0,0 optimal 24 0 - 0 0
9.] Refresh Volume Manager and /etc/vx/disk.info file contents by running:
# vxdisk scandisks
Note: The value of "0xffffffff" states that the device has been marked for removal.
# grep "0xffff" /etc/vx/disk.info
DGC%5FRAID%205%5FCK200053300424%5F60060160E8341600F2BC89206340DE11 emcpower2 0xffffffff 0x4 emcpower2 PP_EMC_CLARiiON CK200053300424
DGC%5FRAID%205%5FCK200053300424%5F60060160E83416004A3CB095BD3BDE11 emcpower4 0xffffffff 0x4 emcpower4 PP_EMC_CLARiiON CK200053300424
DGC%5FRAID%205%5FCK200053300424%5F60060160E83416001C81C15E575FDF11 emcpower3 0xffffffff 0x4 emcpower3 PP_EMC_CLARiiON CK200053300424
# vxddladm assign names
# grep "0xffff" /etc/vx/disk.info
#
Note: As no "0xffffffff" entries are reported, the /etc/vx/disk.info file has been updated as required.
The process is now complete.
10.] Ensure that "vxdisk -e -o alldgs list" does not reference any of the removed EMC CLARiiON devices.
# vxdisk -e -o alldgs list
DEVICE TYPE DISK GROUP STATUS OS_NATIVE_NAME ATTR
disk_0 auto - (51dg) online c2t1d0s2 -
disk_1 auto - (50MP1RP5dg) online c2t2d0s2 -
disk_2 auto c2t0d0 rootdgrp4 online c2t0d0s2 -
emcpower0s2 auto - (datadg) online emcpower0c -
emcpower1s2 auto - (datadg) online emcpower1c -
emcpower5s2 auto - - error emcpower5c -
emcpower6s2 auto - (terrydg) online emcpower6c -
emcpower7s2 auto - - online emcpower7c -
emcpower8s2 auto - (newdg) online emcpower8c -
emcpower9s2 auto - (datadg) online emcpower9c -
emcpower10s2 auto - (datadg) online emcpower10c -
emcpower11s2 auto - (newdg) online emcpower11c -
emcpower12s2 auto - - online emcpower12c -
emcpower13s2 auto - - online emcpower13c -
emcpower14s2 auto - (testdg) online emcpower14c -
emcpower15s2 auto - - online emcpower15c -
emcpower16s2 auto - (datadg) online emcpower16c -
emcpower17s2 auto - (testdg) online emcpower17c -
emcpower18s2 auto - - online emcpower18c -
Note: As shown above, the removed EMC CLARiiON devices are no longer visible to Volume Manager.
# vxdmpadm listenclosure all
ENCLR_NAME ENCLR_TYPE ENCLR_SNO STATUS ARRAY_TYPE LUN_COUNT
===================================================================================
disk Disk DISKS CONNECTED Disk 3
pp_emc1 PP_EMC 000290301414 CONNECTED A/A 5
pp_emc0 PP_EMC 000290300822 CONNECTED A/A 11
Note: DMP no longer reports any EMC CLARiiON devices.
Issue/Introduction
