Description

The following procedure has evolved to reduce the multi-step process surrounding the addition and removal of device paths.

On Linux platforms running Veritas Volume Manager (VxVM) product versions 5.1 SP1 RP4 / 6.0.3.200 onwards, the user no longer needs to manual disable the required DMP paths which are going to be removed at the storage layer.

Note: It is vital that sufficient active paths remain available, so the DMPNODE is not impacted.

The enhanced procedure will use the “Peripheral Qualifier/ Device Type” attribute to establish when a device path has been intentionally masked at the storage layer from the server.
When the “Peripheral Qualifier/ Device Type” reports a non-zero value, this means the device path has been masked at the storage layer.

The user must execute “vxdisk scandisks” directly following the removal of paths at the storage layer to avoid dev reuse cases where previous stale DMP dev addresses are used until VxVM+DMP has deleted them.

 
Support illustrates the migration of devices paths as follows:

Steps:

1 .] New paths are presented to the server.

2.] The OS attempts to detect the new device paths .

#! /bin/ksh

for i in `ls -Ud /sys/class/scsi_host/host*`
do
echo '- - -' > $i/scan
done

3.] DMP detects the new additional paths.

4.] The user will then mask the to be removed OS devices paths at the storage layer.

5.] The DMP incore database (kernel) is refreshed using "vxdisk scandisks", which is responsible for deleting the corresponding stale device paths.

6.] The stale OS device handles are then removed at the OS the level using the 3^rd party script “rescan-scsi-bus.sh –r”.

The "vxdmpadm_remove_list" script is attached to this technical article.
 

Example:

EMC DMX storage will be used to illustrate the evolved process:

1.] The following EMC SYMDEV "0262" device currently has two paths via EMC director "1c" and ports 0 + 1.

# ./vxdmpadm_remove_list | grep emc0_0262
sdam         ENABLED(A)   -          emc0_0262    emc0         c1        -      0
sdan         ENABLED(A)   -          emc0_0262    emc0         c1        -      0
 

2.] Two additional paths are then presented by EMC director "16c" and ports 0 and 1.

# symmask -sid 822 -wwn 210000e08b1c1df0 -dir 16c -p 0 add devs 0262 -nopr
# symmask -sid 822 -wwn 210000e08b1c1df0 -dir 16c -p 1 add devs 0262 -nopr

# symmask -sid 822 refresh

Refresh Symmetrix FA/SE directors with contents of SymMask database 000290300822 (y/[n]) ? y

Symmetrix FA/SE directors updated with contents of SymMask Database 000290300822


3.] The OS is then updated by probing the host based adapters (HBA) in the server to detect the new device paths .
 

4.] In some cases the user will need to run "vxdisk scandisks" manually (where the DMP tunable "dmp_monitor_osevent" is disabled) for DMP+VxVM to detect the presentation of the new additional device paths.
 

# vxdmpadm gettune dmp_monitor_osevent
            Tunable               Current Value  Default Value
------------------------------    -------------  -------------
dmp_monitor_osevent                      on               on
 

# ./vxdmpadm_remove_list | grep emc0_0262
sdam         ENABLED(A)   -          emc0_0262    emc0         c1        -      0
sdan         ENABLED(A)   -          emc0_0262    emc0         c1        -      0
sdao         ENABLED(A)   -          emc0_0262    emc0         c1        -      0
sdap         ENABLED(A)   -          emc0_0262    emc0         c1        -      0
 

5.] Prior to removing the intended device paths, the user can refresh the EMC storage database as follows:

# symcfg -sid 822 discover

Attempting discovery of Symmetrix 000290300822

This operation may take up to a few minutes. Please be patient...
 

6.] The OS native device path content along with the EMC director and port information can be extracted as below:
 

# symdev -sid 822 show 0262 | grep -A+7 "PdevName"
        PdevName                 Type      Type Num    Sts  VBUS TID SYMM Host
        ----------------------------------------------------------------------
        /dev/vx/rdmp/emc0_0262   N/A       FA   01C:1  RW   000  00   07A 07A
        /dev/sdan                N/A       FA   01C:0  RW   000  00   114 014
        /dev/sdam                N/A       FA   01C:1  RW   000  00   07A 07A
        /dev/sdao                N/A       FA   16C:0  RW   000  00   091 091
        /dev/sdap                N/A       FA   16C:1  RW   000  00   033 033
        }
 

7.] In this instance, the unwanted device paths from director "1C", ports 0 and 1 will now be masked from the server:
 

# symmask -sid 822 -wwn 210000e08b1c1df0 -dir 1c -p 0 remove devs 0262
# symmask -sid 822 -wwn 210000e08b1c1df0 -dir 1c -p 1 remove devs 0262
 

8.] Until the storage removal operation is commited (refreshed) the “Peripheral Qualifier/ Device Type” attribute will not be updated.

# ./vxdmpadm_remove_list | grep emc0_0262
sdam         ENABLED(A)   -          emc0_0262    emc0         c1        -      0
sdan         ENABLED(A)   -          emc0_0262    emc0         c1        -      0
sdao         ENABLED(A)   -          emc0_0262    emc0         c1        -      0
sdap         ENABLED(A)   -          emc0_0262    emc0         c1        -      0
 

# symmask -sid 822 refresh -nopr

Symmetrix FA/SE directors updated with contents of SymMask Database 000290300822
 

9.] Now the storage refresh (commit) operation has been performed, the intentionally removed device paths will now report a non-zero value.

# ./vxdmpadm_remove_list | grep emc0_0262
sdam         ENABLED(A)   -          emc0_0262    emc0         c1        -      3f
sdan         ENABLED(A)   -          emc0_0262    emc0         c1        -      3f
sdao         ENABLED(A)   -          emc0_0262    emc0         c1        -      0
sdap         ENABLED(A)   -          emc0_0262    emc0         c1        -      0
 

10.] To destroy the unwanted device paths from the VxVM/DMP database, the "vxdisk scandisks" can now be executed:
 

# vxdmpadm listctlr all
CTLR_NAME       ENCLR_TYPE      STATE        ENCLR_NAME      PATH_COUNT
=========================================================================
c0              Disk            ENABLED      disk                 1
c1              EMC             ENABLED      emc0                 28 <<<
c1              EMC             ENABLED      emc1                 6
c1              EMC_CLARiiON    ENABLED      emc_clariion0        6

# vxdisk scandisks

# vxdmpadm listctlr all
CTLR_NAME       ENCLR_TYPE      STATE        ENCLR_NAME      PATH_COUNT
=========================================================================
c0              Disk            ENABLED      disk                 1
c1              EMC             ENABLED      emc0                 26 <<<
c1              EMC             ENABLED      emc1                 6
c1              EMC_CLARiiON    ENABLED      emc_clariion0        6
 

11.] Once the unwanted device paths have been deleted from VxVM/DMP, the revised number of device paths can be seen.

# ./vxdmpadm_remove_list | grep emc0_0262
sdao         ENABLED(A)   -          emc0_0262    emc0         c1        -      0
sdap         ENABLED(A)   -          emc0_0262    emc0         c1        -      0

12.] The final stage is to remove the corresponding stale OS device handles for the removed device paths.

# ./rescan-scsi-bus.sh -r
Host adapter 0 (mptspi) found.
Host adapter 1 (qla2xxx) found.
.

.
0 new device(s) found.
2 device(s) removed.
 

A final check could be to confirm that I/O is passing through the remaining paths using commands such as iostat & vxdmpadm, prior to adding paths:

# vxdmpadm -z iostat show interval=2 count=10        ## check the new paths showing any IO. Capture these outputs for future reference.

# iostat -cdx 5 20