Description

Veritas introduced a new feature in 5.0 MP3 onwards called the "Data Corruption Prevention Activity" (DCPA) feature which basically puts a bubble around each DMP metadevice.


"Data Corruption Protection Activated (DCPA)"


The Data Corruption Prevention Activity (DCPA) feature will assist with safeguarding your data, and potential prevent data corruption.
This is a proactive message, rather than a warning about specific corruption.

Using the "bubble protector" technology, the feature prevents newly presented paths from merging with an incorrect pre-existing DMP metadevice. This dramatically helps to reduce the chances of data corruption, when the incorrect lun removal steps and critical storage provisioning process are not followed.

Prior to 5.0 MP3 it is very difficult to conclude if a step during the lun removal process failed or was accidentally missed.

Example

 
In this instance, Veritas disk access (da) name "disk_1" has not been removed correctly in the past.
A new device with a different serial has been presented on the same c#t#d# address.

# vxdisk scandisks
VxVM vxdisk ERROR V-5-1-0  Data Corruption Protection Activated - User Corrective Action Needed
VxVM vxdisk INFO V-5-1-0  To recover, first ensure that the OS device tree is up to date (requires OS specific commands).
VxVM vxdisk INFO V-5-1-0  Then, execute 'vxdisk rm' on the following devices before reinitiating device discovery:

        disk_1

 

Additional content can be displayed when recycling vxconfigd.

Note: If running Veritas Cluster Server (VCS), ensure all ServiceGroups are frozen or the cluster is "forced" stop (hastop -local -force) prior to running "vxconfigd -k.

# vxconfigd -k
VxVM vxconfigd NOTICE V-5-1-0 LUN serial number of the OS device path with device number 32/24 has changed from 303530344241303347310000 (c2t0d0) to 3035303342 395A3937350000 (c2t0d0)
VxVM vxconfigd NOTICE V-5-1-13788 Data Corruption Protection Activated - User Corrective Action Needed:
VxVM vxconfigd NOTICE V-5-1-0 To recover, first ensure that the OS device tree is up to date (requires OS specific commands). Then, execute 'vxdisk rm' on the following devices before reinitiating device discovery using 'vxdisk scandisks'

VxVM vxconfigd NOTICE V-5-1-0 disk_1

Note: The Lun Serial Number (LSN) for "c2t0d0" has changed for DMP metadevice entry "disk_1".

Current “vxdisk list” output

# vxdisk list
DEVICE       TYPE            DISK         GROUP        STATUS
disk_0       auto:sliced     c2t2d0       rootdg       online
disk_1       auto            -            -            nolabel                       <<<<< disk replaced incorrectly
emc_clariion0_83 auto:cdsdisk    emc_clariion0_83  rockdg       online
emc_clariion0_84 auto:cdsdisk    emc_clariion0_84  rockdg       online
-            -         replacedisk  rootdg       removed was:disk_1

Recovery Steps:

1.] Inform Veritas Volume Manager (VxVM) that the problematic Veritas disk access (da) name is to be removed.

 
In this instance, the problematic da name is "disk_1".

# vxdisk rm disk_1

2.] Offline the device path(s) corresponding to /dev/rdsk/c#t#d#s2.

In this instance, Veritas disk access name (da) has a single path.

# luxadm -e offline /dev/rdsk/c2t0d0s2

# cfgadm -al
Ap_Id                          Type         Receptacle   Occupant     Condition
c0                             scsi-bus     connected    configured   unknown
c0::dsk/c0t0d0                 CD-ROM       connected    configured   unknown
c1                             fc-fabric    connected    configured   unknown
c1::200000017301bbbd           unknown      connected    unconfigured unknown
c1::500601603021c60e           disk         connected    configured   unknown
c1::500601613021c60e           disk         connected    configured   unknown
c1::500601683021c60e           disk         connected    configured   unknown
c1::500601693021c60e           disk         connected    configured   unknown
c1::5006048c5368e580           disk         connected    configured   unknown
c1::5006048c5368e5a0           disk         connected    configured   unknown
c1::5006048c53697980           disk         connected    unconfigured unknown
c1::5006048c536979a0           disk         connected    configured   unknown
c1::50060e80102c41d0           disk         connected    configured   unknown
c1::50060e80102c41d2           unavailable  connected    unconfigured failed
c1::50060e80102c41d3           unavailable  connected    unconfigured failed
c2                             scsi-bus     connected    configured   unknown
c2::dsk/c2t0d0                 disk         connected    unconfigured unknown   <<<<<<< disk is the removed access point for replaced disk !
c2::dsk/c2t2d0                 disk         connected    configured   unknown
c3                             scsi-bus     connected    unconfigured unknown
usb0/1                         unknown      empty        unconfigured ok
usb0/2                         unknown      empty        unconfigured ok

3.] Clean-up dirty/stale Solaris device tree entries.

  # devfsadm -Cvc disk
devfsadm[16722]: verbose: chown/chmod /devices/scsi_vhci:devctl 0l/3l/0
devfsadm[16722]: verbose: chown/chmod /devices/pci@1e,600000/usb@a:1 0l/3l/0
devfsadm[16722]: verbose: chown/chmod /devices/pci@1e,600000/usb@a:2 0l/3l/0
devfsadm[16722]: verbose: chown/chmod /devices/pci@1e,600000/ide@d:devctl 0l/3l/0
devfsadm[16722]: verbose: chown/chmod /devices/pci@1e,600000/ide@d/sd@0,0:a 0l/3l/0
devfsadm[16722]: verbose: chown/chmod /devices/pci@1e,600000/ide@d/sd@0,0:b 0l/3l/0
devfsadm[16722]: verbose: chown/chmod /devices/pci@1e,600000/ide@d/sd@0,0:c 0l/3l/0
devfsadm[16722]: verbose: chown/chmod /devices/pci@1e,600000/ide@d/sd@0,0:d 0l/3l/0
devfsadm[16722]: verbose: chown/chmod /devices/pci@1e,600000/ide@d/sd@0,0:e 0l/3l/0
devfsadm[16722]: verbose: chown/chmod /devices/pci@1e,600000/ide@d/sd@0,0:f 0l/3l/0
devfsadm[16722]: verbose: chown/chmod /devices/pci@1e,600000/ide@d/sd@0,0:g 0l/3l/0
devfsadm[16722]: verbose: chown/chmod /devices/pci@1e,600000/ide@d/sd@0,0:h 0l/3l/0
devfsadm[16722]: verbose: chown/chmod /devices/pci@1e,600000/ide@d/sd@0,0:a,raw 0l/3l/0
devfsadm[16722]: verbose: chown/chmod /devices/pci@1e,600000/ide@d/sd@0,0:b,raw 0l/3l/0
devfsadm[16722]: verbose: chown/chmod /devices/pci@1e,600000/ide@d/sd@0,0:c,raw 0l/3l/0
devfsadm[16722]: verbose: chown/chmod /devices/pci@1e,600000/ide@d/sd@0,0:d,raw 0l/3l/0
devfsadm[16722]: verbose: chown/chmod /devices/pci@1e,600000/ide@d/sd@0,0:e,raw 0l/3l/0
devfsadm[16722]: verbose: chown/chmod /devices/pci@1e,600000/ide@d/sd@0,0:f,raw 0l/3l/0
devfsadm[16722]: verbose: chown/chmod /devices/pci@1e,600000/ide@d/sd@0,0:g,raw 0l/3l/0
devfsadm[16722]: verbose: chown/chmod /devices/pci@1e,600000/ide@d/sd@0,0:h,raw 0l/3l/0
devfsadm[16722]: verbose: chown/chmod /devices/pci@1c,600000/scsi@2:devctl 0l/3l/0
devfsadm[16722]: verbose: chown/chmod /devices/pci@1c,600000/scsi@2,1:devctl 0l/3l/0
devfsadm[16722]: verbose: chown/chmod /devices/pci@1c,600000/scsi@2/sd@2,0:a 0l/3l/0
devfsadm[16722]: verbose: chown/chmod /devices/pci@1c,600000/scsi@2/sd@2,0:b 0l/3l/0
devfsadm[16722]: verbose: chown/chmod /devices/pci@1c,600000/scsi@2/sd@2,0:c 0l/3l/0
devfsadm[16722]: verbose: chown/chmod /devices/pci@1c,600000/scsi@2/sd@2,0:d 0l/3l/0
devfsadm[16722]: verbose: chown/chmod /devices/pci@1c,600000/scsi@2/sd@2,0:e 0l/3l/0
devfsadm[16722]: verbose: chown/chmod /devices/pci@1c,600000/scsi@2/sd@2,0:f 0l/3l/0
devfsadm[16722]: verbose: chown/chmod /devices/pci@1c,600000/scsi@2/sd@2,0:g 0l/3l/0
devfsadm[16722]: verbose: chown/chmod /devices/pci@1c,600000/scsi@2/sd@2,0:h 0l/3l/0
devfsadm[16722]: verbose: chown/chmod /devices/pci@1c,600000/scsi@2/sd@2,0:a,raw 0l/3l/0
devfsadm[16722]: verbose: chown/chmod /devices/pci@1c,600000/scsi@2/sd@2,0:b,raw 0l/3l/0
devfsadm[16722]: verbose: chown/chmod /devices/pci@1c,600000/scsi@2/sd@2,0:c,raw 0l/3l/0
devfsadm[16722]: verbose: chown/chmod /devices/pci@1c,600000/scsi@2/sd@2,0:d,raw 0l/3l/0
devfsadm[16722]: verbose: chown/chmod /devices/pci@1c,600000/scsi@2/sd@2,0:e,raw 0l/3l/0
devfsadm[16722]: verbose: chown/chmod /devices/pci@1c,600000/scsi@2/sd@2,0:f,raw 0l/3l/0
devfsadm[16722]: verbose: chown/chmod /devices/pci@1c,600000/scsi@2/sd@2,0:g,raw 0l/3l/0
devfsadm[16722]: verbose: chown/chmod /devices/pci@1c,600000/scsi@2/sd@2,0:h,raw 0l/3l/0
devfsadm[16722]: verbose: removing file: /dev/dsk/c2t0d0s0
devfsadm[16722]: verbose: removing file: /dev/dsk/c2t0d0s1
devfsadm[16722]: verbose: removing file: /dev/dsk/c2t0d0s2
devfsadm[16722]: verbose: removing file: /dev/dsk/c2t0d0s3
devfsadm[16722]: verbose: removing file: /dev/dsk/c2t0d0s4
devfsadm[16722]: verbose: removing file: /dev/dsk/c2t0d0s5
devfsadm[16722]: verbose: removing file: /dev/dsk/c2t0d0s6
devfsadm[16722]: verbose: removing file: /dev/dsk/c2t0d0s7
devfsadm[16722]: verbose: removing file: /dev/rdsk/c2t0d0s0
devfsadm[16722]: verbose: removing file: /dev/rdsk/c2t0d0s1
devfsadm[16722]: verbose: removing file: /dev/rdsk/c2t0d0s2
devfsadm[16722]: verbose: removing file: /dev/rdsk/c2t0d0s3
devfsadm[16722]: verbose: removing file: /dev/rdsk/c2t0d0s4
devfsadm[16722]: verbose: removing file: /dev/rdsk/c2t0d0s5
devfsadm[16722]: verbose: removing file: /dev/rdsk/c2t0d0s6
devfsadm[16722]: verbose: removing file: /dev/rdsk/c2t0d0s7

 

4.]

# cfgadm -al
Ap_Id                          Type         Receptacle   Occupant     Condition
c0                             scsi-bus     connected    configured   unknown
c0::dsk/c0t0d0                 CD-ROM       connected    configured   unknown
c1                             fc-fabric    connected    configured   unknown
c1::200000017301bbbd           unknown      connected    unconfigured unknown
c1::500601603021c60e           disk         connected    configured   unknown
c1::500601613021c60e           disk         connected    configured   unknown
c1::500601683021c60e           disk         connected    configured   unknown
c1::500601693021c60e           disk         connected    configured   unknown
c1::5006048c5368e580           disk         connected    configured   unknown
c1::5006048c5368e5a0           disk         connected    configured   unknown
c1::5006048c53697980           disk         connected    unconfigured unknown
c1::5006048c536979a0           disk         connected    configured   unknown
c1::50060e80102c41d0           disk         connected    configured   unknown
c1::50060e80102c41d2           unavailable  connected    unconfigured failed
c1::50060e80102c41d3           unavailable  connected    unconfigured failed
c2                             scsi-bus     connected    configured   unknown
c2::dsk/c2t2d0                 disk         connected    configured   unknown
c2::sd3                        disk         connected    unconfigured unknown   <<<<< access point state changes
c3                             scsi-bus     connected    unconfigured unknown
usb0/1                         unknown      empty        unconfigured ok
usb0/2                         unknown      empty        unconfigured ok

5.]  Refresh VxVM and /etc/vx/disk.info file

# vxdisk scandisks

# vxddladm assign names

NO DCPA message reported at this time.

Note: VxVM disk entry for the problematic disk ( ie disk_1 ), should no longer be presented to “vxdisk list”.

# vxdisk list
DEVICE       TYPE            DISK         GROUP        STATUS
disk_0       auto:sliced     c2t2d0       rootdg       online
emc_clariion0_83 auto:cdsdisk    emc_clariion0_83  rockdg       online
emc_clariion0_84 auto:cdsdisk    emc_clariion0_84  rockdg       online
-            -         replacedisk  rootdg       removed was:disk_1

6.] Reconfigure access point using

# cfgadm -c configure c2::sd3

# cfgadm -al
Ap_Id                          Type         Receptacle   Occupant     Condition
c0                             scsi-bus     connected    configured   unknown
c0::dsk/c0t0d0                 CD-ROM       connected    configured   unknown
c1                             fc-fabric    connected    configured   unknown
c1::200000017301bbbd           unknown      connected    unconfigured unknown
c1::500601603021c60e           disk         connected    configured   unknown
c1::500601613021c60e           disk         connected    configured   unknown
c1::500601683021c60e           disk         connected    configured   unknown
c1::500601693021c60e           disk         connected    configured   unknown
c1::5006048c5368e580           disk         connected    configured   unknown
c1::5006048c5368e5a0           disk         connected    configured   unknown
c1::5006048c53697980           disk         connected    unconfigured unknown
c1::5006048c536979a0           disk         connected    configured   unknown
c1::50060e80102c41d0           disk         connected    configured   unknown
c1::50060e80102c41d2           unavailable  connected    unconfigured failed
c1::50060e80102c41d3           unavailable  connected    unconfigured failed
c2                             scsi-bus     connected    configured   unknown
c2::dsk/c2t0d0                 disk         connected    configured   unknown   <<<<<<< acesss point configured with NEW replaced disk
c2::dsk/c2t2d0                 disk         connected    configured   unknown
c3                             scsi-bus     connected    unconfigured unknown
usb0/1                         unknown      empty        unconfigured ok
usb0/2                         unknown      empty        unconfigured ok

7.] luxadm online of the path(s)

#  luxadm -e online /dev/dsk/c2t0d0s2

8.] Refresh Solaris device tree with NEW disk details

# devfsadm –Cvc disk

9.] Update VxVM

# vxdisk scandisks

10.]  vxdisk list

# vxdisk list
DEVICE       TYPE            DISK         GROUP        STATUS
disk_0       auto:sliced     c2t2d0       rootdg       online
disk_1       auto            -            -            nolabel      <<<<<<    NEW disk now visible
emc_clariion0_83 auto:cdsdisk    emc_clariion0_83  rockdg       online
emc_clariion0_84 auto:cdsdisk    emc_clariion0_84  rockdg       online
-            -         replacedisk  rootdg       removed was:disk_1

Issue corrected. No further DCPA messages are reported.