Fixes to vxattachd and DMP path restoration kernel thread to avoid accessing EMC BCV LUNs in Not Ready (NR) state

book

Article ID: 100028414

calendar_today

Updated On:

Description

Error Message

The Veritas vxattachd daemon captures various DMP event message, like path enable. The vxattachd daemon tries to online the disks which are not in online state.

While doing it, commands like vxdisk scandisks, vxdisk online are executed.

As EMC BCV-NR (Hardware Clone) devices are always in a not-ready (NR) error state, whilst syncing with the source disk image. DMP will continuously try to probe and enable paths for BCV-NR devices. This can lead to slowness and unwanted execution of DMP and VxVM commands.

DMP path restoration kernel thread tries to enable the path which were disabled earlier. So it will attempt to enable the paths for BCV-NR devices as probes will succeed. 

The BCV paths will then be disabled afterwards during vxdisk scandisks as a open request will fail on these devices, because no I/Os are permitted on them (not-ready). 

The path restoration process goes in a loop where restoration kernel thread enables the paths of BCV-NR devices but scandisks will fail such paths due to the open failures. This generates continuous path enable/disable messages.
 

Cause

vxattachd checks column 7 of the "vxdisk -e list " output for the 'bcv-nr' attribute, but the actual output from "vxdisk -e list " output can vary. 

Column 7 doesn't necessarily list the "bcv-nr" attribute if the DMP node has additional status (e.g. thinrclm) and attributes.

DMP path restoration kernel thread checks if open system call is succeeding on the path before enabling it. 

But because of a bug in the open system call validation procedure, the validation is incorrectly passed through and paths were enabled. 
 

Resolution

vxattachd is changed to check for the "NR_DEVICE" property in "vxdisk -p list " output instead. 

For example,

# vxdisk -p list emc0_0fdd
DISK           : emc0_0fdd
VID            : EMC
UDID           : EMC%5FSYMMETRIX%5F000192600969%5F6900FDD000
SCSI_VERSION   : 4
REVISION       : 5874
PORT_SERIAL_NO : 7f-a
PID            : SYMMETRIX
PHYS_CTLR_NAME : 5/0/12/1/0/4/0.0x50000972080f2558
NR_DEVICE      : yes                   <<< vxattachd will now check if NR_DEVICE property is set to "yes" or not.
MEDIA_TYPE     : hdd
LUN_SNO_ORDER  : 400
LUN_SERIAL_NO  : 6900FDD000
LIBNAME        : libvxemc.sl
HARDWARE_MIRROR: yes
DMP_DEVICE     : emc0_0fdd
DDL_DEVICE_ATTR: bcv-nr
CAB_SERIAL_NO  : 000192600969
ATYPE          : A/A
ARRAY_VOLUME_ID: 0FDD
ARRAY_PORT_PWWN: 50:00:09:72:08:0f:25:58
ANAME          : EMC
TRANSPORT      : FC
 

DMP path restoration kernel threads “open” validation is fixed to properly check if the open request succeeds before enabling the paths.

For BCV-NR devices, as the open request will  now fail, the paths for the BCV devices in a not-ready (NR) state will not be enabled and skipped.

Once the NR state is cleared, the BCV image is split and available for import (read-write), the paths will be enabled and the VxVM disk state will change from "error" to "online".
 

 

Applies To

The problem affects VxVM on all platforms without the required fix, which is included in 5.1SP1RP3 
 

Issue/Introduction

Veritas Volume Manager (VxVM) vxattachd and DMP path restoration kernel thread access BCV LUNs in Not Ready (NR) state and cause continuous DMP path enabling and disabling.
The Veritas Volume Manager (VxVM) vxattachd daemon is onlining previously offlined disks. vxattachd is a Volume Manager daemon which interacts with vxnotify to retrieve notification events.
The vxattachd script is located in /opt/VRTS/bin/vxattachd.

vxnotify is a notification vehicle that can retrieve events from vxconfigd.
Once the event has been processed by vxnotify, the vxattachd daemon has something to process.

Additional Information

ETrack: 3038684