VxVM disks marked with error "Disk is not usable" or "No such file or directory" during EMC BCV operations

book

Article ID: 100006680

calendar_today

Updated On:

Description

Error Message

# vxdisk online hdiskpower20      
VxVM vxdisk ERROR V-5-1-531 Device hdiskpower20: online failed:
       No such file or directory

# vxdisk list hdiskpower20
Device:    hdiskpower20
devicetag: hdiskpower20
type:      auto
flags:     error private autoconfig      <<< error
pubpaths:  block=/dev/vx/dmp/hdiskpower20 char=/dev/vx/rdmp/hdiskpower20
guid:      {8ab424de-1dd2-11b2-873a-72c72caa3767}
udid:      EMC%5FSYMMETRIX%5F000292601383%5F60000970000292601383533030434533
site:      -
errno:     No such file or directory      <<<
Multipathing information:
numpaths:   1
hdiskpower21    state=disabled
 

 

# vxdctl enable
VxVM vxdisk ERROR V-5-1-531 Device hdiskpower20: online failed:
       Disk is not usable

# vxdisk list hdiskpower20
Device:    hdiskpower20
devicetag: hdiskpower20
type:      auto
flags:     error private autoconfig             <<< error
pubpaths:  block=/dev/vx/dmp/hdiskpower23 char=/dev/vx/rdmp/rhdiskpower20
guid:      {8ab424de-1dd2-11b2-873a-72c72caa3767}
udid:      EMC%5FSYMMETRIX%5F000292601383%5F60000970000292601383533030434533
site:      -
errno:     Disk is not usable      <<< 
Multipathing information:
numpaths:   1
hdiskpower20    state=disabled

Cause

When a BCV LUN is being established, I/O to the LUN will fail.   If "vxdisk online " is run trying to online the VxVM disk corresponding to the BCV LUN, DMP will disable the DMP paths and VxVM will mark the VxVM disk as "No such file or directory".

The following error messages are logged in the /etc/vx/dmpevents.log file.

Wed Sep 14 17:54:41.494: I/O error occurred on Path hdiskpower20 belonging to Dmpnode hdiskpower20
Wed Sep 14 17:54:41.497: I/O analysis done on Path hdiskpower20 belonging to Dmpnode hdiskpower20
Wed Sep 14 17:54:41.497: SCSI error occurred on Path hdiskpower20: opcode=0x28 reported device not ready (status=0x2, key=0x2, asc=0x4, ascq=0x3)
Wed Sep 14 17:54:41.497: Disabled Path hdiskpower20 belonging to Dmpnode hdiskpower20
Wed Sep 14 17:54:41.497: Disabled Dmpnode hdiskpower20
Wed Sep 14 17:54:41.497: I/O analysis done on Path hdiskpower20 belonging to Dmpnode hdiskpower20
Wed Sep 14 17:54:41.498: Marked as failing Path hdiskpower20 belonging to Dmpnode hdiskpower20

After the VxVM disk was marked with error "No such file or directory", if there is another attempt to online the disk in this state (e.g. by running vxdctl enable), then VxVM will in turn mark the disk with error "Disk is not usable".  (DMP will not log further messages in /etc/vx/dmpevents.log this time.)

 

Resolution

In order to prevent accidental access to the BCV LUNs under BCV establishment, it is advised to offline the BCV LUNs before the establishment.   Please run the following command to offline the VxVM disk corresponding to the BCV LUNs before establishing the BCV LUNs.

# vxdisk offline hdiskpower20
# vxdisk offline hdiskpower21
....

Then establish the BCV LUNs, wait for them to be synchronized with Standard Device, and then split the BCV LUNs.

# symmir -g emcdiskgroup establish

# symmir -g emcdiskgroup query

# symmir -g emcdiskgroup split

# symmir -g emcdiskgroup verify -split

After the BCV LUNs are verified to be split correctly, then run the following command to online VxVM disks offlined previously.

# vxdisk online hdiskpower20
# vxdisk online hdiskpower21

Additional Notes:

When a disk is marked "No such file or directory", the underneath DMP paths are disabled.   The DMP paths will be restored by the DMP Restore Daemon which runs periodically according to the dmp_restore_interval setting (default is 300 seconds).

# vxdmpadm gettune | grep restore
dmp_enable_restore                       on           on
dmp_restore_policy                check_all check_disabled
dmp_restore_interval                    300          300
dmp_restore_cycles                       10           10

After the BCV LUNs are split and become accessible again, the "Disk is not usable" error can be fixed by running "vxdctl enable".

 

Applies To

Veritas Volume Manager (VxVM) with EMC Business Continuance Volume (BCV) environment.

Issue/Introduction

VxVM disks marked with the error "Disk is not usable" or "No such file or directory" during EMC BCV operations