After a BCV establish, when vxdisk scandisks command is run, all vx commands hang due to vxconfigd daemon stuck in biowait.

book

Article ID: 100027037

calendar_today

Updated On:

Description

Error Message

Stack trace of vxconfigd process:

Stack trace for process "vxconfigd" at 0xe0000004f7ba7a80 (pid 1242)
Thread at 0xe0000003beee7380 (tid 1461134)
#0  0xe00000000103eef0:0 in slpq_swtch_core+0x520 ()
   at /ux/core/kern/common/pm/sched/sched_sleep.c:3418
#1  0xe000000000fbcd10:0 in inline real_sleep+0x200 ()
   at /ux/core/kern/common/pm/sched/sched_sleep.c:747
#2  0xe000000000fbcb10:0 in sleep_pc+0x60 ()
   at /ux/core/kern/common/pm/sched/sched_sleep.c:591
#3  0xe000000000f4bda0:0 in inline biowait_rp+0x110 ()
   at /ux/core/kern/common/fs/vfs/vfs_bio.c:2161
#4  0xe000000000f4bc90:0 in biowait+0x80 ()
   at /ux/core/kern/common/fs/vfs/vfs_bio.c:2113
#5  0xe000000002adbde0:0 in dmp_indirect_io+0x9b0 ()
   at ../vxdmp/dmpioctl.c:4020
#6  0xe000000002b225a0:0 in gendmpioctl+0x1230 ()
   at ../vxdmp/dmpdev.c:2582
#7  0xe000000002b1d5c0:0 in dmpioctl+0xe0 ()
   at ../vxdmp/dmpdev_sys.c:295
#8  0xe000000000eb7c10:0 in spec_ioctl+0xf0 ()
   at /ux/core/kern/common/fs/misc/spec_vnops.c:784
#9  0xe000000000f25850:0 in vno_ioctl+0x350 ()
   at /ux/core/kern/common/fs/vfs/vfs_io.c:223
#10 0xe000000000f5af50:0 in ioctl+0x410 ()
   at /ux/core/kern/common/fs/vfs/sys_gen.c:694
#11 0xe000000001030f40:0 in syscall+0x590 ()
   at /ux/core/kern/common/svc/scall/syscall.c:1739
#12 0xe000000121ccc480 in ?? ()

Following message seen in syslog:

NOTICE: VxVM vxdmp V-5-3-0 gendmpstrategy: strategy call failed on bp 0xe0000007200af080, path devno 255/4000

Cause

The issue is seen only when all these conditions are met:

  • HP-UX  11.31 on IA  architecture
  • VxVM 5.1 and above
  • dmp_cache_open tunable is set to OFF
  • Atleast a BCV pair in established state

vxconfigd can hang in biowait in the following 2 scenarios:

  1. vxdisk scandisks command can trigger a device discovery and vxconfigd hangs when it tries to do I/O on a BCV device that's in established state (BCV-NR).
  2. During system boot, if there's a BCV pair that's in established state, vxconfigd can hang. Every subsequent reboots will cause vxconfigd hang, until all the BCV pairs are split.

This issue has been identified to be a bug in VxVM and the bugid is: 2515369.

Resolution

This bug is triggered only if dmp_cache_open tunable is set to OFF. To check the value of this setting, the following command is run:
 

# vxdmpadm gettune dmp_cache_open
Tunable                Current Value      Default Value
----------------       -----------      -------------
dmp_cache_open                off           on
#

The issue is resolved in 5.1SP1RP1HF3. Please contact Veritas Technical support to obtain the hotfix patch and provide the bugid: 2515369 as reference.

Workaround:

Set the dmp_cache_open to ON. The changes are immediate and doesn't require a system reboot.

# vxdmpadm settune dmp_cache_open=on

Applies To

VxVM  5.1  SP1RP1
HP-UX  11.31  IA  architecture
dmp_cache_open tunable set to OFF

Issue/Introduction

EMC  Business Continuance Volume (BCV)  is used for backups. After a BCV  split command, vxdisk scandisks command is run to enable access to BCV  devices. This triggers a device discovery by vxconfigd daemon and it hangs with the error messages listed below. All vx commands hang because vxconfigd is hung. When system is rebooted with BCV  devices in an established state, vxconfigd hangs with the same stack.
Powered by Wolken Software