Volume manager commands hang/slow if BCV mirror is in established state on AIX platform
book
Article ID: 100000764
calendar_today
Updated On:
Description
e.g.# ./symmir -g bcvdg queryDevice Group (DG) Name: bcvdgDG's Type : REGULARDG's Symmetrix ID : 000290301414Standard Device BCV Device State-------------------------- ------------------------------------- ------------Inv. Inv.Logical Sym Tracks Logical Sym Tracks STD BCV-------------------------- ------------------------------------- ------------DEV001 0076 0 BCV001 0078 * 0 SynchronizedDEV002 0077 0 BCV002 0079 * 0 SynchronizedTotal ------- -------Track(s) 0 0MB(s) 0.0 0.0Legend:(*): The paired BCV device is associated with this group.#vxdisk -e -o alldgs listDEVICE TYPE DISK GROUP STATUS OS_NATIVE_NAME ATTRdisk_0 auto vvrdg01 vvrdg online hdisk1 -disk_1 auto - - LVM hdisk0 -emc1_0076 auto d1 bcvdg online hdisk16 stdemc1_0077 auto d2 bcvdg online hdisk17 stdemc1_0078 auto - - error hdisk18 bcvemc1_0079 auto - - error hdisk19 bcv#lsdev -Cc disk |egrep "hdisk18|hdisk19"hdisk18 Available 1Z-08-01 EMC Symmetrix FCP Raid5hdisk19 Available 1Z-08-01 EMC Symmetrix FCP Raid5
Workaround
1. For VERITAS Volume manager devices only
Offline VERITAS volume manager devices before "symmir establish" command gets called in customer's BCV script.
e.g.
#vxdisk offline emc1_0078#vxdisk offline emc1_0079
- Perform BCV establish and split operation
# ./symmir -g bcvdg split
#vxdisk online emc1_0078#vxdisk online emc1_0079
Above work-around will resolve MonitorTimeout issue for VERITAS volume manager Diskgroups in SF-HA cluster.
2. For VERITAS volume manager diskgroups and AIX LVM Volume groups or/only LVM Volume groups
Change BCV devices state from AVAILABLE to DEFINED before "symmir establish" command gets called in customer's BCV script.
e.g.
#vxdisk rm emc1_0078#vxdisk rm emc1_0079
- Change BCV device status from AVAILABLE to DEFINED using EMC rmbcv command
#/usr/lpp/EMC/Symmetrix/bin/rmbcv hdisk18#/usr/lpp/EMC/Symmetrix/bin/rmbcv hdisk19
- Perform BCV establish and spit operation
# ./symmir -g bcvdg establish -full# ./symmir -g bcvdg split
- Change BCV device status from DEFINED to AVAILABLE using EMC mkbcv command
# /usr/lpp/EMC/Symmetrix/bin/mkbcv -a
- Rescan VxVM devices and import BCV diskgroup
# vxdisk scandisks
Above work-around will resolve MonitorTimeout issue for both VERITAS volume manager Diskgroups and LVM volume groups in SF-HA cluster
- Additional notes on Veritas Cluster (VCS) Environment
If the system is under the control of VCS and DiskGroup resource is in use and DiskGroup resource attribute PanicOnDGLoss is set to 1 (which is the default value), the above problem can cause consecutive DiskGroup monitor timeouts. After 4 consecutive DiskGroup monitor timeouts, the DiskGroup agent will call the clean entry point to halt the system using "/usr/sbin/halt -q". Please note that the "halt -q" command will not generate any kernel core and will just halt the system. VCS engine log on this system may not contain the messages as listed in the Error section of this article. Please check the VCS engine log on the other nodes in the VCS cluster to confirm if the system was halted because of PanicOnDGLoss.
Resolution
When BCV devices are in established state it does not allow any ioctl from external application like VxVM/LVM which results in application command hang or system wide hang issue.
EMC is aware of this issue and they have documented issue under their Primus(R) eServer solution (emc115907) which provides procedure of full establish including rmbcv and mkbcv commands which marks the BCV luns in a "Defined" state as seen by lsdev -Cc disk command
On AIX plat form, customer must need to change BCV device status to "Defined" state before calling BCV establish script
How to check whether BCV devices are in established state or not
Issue/Introduction
Volume manager commands hang/slow if BCV mirror is in established state on AIX platform
Was this article helpful?
thumb_up
Yes
thumb_down
No