Error Message

The following errors are reported in engine_A.log

YYYY/MM/DD HH:MM:SS VCS NOTICE V-16-1-10301 Initiating Online of Resource my_dg (Owner: unknown, Group: MYSG) on System my_server01
YYYY/MM/DD HH:MM:SS VCS WARNING V-16-10021-1074 (my_server01) CVMVolDg:my_dg:online:setup_vxnotify: old vxnotify of pid 4332 will be killed. my pid is 4356
YYYY/MM/DD HH:MM:SS VCS ERROR V-16-10021-1068 (my_server01) CVMVolDg:my_dg:online:cvmvoldg_get_volstate: error running vxprint command for volume sw
YYYY/MM/DD HH:MM:SS VCS ERROR V-16-10021-1011 (my_server01) CVMVolDg:my_dg:online:enable_vol: failed to enable volume sw in dg my_dg. vxvol returned 11
YYYY/MM/DD HH:MM:SS VCS ERROR V-16-10021-1068 (my_server01) CVMVolDg:my_dg:online:cvmvoldg_get_volstate: error running vxprint command for volume sw
YYYY/MM/DD HH:MM:SS VCS ERROR V-16-10021-1068 (my_server01) CVMVolDg:my_dg:online:cvmvoldg_get_volstate: error running vxprint command for volume sw
YYYY/MM/DD HH:MM:SS VCS ERROR V-16-10021-1068 (my_server01) CVMVolDg:my_dg:online:cvmvoldg_get_volstate: error running vxprint command for volume sw
YYYY/MM/DD HH:MM:SS VCS ERROR V-16-10021-1068 (my_server01) CVMVolDg:my_dg:online:cvmvoldg_get_volstate: error running vxprint command for volume sw
YYYY/MM/DD HH:MM:SS VCS ERROR V-16-10021-1068 (my_server01) CVMVolDg:my_dg:online:cvmvoldg_get_volstate: error running vxprint command for volume sw
YYYY/MM/DD HH:MM:SS VCS ERROR V-16-10021-1071 (my_server01) CVMVolDg:my_dg:online:cvmvoldg: Failed to enable state of volume sw in dg
YYYY/MM/DD HH:MM:SS VCS ERROR V-16-10021-1026 (my_server01) CVMVolDg:my_dg:online:checkall: startall failed
YYYY/MM/DD HH:MM:SS VCS ERROR V-16-10021-1045 (my_server01) CVMVolDg:my_dg:online:Initial check failed
YYYY/MM/DD HH:MM:SS VCS INFO V-16-2-13001 (my_server01) Resource(my_dg): Output of the completed operation (online)
VxVM vxprint ERROR V-5-1-924 Record sw not found
iter=0
VxVM vxprint ERROR V-5-1-924 Record sw not found
iter=1
VxVM vxprint ERROR V-5-1-924 Record sw not found
iter=2
VxVM vxprint ERROR V-5-1-924 Record sw not found
iter=3
VxVM vxprint ERROR V-5-1-924 Record sw not found
iter=4
VxVM vxprint ERROR V-5-1-924 Record sw not found
YYYY/MM/DD HH:MM:SS VCS ERROR V-16-2-13066 (my_server01) Agent is calling clean for resource(my_dg) because the resource is not up even after online completed.

The issue is caused by the way that the CVMVolume attribute is modified using VCS Console resulting in an invalid entry in the VCS configuration file main.cf (see below).

        CVMVolDg my_dg (
                Critical = 1
                CVMDiskGroup = mydg
                CVMVolume = { data01vol, data02vol, data03vol, redo01vol,
                         redo02vol,
                         "",
                         redo07vol}
                CVMActivation = sw
                )
 

To fix the issue, implement the procedure below.

1) Edit the main.cf file on one of the VCS nodes to remove the line containing "",.

2) Verify the main.cf file edited by running

#/opt/VRTS/bin/hacf -verify /etc/VRTSvcs/conf/config

3) Copy the updated the main.cf file to the other nodes in the cluster

4) Restart Veritas High Availability Deamon (HAD) on all cluster nodes..

NOTE: HAD can be stopped with the -force switch to allow all the Service groups to be in Online state while HAD is stopped..