Cluster Volume Manager (CVM) join fails if vxdiskadm is used to prevent multipathing of all paths
Article ID: 100005399
Updated On:
Description
Error Message
Feb 21 13:19:10 sys2 daemon:notice Had[389322]: VCS ERROR V-16-1-1005 (sys2) CVMCluster:???:monitor:node
- state: out of cluster reason: Cannot find disk on slave node: retry to add a node failed
vxconfigd logged the following error messages.
VxVM vxconfigd NOTICE V-5-1-7900 CVM_VOLD_CONFIG command received
VxVM vxconfigd NOTICE V-5-1-7899 CVM_VOLD_CHANGE command received
VxVM vxconfigd WARNING V-5-1-8222 slave: missing disk 1257674215.116.c7prd04
VxVM vxconfigd WARNING V-5-1-7830 cannot find disk 1257674215.116.c7prd04
VxVM vxconfigd ERROR V-5-1-11092 cleanup_client: (Cannot find disk on slave node) 222
VxVM vxconfigd ERROR V-5-1-11467 kernel_fail_join() : Reconfiguration interrupted: Reason is retry to add a node failed (13, 0)
VxVM vxconfigd NOTICE V-5-1-7901 CVM_VOLD_STOP command received
Error code 222 is defined as:
#define VE_CLUSTER_DB 222 /* Cannot find disk on slave node */
Cause
Customer used vxdiskadm to prevent multipathing of all paths because customer preferred to use AIX MPIO instead of Veritas Dynamic Multipathing (VxDMP).
Please note the use of MPIO in Veritas Volume Manager (VxVM) environment is not recommended by Symantec. Please refer to the Veritas Storage Foundation and High Availability Solutions Hardware TechnNote (SymWISE article TECH47728) for details of the supportability of AIX MPIO.
output of vxdmpadm listenclosure all
-------------------------------------
ENCLR_NAME ENCLR_TYPE ENCLR_SNO STATUS ARRAY_TYPE LUN_COUNT
=====================================================================================
other_disks OTHER_DISKS OTHER_DISKS CONNECTED OTHER_DISKS 70
As all paths were prevented from multipathing through DMP, the udid will contain disk name, and it would cause problem on cvm join as the udid becomes different for same disk.
$ grep udid vxdisk_list_hdisk67
udid: IBM%5F2107900%5FOTHER%5FDISKS%5Fc7prd04%5F%2Fdev%2Frhdisk67
^^^^^^
$ grep udid vxdisk_list_hdisk70
udid: IBM%5F2107900%5FOTHER%5FDISKS%5Fc7prd03%5F%2Fdev%2Frhdisk70
^^^^^^
Resolution
If customer insists on using AIX MPIO (which is not recommended by Veritas), then DMP should at least be configured as JBOD, so udid will get disk serial number instead of disk name.
Steps to change to JBOD mode:
(1) exclude the Array Support Library (ASL) for the corresponding diskarray. Please note that if the disks are under the control of AIX MPIO, the VxDMP ASL should not be used.
# vxddladm excludearray libname=
(2) add array as jbod
# vxddladm addjbod vid=
(3) allow dmp
# vxdiskadm
17 Allow multipathing/Unsuppress devices from VxVM's view
6 Allow multipathing of a disk by VxVM
Enter a pathname or pattern : [
output of vxdmpadm listenclosure all
---------------------------------------------------
ENCLR_NAME ENCLR_TYPE ENCLR_SNO STATUS ARRAY_TYPE LUN_COUNT
=====================================================================================
disk Disk DISKS CONNECTED Disk 70
Applies To
AIX 5.3 / SFCFS5.0MP3 / MPIO / DS8K
Issue/Introduction
