Slave node may fail to join CVM membership and report missing disk when a shared diskgroup is initially imported with a CLONE type and the clone_disk flags have been turned off following the import event
Article ID: 100013948
Updated On:
Description
Error Message
The CVM MASTER server is not reporting any udid_mismatch state for any of the imported shared diskgroup. The clone_disk flag has been turned off following the import operation for the shared diskgroup on the MASTER server.
The SLAVE server is now attempting to obtain CVM membership.
Sample error messages seen from the SLAVE server:
Sep 2 13:14:48 berty vxvm: vxconfigd: V-5-1-16760 Disk Group sharedg has a mix of standard and cloned disks: Trying to import the disk group from the clone disks because of the useclonedev=on option is specified
Sep 2 13:14:48 berty vxvm: vxconfigd: V-5-1-16760 Disk Group sharedg has a mix of standard and cloned disks: Trying to import the disk group from the clone disks because of the useclonedev=on option is specified
.
.
Sep 2 13:14:56 berty vxvm: vxconfigd: V-5-1-16751 import_start: disk 1379712922.1219.berty (udid ) not found, flags 0x808
Sep 2 13:14:56 berty vxvm: vxconfigd: V-5-1-16751 import_start: disk 1379712922.1219.berty (udid ) not found, flags 0x808
.
NOTE: The disk id 1379712922.1219.berty is visible from both the MASTER and SLAVE servers. Both servers can see the same physical devices.
.
.
Sep 2 13:14:56 berty vxvm: vxconfigd: V-5-1-16751 import_start: disk 1379712922.1219.berty (udid ) not found, flags 0x808
Sep 2 13:14:56 berty vxvm: vxconfigd: V-5-1-16751 import_start: disk 1379712922.1219.berty (udid)
Sep 2 13:14:56 berty vxvm: vxconfigd: V-5-1-16751 import_start: disk 1379712922.1219.berty (udid ) not found, flags 0x808
Sep 2 13:14:56 berty vxvm: vxconfigd: V-5-1-16751 import_start: disk 1379712922.1219.berty (udid)
Sep 2 13:16:09 berty Had[6156]: VCS ERROR V-16-20006-1005 (berty) CVMCluster:cvm_clus:monitor:node - state: out of cluster
.
Sep 2 13:16:09 berty reason: Disk for disk group not found: retry to add a node failed
Sep 2 13:16:09 berty Had[6156]: VCS ERROR V-16-20006-1005 (berty) CVMCluster:cvm_clus:monitor:node - state: out of cluster
Sep 2 13:16:09 berty reason: Disk for disk group not found: retry to add a node failed
Cause
This issue only applies to shared diskgroups. Private diskgroups are not impacted.
As the diskgroup was initially imported with a "clone" type. It is not enough to turn off the "clone_disk" flag.
Resolution
Workaround:
1.] In the event that the SLAVE node fails to detect the devices correctly, the clone_disk flag can be set against the corresponding disks in the impacted diskgroup.
# vxdisk set
Once the clone_disk flag has been set for the respective disks, proceed to start CVM. The SLAVE node should now be able to obtain CVM membership and locate all the required disks.
2.] In order to correct the situation long term, the shared diskgroup must be deported.
Once the diskgroup is deported, ensure the "clone_disk" flag is turned off against all the corresponding disks associated with the shared diskgroup.
Now re-import the shared diskgroup using VCS on the MASTER server.
The shared diskgroup will no longer be imported with a diskgroup clone type. The diskgroup will now be classified with a standard diskgroup classification.
The SLAVE server should now be able to join the CVM cluster.
Applies To
Cross platform
Issue/Introduction
Even though the clone_disk flag has been turned off for all the disks in the associated imported diskgroup, the diskgroup type will remain classified as a "clone" diskgroup type. In the situation where the SLAVE node now needs to join the CVM cluster membership, it may fail to find the required disk devices as it is searching for "cloned" devices. Even though the clone_disk flag has been turned off at the disk level, the diskgroup classification type remains unchanged as "clone".
Thus, when scanning for the devices associated with the shared diskgroup, it fails to find a disk of a clone type and CVM membership fails.
