VMware reporting SCSI reservation conflicts and incorrect number of fencing keys per node until perennially-reserved flag is set to true

book

Article ID: 100058966

calendar_today

Updated On:

Description

Error Message

Sample oiutput

The directory /var/VRTSvcs/log/vxfen/ contains the Veritas vxfen related log files.

# cat vxfen.server101
Reading SCSI Registration Keys...

Device Name: /dev/vx/rdmp/c_7000_1
Total Number Of Keys: 8
key[0]:
[Numeric Format]: 86,70,54,54,67,68,48,49
[Character Format]: VF66CD01
* [Node Format]: Cluster ID: 26317 Node ID: 1 Node Name: server101
key[1]:
[Numeric Format]: 86,70,54,54,67,68,48,49
[Character Format]: VF66CD01
* [Node Format]: Cluster ID: 26317 Node ID: 1 Node Name: server101
key[2]:
[Numeric Format]: 86,70,54,54,67,68,48,49
[Character Format]: VF66CD01
* [Node Format]: Cluster ID: 26317 Node ID: 1 Node Name: server101
key[3]:
[Numeric Format]: 86,70,54,54,67,68,48,49
[Character Format]: VF66CD01
* [Node Format]: Cluster ID: 26317 Node ID: 1 Node Name: server101
key[4]:
[Numeric Format]: 86,70,54,54,67,68,48,48
[Character Format]: VF66CD00
* [Node Format]: Cluster ID: 26317 Node ID: 0 Node Name: server102
key[5]:
[Numeric Format]: 86,70,54,54,67,68,48,48
[Character Format]: VF66CD00
* [Node Format]: Cluster ID: 26317 Node ID: 0 Node Name: server102
key[6]:
[Numeric Format]: 86,70,54,54,67,68,48,48
[Character Format]: VF66CD00
* [Node Format]: Cluster ID: 26317 Node ID: 0 Node Name: server102
key[7]:
[Numeric Format]: 86,70,54,54,67,68,48,48
[Character Format]: VF66CD00
* [Node Format]: Cluster ID: 26317 Node ID: 0 Node Name: server102

 

Cause


There is a device flag called “Perennially Reserved” which tells the host that RDM device should not be scanned and is used elsewhere (perennially) in the environment.

Before vSphere 7, this flag is enabled via CLI and requires the UUID (naa.ID).

To validate create a file called /tmp/disk on each host and populate the file with the corresponding DMPNODE names as follows:

Example


# cat /tmp/disk
/dev/vx/rdmp/c_7000_0
/dev/vx/rdmp/c_7000_1
/dev/vx/rdmp/c_7000_2
/dev/vx/rdmp/c_7000_3

 

- Save the file and run vxfenadm commands as per below on each node:
 

# vxfenadm -r all -f /tmp/disk >> /tmp/vxfen.`hostname`
# vxfenadm -s all -f /tmp/disk >> /tmp/vxfen.`hostname`

 

Resolution


Veritas recommends contacting VMware Support for guidance surrounding setting the VMware “Perennially Reserved” device flag.

VMware recommends implementing perennial reservation for all ESXi hosts hosting VM nodes with pRDMs.

The setting uses SCSI-3 persistent reservation to control locking between the nodes and blocks the hosts from being able to read them.

VMware article
https://kb.vmware.com/s/article/1016106
 

Issue/Introduction


VMware environments may report an incorrect number of fencing keys per node for a single pathed device.

In this instance, the DMPNODE is reporting 4 keys for a single pathed devices, instead of 1 key per node.