Verifying SCSI3-PGR compliance for VCS vxfen: Preliminary Not-Comprehensive Manual Test
Article ID: 100026270
Updated On:
Resolution
===========Initial setup for the VCS vxfen “Preempt Game”
1.0 On both nodes hostA and hostB you will need to select a single free currently not-used LUN from the array for your SCSI3-PGR testing.
---- Next you need to establish a file with a single device listed. In this example we’ve created a file “/tmp/diskfile”
---- For Solaris inside this file place the OS ”CTDs2” device path designation: /dev/rdsk/c2t50060E80104EF3F1d3s2
---- For Linux inside this file place the OS /dev/ device path designation: /dev/
---- (For this test we are using one of two paths; you may choose to test both paths by placing both device paths into the diskfile)
---- Alternatively you may also use the VxVM “vxdmpnode” designation too; for example:
--for Solaris: /dev/vx/rdmp/c2t50060E80104EF3F1d3s2
--for Linux : /dev/san_vc0_0s3
---- Note: The examples below are from the Solar environment but are analogous in other OS environments.
======== Establish both host registration and write-exclusive keys
2.0 Let’s make hostA register a key to get into the “preempt game”
hostA# vxfenadm -m -k A1 -f /tmp/diskfile
Registration completed for disk path: /dev/rdsk/c2t50060E80104EF3F1d3s2
----Let’s also make hostB register a key to get into the “preempt game”
hostB# vxfenadm -m -k B2 -f /tmp/diskfile
Registration completed for disk path: /dev/rdsk/c2t50060E80104EF3F1d3s2
----Now confirm both host A and host B are registered for the ”preempt game”
Host[A|B]# vxfenadm -g all -f /tmp/diskfile
Device Name: /dev/rdsk/c2t50060E80104EF3F1d3s2
Total Number Of Keys: 2
key[0]:
Key Value [Numeric Format]: 65,49,45,45,45,45,45,45
Key Value [Character Format]: A1------
key[1]:
Key Value [Numeric Format]: 66,50,45,45,45,45,45,45
Key Value [Character Format]: B2------
------note that the key is always an eight byte key or ascii character; the dashes are pad characters
=========Game Rule: “only one write exclusive key on the LUN at a time”
3.0 Now we place “WRITEEXCLUSIVEREGISTRANTSONLY “ reservation key on the LUN
------on hostA let’s establish its key “A1” reservation key on the LUN
HOST# vxfenadm -n -k A1 -f /tmp/diskfile
VXFEN vxfenadm NOTICE Reservation completed for disk path: /dev/rdsk/c2t50060E80104EF3F1d3s2
------Can hostB establish its “B2” write-exclusive reservation key on the disk group LUNs?
hostB# vxfenadm -n -k B2 -f /tmp/diskfile
VXFEN vxfenadm ERROR V-11-2-1118 Reservation FAILED for: /dev/rdsk/c2t50060E80104EF3F1d3s2
VXFEN vxfenadm ERROR V-11-2-1133 Error returned
----- hostB cannot because hostA has established its write-exclusive key on the LUN
------on hostB we see hostA’s key is on the disk group LUNs
hostB# vxfenadm -r all -f /tmp/diskfile
Device Name: /dev/rdsk/c2t50060E80104EF3F1d3s2
Total Number Of Keys: 1
Key[0]:
Reservation Type: SCSI3_RESV_WRITEEXCLUSIVEREGISTRANTSONLY
Key Value [Numeric Format]: 65,49,45,45,45,45,45,45
Key Value [Character Format]: A1------
===== From hostB let’s prempt hostA’s key “A1” and replace it with hostB “B2” key
4.0 If vxfen needs to “preempt” another hosts reservation, we can do that
hostB# vxfenadm -p -k B2 -f /tmp/diskfile -v A1
Key: B2------ preempted the key: A1------ on disk /dev/rdsk/c2t50060E80104EF3F1d3s2
--------- now we see hostA has been bumped/preempted in the “preempt game”
hostB# vxfenadm -g all -f /tmp/diskfile
Device Name: /dev/rdsk/c2t50060E80104EF3F1d3s2
Total Number Of Keys: 1
key[0]:
Key Value [Numeric Format]: 66,50,45,45,45,45,45,45
Key Value [Character Format]: B2------
--------- we also see hostB has successfully replaced/preempted hostA’s key with hostB’s write-exclusive key, “B2”
hostB# vxfenadm -r all -f /tmp/diskfile
Device Name: /dev/rdsk/c2t50060E80104EF3F1d3s2
Total Number Of Keys: 1
Key[0]:
Reservation Type: SCSI3_RESV_WRITEEXCLUSIVEREGISTRANTSONLY
Key Value [Numeric Format]: 66,50,45,45,45,45,45,45
Key Value [Character Format]: B2------
======= hostA gets back again into the “preempt game”
5.0 If vxfen arbitration leads to preempting hostB, then hostA re-registers to get back into the “game”
hostA# vxfenadm -m -k A1 -f /tmp/diskfile
Registration completed for disk path: /dev/rdsk/c2t50060E80104EF3F1d3s2
-----we now see hostA is now back into the “game”
hostA# vxfenadm -g all -f /tmp/diskfile
Device Name: /dev/rdsk/c2t50060E80104EF3F1d3s2
Total Number Of Keys: 2
key[0]:
Key Value [Numeric Format]: 65,49,45,45,45,45,45,45
Key Value [Character Format]: A1------
key[1]:
Key Value [Numeric Format]: 66,50,45,45,45,45,45,45
Key Value [Character Format]: B2------
------Then vxfen arbitration may now allow vxfen to preempt hostB’s write-exclusive key
hostA# vxfenadm -p -k A1 -f /tmp/diskfile -v B2
Key: A1------ preempted the key: B2------ on disk /dev/rdsk/c2t50060E80104EF3F1d3s2
----------- hostA has successfully bumped hostB and establishes hostA’s key on the LUNs
hostA# vxfenadm -r all -f /tmp/diskfile
Device Name: /dev/rdsk/c2t50060E80104EF3F1d3s2
Total Number Of Keys: 1
Key[0]:
Reservation Type: SCSI3_RESV_WRITEEXCLUSIVEREGISTRANTSONLY
Key Value [Numeric Format]: 65,49,45,45,45,45,45,45
Key Value [Character Format]: A1------
======= Exercise Conclusion
6.0 If you are able to perform the sequence depicted above of host registration and key insert and preempt write-exclusive keys back and forth, then this is an indication that your array is SCSI3-PGR capable.
======= Clean up after testing
7.0 To remove both host registration key and the write-exclusive keys from the LUN you may use these commands:
hostA# vxfenadm –x –k A1 –f /tmp/diskfile
-----this must be done from hostA to remove both hostA keys
hostB# vxfenadm –x –k B2 –f /tmp/diskfile
-----this must be done from hostB to remove both hostA keys
-----for other vxfenadm command options, refer to Chapter 5 of the Cluster Server Installation Guide
Applies To
VCS Fencing with SCSI-3 PGR disks
Issue/Introduction
The most comprehensive VCS I/O fencing (vxfen) test for SCSI3-PGR compliance for a storage array is the use of the /opt/VRTSvcs/vxfen/bin/vxfentsthdw utility. The vxfentsthdw utility is an elaborate script used to simulate all vxfen SCSI3-PGR operations in the management of fencing keys on LUNs along with the vxfen preempt arbitration operation to avoid VCS disk group import concurrency violations leading to file system corruption. The vxfentsthdw also requires node-to-node root-no-password-prompt rsh or ssh in order to perform comprehensive all-node vxfen tests on targeted LUNs exercising SCSI3-PGR. Please see Chapter 5 “Setting up I/O fencing” of the Custer Server Installation Guide for further I/O fencing (vxfen) and vxfentsthdw details.
If the host/cluster node environment has restrictions on the configuration rsh and ssh, then you may consider using the procedure below as non-comprehensive test to at least confirm some likelihood that the array meets SCSI3-PRG compliance for targeted LUNs from the array.
The vxfenadm commands below on each node (2 node cluster) and the sequence depicted will walk you through the type a vxfen would do to insert SCSI3-PGR keys or to “preempt” another nodes keys during an vxfen arbitration actions. I call this the “Preempt Game”.
Was this article helpful?
Yes
No
Powered by
