Error Message

in /var/VRTSvcs/log/vxfen/vxfen.log :
VXFEN vxfenconfig ERROR V-11-2-1041 Snapshot for this node is different from that of the running cluster.

in messages file :
kernel: VXFEN ERROR V-11-1-11 Snapshot different. Dropping out of cluster.VXFEN INFO V-11-1-35 Fencing driver going into RUNNING state

vxfenconfig -l shows you what is currently running :

# vxfenconfig -l

I/O Fencing Configuration Information:
======================================

security=0
fips_mode=0
single_cp=0
[192.168.10.114]:14250 {97a1c15e-1dd1-11b2-9018-03750f5baaf6}
/dev/vx/rdmp/emc0_dd2  5123456000000001
/dev/vx/rdmp/emc0_dd3  5123456000000002


Now, on the node on which you are trying to start VXFEN and failed, have a look at /var/VRTSvcs/log/vxfen/vxfen.log :
Fri Apr 24 01:46:10 PDT 2015 executing local_info.sh: begin
Fri Apr 24 01:46:12 PDT 2015 output was security=0
fips_mode=0
single_cp=0
[192.168.10.114]:14250 {4b4ab030-1dd2-11b2-9698-957e5a1f7f1f}
/dev/vx/rdmp/emc0_dd2  5123456000000001
/dev/vx/rdmp/emc0_dd3  5123456000000003
Fri Apr 24 01:46:12 PDT 2015 executing local_info.sh: end
Fri Apr 24 01:46:12 PDT 2015 calling regular vxfenconfig
Fri Apr 24 01:46:23 PDT 2015 return value from above operation is 1
Fri Apr 24 01:46:23 PDT 2015 output was VXFEN vxfenconfig ERROR V-11-2-1041 Snapshot for this node is different from that of the running cluster.

In that case we have 2 differences :
-the uuid of the CP server is different
-one of the disk id is different.

FYI, the uuid of the CP server, can be found on the CP server /etc/VRTScps/db/current/cps_uuid

FYI, the disk id may changed if customer is using scsi3_disk_policy=raw and the device name has changed after that vxfen started.

At this stage, it is more adequate to stop fencing on all nodes an restart it.

Note that when you stop fencing on all nodes, you shouldn't have any scsi3-PR keys.
If you still have keys, you may need to remove them before starting vxfen again.

To see scsi3-PR keys on a disk :
# vxfenadm -s /dev/sdX
or
# vxfenadm -s /dev/vx/rdmp/emc0_dd2

To remove keys, create a file with the disk device names
# cat /tmp/disks
/dev/sdbj
/dev/sdc

And remove the all keys on theses disks:
# vxfenadm –a –k tmp –f /tmp/disks
# vxfenadm –c –k tmp –f /tmp/disks

When VXFEN started on the first node it used specific coordinator disks or/and CP servers. Now if you try to start VXFEN on the second node and either disks and/or CP servers have changed, it won't start because it is different from the what is currently running on the first node.