V-15-1-20037 - Gabconfig shows new node #3 in jeopardy. Node #3 was just added to the cluster.

book

Article ID: 100022433

calendar_today

Updated On:

Description

Error Message

LLT WARNING V-14-1-10140 rcv unexpected pkt (type 2, nid 3) in connwait_recv
GAB INFO V-15-1-20036 Port a gen b59216 membership 01234
GAB INFO V-15-1-20037 Port a gen b59216 jeopapardy ; 3


Resolution

Running/opt/VRTS/bin/lltstat -nvv on any of the original clusternodes  shows node 3 is down:
 
LLT nodeinformation:
   Node                State    Link  Status  Address
  * 0nyvmdev01         OPEN    
                                 bge2  UP      00:0B:5D:E5:54:48
                                 bge0  UP      00:0B:5D:E5:54:46
   1 nyvmdev02         OPEN    
                                 bge2  UP      00:0B:5D:E5:54:B0
                                 bge0  UP      00:0B:5D:E5:54:AE
   2 nyvmdev03         OPEN    
                                 bge2  UP      00:0B:5D:E5:52:CA
                                 bge0  UP      00:0B:5D:E5:52:C8
   3 nyvmdev04         OPEN    
                                 bge2  DOWN    00:0B:5D:E5:53:D6
                                 bge0  DOWN    00:0B:5D:E5:53:D4
   4 nyvmdev05         OPEN    
                                 bge2  UP      00:0B:5D:E5:50:78
                                 bge0  UP      00:0B:5D:E5:50:76
 
However,running /opt/VRTS/bin/lltstat -nvv on node #3 shows all nodes are up:
 
LLT nodeinformation:
   Node                State    Link  Status  Address
   0 nyvmdev01         OPEN    
                                 bge2  UP      00:0B:5D:E5:54:48
                                 bge0  UP      00:0B:5D:E5:54:46
   1 nyvmdev02         OPEN    
                                 bge2  UP      00:0B:5D:E5:54:B0
                                 bge0  UP      00:0B:5D:E5:54:AE
   2 nyvmdev03         OPEN    
                                 bge2  UP      00:0B:5D:E5:52:CA
                                 bge0  UP      00:0B:5D:E5:52:C8
  *3 nyvmdev04         OPEN    
                                 bge2  UP      00:0B:5D:E5:53:D6
                                 bge0  UP      00:0B:5D:E5:53:D4
   4 nyvmdev05         OPEN    
                                 bge2  UP      00:0B:5D:E5:50:78
                                 bge0  UP      00:0B:5D:E5:50:76
 
All configuration files on all nodes were found to be correct and there were no hardware problems found. The newly added node #3 was found to be running an older revision of Storage Foundation HA (SF-HA).  Upgrading node #3 to the same version of SF-HA  as the other nodes in the cluster resolved the problem. For proper operation, all nodes in a cluster must be running the same version and patch level of SF-HA.
 
 

 

Issue/Introduction

Gabconfig shows new node #3 in jeopardy. Node #3 was just added to the cluster.