NIC Resource goes to faulted state with error "VCS ERROR V-16-1-54031"

book

Article ID: 100047811

calendar_today

Updated On:

Description

Cause:

Due to sudden network issue in customer's environment,sometimes NIC resource shows faulted and it gets online again in next minute.
Example messages as below:

xxxx/xx/xx 16:11:48 VCS ERROR V-16-1-54031 Resource NIC2 (Owner: Unspecified, Group: xxxxx) is FAULTED on sys xxxxx
xxxx/xx/xx 16:12:50 VCS INFO V-16-1-10299 Resource NIC2 (Owner: Unspecified, Group: xxxxx) is online on xxxxx(Not initiated by VCS)
xxxx/xx/xx 20:27:47 VCS ERROR V-16-1-54031 Resource NIC2 (Owner: Unspecified, Group: xxxxx) is FAULTED on sys xxxxx
xxxx/xx/xx 20:28:48 VCS INFO V-16-1-10299 Resource NIC2 (Owner: Unspecified, Group: xxxxx) is online on xxxxx(Not initiated by VCS)
xxxx/xx/xx 21:39:44 VCS ERROR V-16-1-54031 Resource NIC2 (Owner: Unspecified, Group: xxxxx) is FAULTED on sys xxxxx
xxxx/xx/xx 21:40:46 VCS INFO V-16-1-10299 Resource NIC2 (Owner: Unspecified, Group: xxxxx) is online on xxxxx(Not initiated by VCS)
xxxx/xx/xx 11:05:43 VCS ERROR V-16-1-54031 Resource NIC2 (Owner: Unspecified, Group: xxxxx) is FAULTED on sys xxxxx
xxxx/xx/xx 11:06:44 VCS INFO V-16-1-10299 Resource NIC2 (Owner: Unspecified, Group: xxxxx) is online on xxxxx(Not initiated by VCS)

Due to the above NIC Fault the IP resource dependent on it also goes faulted and creates outage.

Checked the customer's NIC resource configuration and found that there is no NetworkHosts added

NIC NIC1 (
                Critical = 0
                Device = aggr1
                NetworkType = ether
                )

 

Solution:

Suggested customer to implement NetworkHosts for NIC resource.

Below commands can be used for adding the NetworkHosts attribute to the configuration:

haconf -dump -makero 
hares -modify NIC1 NetworkHosts  < Highly available IP in the network mostly gateway >  <second IP (optional)>
haconf -dump -makerw

More Explanation of NetworkHosts is as below

List of hosts on the same network that are pinged to determine if the network connection is alive. Enter the IP address of the host, instead of the host name, to prevent the monitor from timing out. DNS lookup causes the ping to hang. If more than one network host is listed, the monitor returns ONLINE if at least one of the hosts is reachable.
If you do not specify network hosts, the monitor tests the NIC by sending pings to the broadcast address on the NIC.
For a virtual device, you must configure the NetworkHosts attribute. Veritas recommends configuring more than one host to take care of the NetworkHost itself failing.
Type and dimension: string-vector
Example: { "166.96.15.22", "166.97.1.2" }
 

Issue/Introduction

VCS ERROR V-16-1-54031 NIC resource goes into faulted state and due to which IP resource also goes offline and creates outage.