Mount and IP resource states are showing offline in Veritas Infoscale Operations Manager GUI (VIOM) when they are available at the server level.
Article ID: 100042345
Updated On:
Description
Error Message
Mhrun.log
2018-03-07 05:19:46 [debug] 12346 execute: Running command: /opt/VRTSvcs/bin/hares -probe
2018-03-07 05:19:46 [debug] 12346 execute: execute: command failed. rc=256
2018-03-07 05:19:46 [debug] 12346 execute: err=VCS WARNING V-16-1-10268 Agent Mount for resource
Cause
Mount and IP agents got failed due to performance issues and High Availability daemon (HAD) restarted by hashadow.
-- AGENTS FAILED
-- Type System
J IP Node name
J Mount Node name
root 14269 1 0 Feb16 ? 00:22:59 /opt/VRTSvcs/bin/had -restart
2018/02/16 02:04:18 VCS ERROR V-16-2-13120 Thread(140665943332672) Error receiving from the engine. Agent(IP) is exiting.
2018/02/16 02:04:18 VCS ERROR V-16-2-13120 Thread(139750756890432) Error receiving from the engine. Agent(Mount) is exiting.
2018/02/16 02:16:48 VCS WARNING V-16-1-10023 Agent IP not sending alive messages since Fri Feb 16 02:14:32 2018
2018/02/16 02:27:48 VCS ERROR V-16-1-10008 Agent IP has faulted 6 times since Fri Feb 16 02:16:48 2018
2018/02/16 02:27:48 VCS ERROR V-16-1-10009 Agent IP has faulted 6 times in less than 950 seconds -- Will not attempt to restart. Correct the problem and use haagent -start to start the agent
2018/02/16 02:27:48 VCS ERROR V-16-1-10008 Agent Mount has faulted 6 times since Fri Feb 16 02:16:48 2018
2018/02/16 02:27:48 VCS ERROR V-16-1-10009 Agent Mount has faulted 6 times in less than 950 seconds -- Will not attempt to restart. Correct the problem and use haagent -start to start the agent
Resolution
Steps to fix the issue : -
1. Freeze running Service groups.
2. Kill these two PID's
root 14388 1 0 Feb16 ? 00:10:25 /opt/VRTSvcs/bin/Mount/MountAgent -type Mount
root 14394 1 0 Feb16 ? 00:10:29 /opt/VRTSvcs/bin/IP/IPAgent -type IP
3. Verify if above mentioned PID's got killed.
4. Start agent by using following command and verify hastatus -sum and make sure now none of the agents showing in failed state.
# haagent -start Mount -sys
# haagent -start IP -sys
5. Once everything fine on host side. Now initiate discovery from VOM management server and verify resources states.
Issue/Introduction
