VCS unable to online WebSphere resources

book

Article ID: 100015908

calendar_today

Updated On:

Description

Error Message

Example of VCS first probe errors found in VCS engine_A log:

2015/05/24 16:35:47 VCS NOTICE V-16-1-10619 'HAD' starting on: SERVER2

2015/05/24 16:35:59 VCS INFO V-16-1-10304 Resource WAS_ApplicationServer3 (Owner: Unspecified, Group: WAS) is offline on SERVER2 (First probe)
2015/05/24 16:35:59 VCS INFO V-16-1-10304 Resource WAS_ApplicationServer2 (Owner: Unspecified, Group: WAS) is offline on SERVER2 (First probe)
2015/05/24 16:36:00 VCS INFO V-16-1-10304 Resource WAS_ApplicationServer1 (Owner: Unspecified, Group: WAS) is offline on SERVER2 (First probe)​
2015/05/24 16:36:00 VCS INFO V-16-1-10297 Resource WAS_NodeAgent1 (Owner: Unspecified, Group: WAS) is online on SERVER2 (First probe)
2015/05/24 16:36:00 VCS NOTICE V-16-1-10442 Initiating auto-start online of group WAS on system SERVER2
2015/05/24 16:36:00 VCS NOTICE V-16-1-10301 Initiating Online of Resource WAS_ApplicationServer1 (Owner: Unspecified, Group: WAS) on System SERVER2
2015/05/24 16:36:00 VCS NOTICE V-16-1-10301 Initiating Online of Resource WAS_ApplicationServer2 (Owner: Unspecified, Group: WAS) on System SERVER2
2015/05/24 16:36:00 VCS NOTICE V-16-1-10301 Initiating Online of Resource WAS_ApplicationServer3 (Owner: Unspecified, Group: WAS) on System SERVER2

2015/05/24 16:42:50 VCS ERROR V-16-2-13066 (SERVER2) Agent is calling clean for resource(WAS_ApplicationServer3) because the resource is not up even after online completed.
2015/05/24 16:42:50 VCS ERROR V-16-2-13066 (SERVER2) Agent is calling clean for resource(WAS_ApplicationServer1) because the resource is not up even after online completed.
2015/05/24 16:42:50 VCS ERROR V-16-2-13066 (SERVER2) Agent is calling clean for resource(WAS_ApplicationServer2) because the resource is not up even after online completed.

Cause

There was a stale NodeAgent PID (process ID) file located WAS_HOME folder and when server was rebooted and if the process ID was running VCS is reporting NodeAgent resource as online (on first probe).

WebSphere JVM/ApplicationServer resources were faulting because they depend on WebSphere NodeAgent

Resolution

Follow the below procedure to rename the stale PID file:
  1. Go to folder WAS_HOME\profiles\$ServerProfile\logs\$ServerType. (eg. D:\WebSphere\AppServer\profiles\node01\logs\nodeagent)
  2. Locate the stale PID file (eg. nodeagent.pid)
  3. Rename the file and reboot the server.
Note:
WAS_HOME = D:\WebSphere\AppServer
ServerProfile = node01
ServerType = nodeagent


​Veritas has identified the WebSphere agent resource monitoring issue with the current agent code where the agent does not check the process of the stale PID file with the required WAS pattern. This issue is addressed in the next release of WebSphere Agent (Agentpack 3Q2015). Please contact Veritas Technical Support for additional questions or concerns regarding this technical article.

Issue/Introduction

Post reboot, Veritas Cluster Server (VCS) is unable to online WebSphere resources.

Additional Information

ETrack: 3815714