Veritas Technical Support has found that certain customers face problems with probing and/or Online of existing resources after successful VCS upgrades.

It has been identified that if all the resources of a particular Type fail to probe or online, then it could more likely be an issue with the /etc/VRTSvcs/conf/config/types.cf not updated correctly after the upgrade.
The CPI Installer will automatically update the types.cf as required correctly as long as the correct procedures have been followed from the Installation Guides.

Hence if customer has performed a successful upgrade of VCS and face the problem of resources unable to probe or online then we need to first verify some points regarding the upgrade:

1) VCS should be running well under the older version before upgrade

2) The users are using Cross Platform Installer (CPI) to upgrade as per instructions in the Installation Guide Document

3) The users did not use any other manual steps to modify/update *.cf files under /etc/VRTSvcs/conf/config/

4) The users "should NOT" open a Java Graphical User Interface (GUI) to forcibly change resources/types during the upgrade

5) The CPI upgrade flow did not fail or print any warning/error information from the screen output

If we identify that the types.cf files are not updated correctly after the successful upgrade, then kindly follow the procedure below to update the correct *.cf files:

1.) Ensure if the cluster is running and forms a part of the same cluster membership as the other nodes
# gabconfig -a
# hastatus -sum

2.) Freeze all the Service Groups persistently
# hagrp -freeze -persistent

3.) Dump and save the existing configuration
# haconf -dump -makero

4.) Stop the Cluster services with the "-force" option to ensure that the production is not affected
# hastop -all -force

5.) Confirm that the VCS services have stopped
# hastatus -sum
# ps -elf | grep -i had

6.) Compare the updated default types.cf file with the existing /etc/VRTSvcs/conf/config/types.cf, check the timestamps and cksum:
# diff /etc/VRTSvcs/conf/types.cf /etc/VRTSvcs/conf/config/types.cf
# ls -l /etc/VRTSvcs/conf/types.cf /etc/VRTSvcs/conf/config/types.cf
# cksum /etc/VRTSvcs/conf/types.cf
# cksum /etc/VRTSvcs/conf/config/types.cf

The timestamp on the /etc/VRTSvcs/conf/types.cf file should be of the correct version of the upgraded version. This can be confirmed also by quickly looking up and referencing with the Agent definitions in the Bundled Agent Reference Guide for the version of the product that we have upgraded to.

7.) Once verified that the correct file has not been updated to /etc/VRTSvcs/conf/config/types.cf, manually backup the original file and copy the correct file of the version to the config directory
# cd /etc/VRTSvcs/conf/config
# cp types.cf types.cf.orig
# cp /etc/VRTSvcs/conf/types.cf /etc/VRTSvcs/conf/config/types.cf

8.) Check for syntax errors in the VCS configuration. Only when this step has succeeded without any output error/warning should we proceed to start VCS successfully, or we have to make more changes into /etc/VRTSvcs/conf/config/main.cf or /etc/VRTSvcs/conf/config/types.cf or any other specific /etc/VRTSvcs/conf/config/*.cf for specific agents having a separate *.cf file.

# hacf -verify /etc/VRTSvcs/conf/config

If there is any failing messages from "hacf -verify" command above, then we need to check in detail manually:

a) For each resource defined in main.cf, make sure there is no missing correlated type in types.cf or in any other *.cf files that have been included into main.cf or types.cf;

b) For each Attribute relating to a Resource configured in the main.cf, make sure there is a corresponding Attribute defined in its Type definition in the *types.cf . This can be verified from the Bundled Agent's Guide or the relevant Agent's Installation/Configuration guide for the present version. The Release Notes document of the current version will also mention if there are any new attributes added to a Resource type.

Proceed to step 9 only if the step 8 completes successfully.

9.) Restart the cluster services on each node one at a time
# hastart

10.) Verify that the resources which were having problems probing have now been probed successfully on the affected systems:
# hastatus -sum
# hares -probe -sys

We can verify from the /opt/VRTSvcs/log/engine_A.log to check if the resource probe has been successful or if it still had problems. The "hastatus -sum" output should not show any resources as "RESOURCES NOT PROBED"
 

If customers encounter the similar issue, we request customers to open a Case with Veritas Technical Support to assist us in identifying the root cause of this issue. Kindly provide us with the below files for analysis. Please also inform the Support Engineer about reference to this article: 100020348

a) # tar -cvf conf.tar /etc/VRTSvcs/conf/                              

b) # tar -cvf CPIlog.tar /opt/VRTS/install/logs/                      

The above two directories are extremely important for us to address the root cause of this issue.

If the above procedure does not resolve the probe of the resources, then we need to look into each resource configuration and identify if there are missing Attributes for that Resource type.

In newer releases, some resources types may add new Attributes which may be required for the configuration for those resource Types in the newer versions and hence we may need to modify the respective resources in the VCS configuration in the main.cf or by using the "hares -modify" commands. These details would be documented in the Release Notes documents for that particular version.

If the upgrade has been performed from a much older release, then it is likely that some resource attributes have been added/deleted from previous versions and hence our configuration in VCS may require changes according to the requirements. Hence it would be helpful to reference the Release Notes document and the VCS Bundled Agent's Guide for the upgraded version of the product.