The default Hard Number of Open File Descriptor Limit (ulimit -n) can been modified manually as a workaround to make the ulimit settings to be soft limits instead of hard limits in the VCS environment configuration file: /opt/VRTSvcs/bin/vcsenv

The modified file should look like below in the file /opt/VRTSvcs/bin/vcsenv

# Setting ulimit.
# Common For Linux, HP-UX, SunOS & AIX
ulimit -t unlimited     # CPU Time
ulimit -c unlimited     # Core File Size
ulimit -d unlimited     # Data Seg Size
ulimit -f unlimited     # File Size
ulimit -S -n 2048          # File Descriptor  << Modified Soft Limits instead of default hard limit

 

Please note that VCS cluster services need to be restarted for the settings to take effect. A 'hastop' with the force option and then a 'hastart' will not resolve the issue for the processes that have already been started by VCS, and only new processes spawned by VCS after the force stop and start will have the new limits set. Hence it is strongly recommended to restart VCS cluster services completely, which will ensure that all the processes spawned by VCS are having the same ulimit settings.

Stop the cluster:

# hastop -all

Edit the VCS environment configuration file /opt/VRTSvcs/bin/vcsenv using any text editor like vi and modify the hard limits to soft limits as shown above. 

Start the cluster on each node:

# hastart

Veritas Cluster Server 5.1 SP1 introduces ulimit settings configured in the /opt/VRTSvcs/bin/vcsenv which are set up while starting up VCS. This new feature is introduced to prevent any low system settings from affecting the VCS HAD process. This means that every process spawned by the VCS will have the same ulimit settings defined in the configuration file. However, the ulimit settings by default are set to hard limits instead of soft limits, especially for the "ulimit -n" setting for the permissible number of open file descriptors. The default value of the open file descriptor is set to 2048, which could be lesser than the required open file descriptors for applications and/or database. # grep -i ulimit /opt/VRTSvcs/bin/vcsenv
# Setting ulimit.
ulimit -t unlimited     # CPU Time
ulimit -c unlimited     # Core File Size
ulimit -d unlimited     # Data Seg Size
ulimit -f unlimited     # File Size
ulimit -n 2048          # File Descriptor