Diskgroup "filesystem" information is missing after VCS stops and restarts in VOM (Operations Manager) 3.1

book

Article ID: 100025596

calendar_today

Updated On:

Cause

This behavior occurs under these conditions:

1. The filesystem belongs to a volume that is defined in a shared Cluster Diskgroup, so this diskgroup is shared across 2 hosts in a cluster.

2. When the cluster is started, the shared diskgroup is imported on both hosts and the volume/filesystem is mounted.

3. The VOM discovery process on both hosts in the cluster runs and recognises the changed volume and filesystem. It notifies VOM central database about them and adds them to /var/opt/VRTSsfmh/SF/._SF_VOLUME_r.dat and /var/opt/VRTSsfmh/SF/._SF_FILESYSTEM_r.dat on both hosts.

4. The VOM GUI shows the diskgroup, volume, and filesystem as belonging to both hosts.

5. If you umount the filesystem on one host in the cluster, the next VOM discovery does not see that the filesystem has been umounted. The /var/opt/VRTSsfmh/SF/._SF_FILESYSTEM_r.dat file still contains the unmounted filesystem - no change has been detected. The VOM GUI still indicates that the filesystem is mounted on this host.

6. If the VCS cluster on this same node is stopped, the next VOM discovery detects that the volume is gone and sends a request to the VOM database to delete the volume record in the volume table. A post trigger on this volume table in the VOM database then fires and deletes the corresponding filesystem record in the filesystem table.

7. At this stage, the VOM GUI knows nothing about the volume or the filesystem.

  • The host where the VCS cluster was stopped, knows nothing about the volume, or filesystem, but the files in /var/opt/VRTSsfmh/SF/ are not showing this correctly.
  • ._SF_VOLUME_r.dat has the volume removed.
  • But ._SF_FILESYSTEM has the filesystem belonging to this volume still in the file 
  • The other host, which still has the shared volume/filesystem mounted but this is no longer shown in the VOM GUI as having this Volume or Filesystem.

8. When the VCS cluster is restarted, the shared diskgroup, volume, and filesystem is mounted on the host. The next VOM discovery, recognises the volume change, but not the filesystem change as the ._SF_FILESYSTEM_r.dat file already has the filesystem included. This means only the volume change is reported to VOM database and shows up in the VOM GUI. The related filesystem record (including sizes) is not sent to the VOM Database, thus resulting in a missing filesystem records in the VOM Database. The other hosts VOM Discovery, though it can see the filesystem, never reports the filesystem to the VOM Database because it too has the filesystem record in its ._SF_FILESYSTEM_r.dat file already.

9. As a result, the VOM GUI shows the these conditions:

  • The "volume" display has no filesystem attributes for the selected volume and
  • The "filesystem" display has no filesystem record at all for either hosts, as there is no fileystem record in the table for this volume.

 

Resolution

A hotfix has been created to prevent this issue from occuring.

Download hotfix vom-HF030183000-09 from Veritas SORT:
https://sort.Veritas.com/patch/detail/5180/0/cGF0Y2gvc2VhcmNobWF0cml4LzcvMy8x


As a work-around, it is also possible to clear this condition by doing a manual VOM full rescan (/opt/VRTSsfmh/bin/mh_ctl.pl --rescan --ensure) on each of the affected hosts.


Issue/Introduction

This article discusses a known issue where the "filesystem" information is missing after VCS stops and restarts in VOM 3.1

Additional Information

ETrack: 2349799