Filesystems getting corrupted after dynamic LUN expansion and running vxdisk resize on Red Hat 9.

book

Article ID: 100070895

calendar_today

Updated On:

Description

Error Message

Symptoms:

  1. Do Dynamic Lun Expansion (DLE) of LUN.
  2. Checking in /var/log/messages, the following error is seen:
    Aug 21 11:00:08 server101 kernel: VxVM vxio V-5-3-2437 vol_dev_open: open failed for devno c900063 error: -6 bdev: fffffffffffffffa
  3. The filesystem gets corrupted on the 2nd attempt to do disk resize and the filesystem won't get mounted after mount:
    # mount -t vxfs /dev/vx/dsk/testdg1/vol3 /mnt/vol3
    UX:vxfs mount.vxfs: ERROR: V-3-20023: file system on /dev/vx/dsk/testdg1/vol3 has structural damage
  4. The corruption happens on InfoScale 8.0.x + RHEL 9 (not earlier operating systems)

 

Cause

While performing actions like Dynamic LUN Expansion (DLE), the OS removes partitions when it detects a disk change. The partition is not added back by the operating system because Dynamic Multipathing (DMP) doesn't revalidate the disk during the disk open process in RHEL9 and upstream. This loss of the partition causes device open failures, resulting in corruption. 

 

Resolution

a) If the issue hasn't occurred yet:

Please raise a case with Veritas Support and request a private hot-fix for the issue - Volume Manager 8.0.0.3100 or 8.0.2.1504.

Alternatively, avoid dynamic LUN expansion to grow disk (and volume/filesystem). Add new storage disk and add it to the diskgroup and utilize it to expand the volume/filesystem.

b) If issue has occurred: 

Contact Veritas Support.

 

Issue/Introduction

Affected environments: Infoscale 8.0/8.0.x on RHEL9

Additional Information

JIRA: STESC-9108