dalloc (Delayed Allocation) can cause silent data corruption on VxFS 6.x, with HP-UX 11.31

book

Article ID: 100009851

calendar_today

Updated On:

Description

Error Message

There are no error messages reported.

 

Detailed explanation of the above workaround

  • For a mounted filesystem, the vxtunefs command can be used to set tuning parameters. However, values that are set using the vxtunefs command are not persistent across system reboots, or across file system remounts (umount and mount back).
  • To make the setting persistent, update the /etc/vx/tunefstab file. This file contains tuning parameters for Veritas File Systems which are set automatically during the file system mount.
    Each entry in the tunefstab file is a line of fields in one of the following formats:
    • block_device          tunefs options
    • system_default        tunefs options
  • "block_device" is the name of the device on which the file system is mounted. If there is more than one line that specifies the options for a device, each line is processed, and the options are set in order.
  • In place of "block_device," "system_default" specifies tunables for each device to process. If an entry for both a "block_device" and "system_default" exists, then the device value takes precedence.

 
 

Steps for making the VxFS File system tuning values persistent according to the file system


1. For the file system that is already mounted:

# vxtunefs -s -o dalloc_enable=0 /$MOUNT_POINT 

Example:

# vxtunefs -s -o dalloc_enable=0 /testmnt1 


2. Create the /etc/vx/tunefstab file, if it does not exist, and add the entry:

/dev/vx/dsk/$DISKGROUP/$VOLUME dalloc_enable=0 

Example:

/dev/vx/dsk/testdg/testvol1 dalloc_enable=0 

          
3. To confirm the value after the mount:
 

# vxtunefs /testmnt1 | grep "dalloc_enable"dalloc_enable = 0 

Example, using a single file system:

For mounted filesystems:

# vxtunefs /testmnt1 |grep "dalloc_enable"dalloc_enable = 1# vxtunefs -s -o dalloc_enable=0 /testmnt1UX:vxfs vxtunefs: INFO: V-3-22525: Parameters successfully set for /testmnt1# vxtunefs /testmnt1 |grep "dalloc_enable"dalloc_enable = 0 


To make it persistent, add an entry to the /etc/vx/tunefstab file.

# cat /etc/vx/tunefstab/dev/vx/dsk/testdg/testvol1 dalloc_enable=0# mount -F vxfs /dev/vx/dsk/testdg/testvol1 /testmnt1/# vxtunefs /testmnt1/ | grep "dalloc_enable"dalloc_enable = 0 

  
Example, using multiple file systems:

# mount -F vxfs /dev/vx/dsk/testdg/testvol1 /testmnt1/# vxtunefs /testmnt1/ | grep "dalloc_enable"dalloc_enable = 1

# mount -F vxfs /dev/vx/dsk/testdg/testvol2 /testmnt2# vxtunefs /testmnt2/ | grep "dalloc_enable"dalloc_enable = 1

# mount -F vxfs /dev/vx/dsk/testdg/testvol3 /testmnt3# vxtunefs /testmnt3/ | grep "dalloc_enable"dalloc_enable = 1

# cat /etc/vx/tunefstab/dev/vx/dsk/testdg/testvol1 dalloc_enable=0/dev/vx/dsk/testdg/testvol2 dalloc_enable=0/dev/vx/dsk/testdg/testvol3 dalloc_enable=0

# umount /testmnt1/# umount /testmnt2/# umount /testmnt3/# mount -F vxfs /dev/vx/dsk/testdg/testvol1 /testmnt1/

# mount -F vxfs /dev/vx/dsk/testdg/testvol2 /testmnt2/

# mount -F vxfs /dev/vx/dsk/testdg/testvol3 /testmnt3/

# vxtunefs /testmnt1/ | grep "dalloc_enable"dalloc_enable = 0

# vxtunefs /testmnt2/ | grep "dalloc_enable"dalloc_enable = 0

# vxtunefs /testmnt3/ | grep "dalloc_enable"dalloc_enable = 0# 

 

Steps for making the VxFS file system tuning values persistent for all file system, the system-wide default setting

This is helpful if you need to apply the tuning system-wide. If the "system_default" is specified instead of "block_device," the tunable setting will be applied to all of the block devices when mounting the file systems.
 
1. Create the /etc/vx/tunefstab file, if it is not there already.

# touch /etc/vx/tunefstab# ls -l /etc/vx/tunefstab-rw-r--r-- 1 root system 75 Aug 21 17:11 /etc/vx/tunefstab 

2. Add the entry to the /etc/vx/tunefstab file.

system_default dalloc_enable=0 

3. This can be confirmed after mounting the file system on the device, using the vxtunefs –p command.

# vxtunefs /testmnt1/ | grep "dalloc_enable"dalloc_enable = 0

 
Example:

# cat /etc/vx/tunefstabsystem_default dalloc_enable=0

# mount -F vxfs /dev/vx/dsk/testdg/testvol1 /testmnt1/

# mount -F vxfs /dev/vx/dsk/testdg/testvol2 /testmnt2/

# mount -F vxfs /dev/vx/dsk/testdg/testvol3 /testmnt3/

# vxtunefs /testmnt1/ | grep "dalloc_enable"dalloc_enable = 0

# vxtunefs /testmnt2/ | grep "dalloc_enable"dalloc_enable = 0

# vxtunefs /testmnt3/ | grep "dalloc_enable"dalloc_enable = 0 


Note: The system_default setting is overridden if a block_device is also explicitly specified.
 
Example:
if we want to change the system-wide default for "dalloc_enable" to 0, but do not want to set it for /dev/vx/dsk/testdg/testvol3, then we can do the following:

# cat /etc/vx/tunefstabsystem_default dalloc_enable=0/dev/vx/dsk/testdg/testvol3 dalloc_enable=1

# mount -F vxfs /dev/vx/dsk/testdg/testvol1 /testmnt1/

# mount -F vxfs /dev/vx/dsk/testdg/testvol2 /testmnt2/

# mount -F vxfs /dev/vx/dsk/testdg/testvol3 /testmnt3/

# vxtunefs /testmnt1/ | grep "dalloc_enable"dalloc_enable = 0# vxtunefs /testmnt2/ | grep "dalloc_enable"dalloc_enable = 0

# vxtunefs /testmnt3/ | grep "dalloc_enable"dalloc_enable = 1 

 

Applies to:

  • VxFS 6.0, 6.0RP1, 6.0.1 & 6.0.3 ,on HP-UX 11.31 only
  • Locally mounted file systems. CFS (Cluster File System) is not at risk.

Cause

VxFS (Veritas File System) 6.x includes a new feature called delayed allocation also known as dalloc. This feature is enabled by default for local, mounted file systems. It is not supported for cluster mounted file systems.

Asynchronous writes, that extend the size of a file, will create memory pages, and make them "dirty." New extents can therefore be allocated when the dirty pages are flushed to disk (via background processing) rather than allocating the extents in the same context as the write I/O. The delayed allocation feature therefore delays the extent allocation until the dirty pages are flushed to disk.

When the dalloc feature is enabled on VxFS, it is possible for sequential, allocating writes to allow stale data into the contents of a file. The dalloc feature allows VxFS to allocate extents to the file after the write has been completed. The stale data can occur as the OS tries to flush the dirty pages to disk, but does not yet have extents in the allocated area. When VxFS receives the pageout request from the OS, it erroneously sets a flag to free the pages for the area that is covered by the delayed allocation. The newly-written data is not flushed to the disk. This results in stale data in the file.

This only applies to Storage Foundation 6.x.x, with HP-UX 11.31, where dalloc is enabled. Cluster File Systems (CFS) are not at risk.

The stale data can occur as the OS tries to flush the dirty pages to disk, but does not yet have extents in the allocated area. When VxFS receives the pageout  request from the OS, it erroneously sets a flag to free the pages for the area that is covered by the delayed allocation. The newly-written data is not flushed to the disk. This results in stale data in the file.

Resolution

A supported hotfix has been made available for this issue for VxFS 6.0.3. Please contact Veritas Technical Support to obtain this fix.

OR

Upgrade to Storage Foundation 6.0.5 that has the fix included in it.

 

A workaround is also available for environments where the Hotfix is not installed which involves disabling the delayed allocation (dalloc) feature on the VxFS filesystems.

The details are below:

Notes:

  • If a VxFS file system is resized or re-mounted (mount -o remount) after dalloc is disabled, the resize/remount operation may re-enable dalloc. The dalloc feature will have to be manually disabled again with vxtunefs.
  • For the O/S file systems (/, /usr, /stand, /opt & /var), 'dalloc' cannot be enabled or disabled persistently using the tunefstab file below. Therefore 'dalloc' can only be enabled/disabled on these file systems with the vxtunefs command and will need to be manually reset after each reboot.
  • Care must be taken in a Oracle Cluster Ready Services (CRS) environment.  When using CRS, as a change using vxtunefs to the CRS volume could cause a delay in heartbeat,  please first disable CRS monitoring while make a vxtunefs change to /crs. 


1. For a filesystem that is already mounted:

# vxtunefs -s -o dalloc_enable=0 $MOUNT_POINT
Example:# vxtunefs -s -o dalloc_enable=0 /testmnt1 


2. To make the value persistent after a reboot, add an entry to /etc/vx/tunefstab. This file contains tuning parameters for Veritas File Systems that are set automatically during the file system mount.

The entry in the tunefstab file can set according to the file system or to the system default.


To set according to the file system:

/dev/vx/dsk/$DISKGROUP/$VOLUME dalloc_enable=0
Example:# cat /etc/vx/tunefstab/dev/vx/dsk/testdg/testvol1 dalloc_enable=0
To set according to the system default:# cat /etc/vx/tunefstabsystem_default dalloc_enable=0 

 

Additional Information

ETrack: 3244048 ETrack: 3264506