Error Message

 

In workloads that create files and write to them in a sequential pattern growing them to reasonably large files, VxFS tries to optimize the creation and allocation of extents to those files. The size of extents chosen for sequential extending writes can be tuned via the VxFS tunable max_seqio_extent_sz. This was set as default to 2K on releases prior to VxFS 7.2 and from VxFS 7.2 onwards the default value was increased to 32k.

 

The reason for the increased default value was to optimize extending writes and reduce fragmentation pressure seen at some customers. Creating larger extents at this time means that VxFS can pre-create the pages for these writes and a change in VxFS 7.2 to the method used for creating pages and the locking around this was also implemented. This was to move away from an interface being deprecated by Oracle in Solaris 11.3. This optimization significantly improves the speed of the page creation. This combined with the max_seqio_extent_sz  tunable change meant that VxFS 7.2 (and above) can consume pages quicker, but the flushing mechanism is starting to lag behind this creation.

 

VxFS is often tuned so that max_diskq is  10M and this has an effect that it allows writes to go faster which allows more write threads to consume the memory quicker. Eventually VxFS  will throttle the page flushing leading eventually to a system that becomes unresponsive while the operating system attempts to flush out the dirty pages.

In the event that this memory usage behaviour is observed on a system with a workload that sequentially creates large files (such as backups), Veritas strongly recommends that the max_seqio_extent_sz should be set to the pre-VxFS 7.2 value or tuned to a size that is manageable on VxFS 7.2 and above.

 

This can be set on the mounted filesystem by running:

# /opt/VRTS/bin/vxtunefs -o max_seqio_extent_size=2048

# /opt/VRTS/bin/vxtunefs -p | grep -i max_seqio   // to check the value

 

If the filesystem in question is a local filesystem (not CFS), then Veritas also strongly recommends that dalloc is disabled on that filesystem. Having dalloc enabled speeds up the allocation and allows larger extents again and would counteract the tuning of the max_seqio_extent_sz=2048. In the case of CFS filesystems,  having dalloc enabled will not have the same effect

 

# /opt/VRTS/bin/vxtunefs -p | grep -i dalloc_enable   // to check the value 0=disabled, 1=enabled

# /opt/VRTS/bin/vxtunefs -o dalloc_enable=0  // to disable it

 

 

If max_diskq is tuned to a value (such as 10M), then it may also be necessary to set this to a lower value.

 

# /opt/VRTS/bin/vxtunefs -p | grep -i max_diskq // to check its current value

 

However to start with, Veritas would recommend the following:

 

1. Setting max_seqio_extent_size to 2048

2. If the filesystem is local, then dalloc should be disabled

 

and then monitor the memory usage with these settings in place and if max_diskq is set to 10M and the memory usage is still high, set this tunable to a lower value.

 

During a recent investigation into increased memory usage of VxFS 7.3.1 on Solaris 11.3 when using a workload involving the creation of large files, it has been observed that the free memory drops quickly and can in certain circumstances lead to the system becoming unresponsive. Veritas has found this behaviour to be a result of a tunable change in the VxFS 7.2 release (and above) combined with a change to the page create method. This change to the page create method resulted in a performance improvement for write code paths that need to create pages. It is important to note that this memory usage is not observed all the time and/or on all systems running VxFS 7.2 or above on Solaris 11.3.

ETrack: 3940361