Description

When investigating cases of suspected memory corruption on the Solaris platform, for example unexpectedly freed buffers, buffer over runs and so on, enabling of the kernel memory allocators debug features can be an extremely useful aid to subsequent crash dump analysis. These debug features are able to perform the following checks/logging on a running system:

Freed buffer checking - 0xdeadbeef is written into buffers when freed making it easy to check if a freed buffer is incorrectly being referenced

Buffer overrun checking - a redzone is appended to all allocated buffers when allocated. The redzone is then populated with 0xfeedface making it easy to see if user data is overrunning its allocated buffer space

Uninitialised data checking - 0xbbadcafe is written into all words of a buffer when that buffer is allocated. This makes is east to check if data within a buffer is incorrectly being referenced without first being initialised

Memory allocation logging - an area of debug data following the redzone is appended to all buffers when allocated. This debug area is known as the buftag area and contains information which the kernel memory allocator can use to determine if the user part of the buffer becomes corrupted, and also to provide a transaction trail of operations on the buffer (for example information on the last thread to perform an allocation and so on)

The debug features of the kernel memory allocator can be enabled at run time or at boot time via use of the kmem_flags variable in /etc/system.

Note that enabling debugging at run time is of extremely limited use as the debug features will only be applied to kernel memory caches created after debugging is enabled. As the vast majority of caches are created at boot time this means that the majority of caches will not have debugging applied to them. For this reason this option is not discussed further here.

Enabling the kernel memory allocators debug features at boot time is controlled via setting the kmem_flags variable in /etc/system to a specific value. For example:

# grep kmem_flags /etc/system
set kmem_flags=0xf

The value of kmem_flags determines which features are enabled according to the following table:

#define      KMF_AUDIT      0x00000001      /* transaction auditing */
#define      KMF_DEADBEEF      0x00000002      /* deadbeef checking */
#define      KMF_REDZONE      0x00000004      /* redzone checking */
#define      KMF_CONTENTS      0x00000008      /* freed-buffer content logging */
#define      KMF_LITE        0x00000100      /* lightweight debugging */

Note that setting kmem_flags=0 disables all kernel memory allocator debug features.

For example to enable KMF_CONTENTS and KMF_AUDIT, kmem_flags should be set to 0x9.

When kernel memory allocation debugging is required Veritas normally recommends setting kmem_flags to 0xf to enable KMF_AUDIT, KMF_DEADZONE, KMF_REDZONE, and KMF_CONTENTS.

Note that when kernel memory allocation debug is enabled there will be some degredation in performance of memory operations on the system. The amount of degredation depends on the options enabled. For this reason it is not recommended to enable kmem_flags by default on all systems. For more information contact the operating system vendor.