AIX: Kernel pinned memory usage grows rapidly while a large backup is being performed.
Article ID: 100002407
Updated On:
Resolution
Symptoms:
It isobserved that pinned memory size, from the svmon -G output, keeps growingwhenever a backup performed. With default settings of VxFS tuneables it ispossible for VxFS to use nearly all of the pinned heap.
Cause:
The backupneeds to access the mtime(modification time) for each file to select files forthe backup operation. Since mtime is stored in the inode of each file, eachinode needs to be read into memory which would cause the inode cache, as definedby vxfs_ninode, to fill up.
Duringbackup, the geometry of the file on disk would also be required by NBU and,internally, VxFS would need to read the bitmap of files, which would increaseour buffer cache usage.
VxFS usespinned memory for inode cache, buffer cache and other kernel data structureslike locks etc.
The amountof maximum memory that could be used for inode cache and buffer cache is basedon the system configuration[size of physical memory and number ofCPUs].
Resolution:
By defaultAIX allows up to 80% of the physical memory to be pinned.
Aconfiguration that commonly uses a much smaller foot print of files will noticea significant increase in cache sizes during backup, especially if the backupincludes many 1000's of files that would not normally be accessed.
Our VxFSdefault inode and buffer caches max limits may be too high for thisconfiguration/workload and should be monitored and tuned downward to preventVxFS caches consuming larger amounts of pinned_heap unnecessarily during abackup.
Monitor theVxFS inode and the buffer cache during peak normal workload (not during backup)using vxfsstat, and then reduce both their sizes using the tunablesbelow.
Thevxfs_ninode and vx_bc_bufhwm can be tuned down to reduce the cache maximum sizesso that a backup will not use too large an amount of pinned heap.
Collect thebuffer cache and inode cache usage using vxfsstat and also vmstat at the sametime.
- monitorinode and buffer cache usage with vxfsstat -bi.
- captureoutput of vmstat -v.
Issue/Introduction
AIX: Kernel pinned memory usage grows rapidly while a large backup is being performed.
Was this article helpful?
Yes
No
Powered by
