Mount of VxFS filesystem on AIX fails with out of memory error

book

Article ID: 100028824

calendar_today

Updated On:

Description

Error Message

Example mount failure:

#/opt/VRTS/bin/mount -V vxfs -o cluster /dev/vx/dsk/testdg/testvol1 /testvol1
UX:vxfs mount: ERROR: V-3-20: There is not enough memory available now.
UX:vxfs mount: ERROR: V-3-21256: cannot mount /dev/vx/dsk/testdg/testvol1

A truss of the failing mount command will show the vmount() system call failing with ENOMEM, for example:

56688880: 75629007: 0.2183:        vmount(0x300FB388, 0x000008B4) Err#12 ENOMEM







Cause

VxFS mount can fail with ENOMEM (out of memory) in the presence of other THRPGIO file systems such as Oracle's ACFS. 
Use kdb to find whether there are any THRPGIO file systems other than VxFS in use.

First, find all THRPGIO pdt's.

echo "pdt *" | kdb_64 /dev/pmem /unix | grep THRPGIO

Then for each of these, use the dev_t  address in conjunction with the "lke -s" command to see which module uses this pdt. 

if it shows /usr/lib/drivers/oracleacfs.ext module,  then this particular THRPGIO pdt belongs to oracle ACFS. 

 

For example, this particular pdt belongs to ACFS:

 

(0)> pdt * | grep THRPGIO                                                             
                                                        
vmmary_pdt+007C08 0093 FFFFFFFF F1000000C0906178 00000000 00000000      
00000000 local client(THRPGIO)   

(0)> pdt 93

PDT address F1000F0009507C08 entry 0093 of 1FFF, type: LOCAL CLIENT -- THRPGIO
eye catch             (_eyec)   : 706474564D4D0093
next pdt on i/o list  (nextio)  : FFFFFFFF
dev_t or strategy ptr (device)  : F1000000C0906178
last frame w/pend I/O (iotail)  : FFFFFFFFFFFFFFFF
free buf_struct list  (bufstr)  : F1000A060BB10000
total buf structs     (nbufs)   : 00C8
available (PAGING)    (avail)   : 0001
disk map srval        (dmsrval) : 00000000
i/o's not finished    (iocnt)   : 00000000
old i/o's unfinished  (oldio)   : 00000000
device wait list (devwait)      : 0000000000000000
buffer wait list (bufwait)      : 0000000000000000
logical volume lock (lock)      :@F1000F0009507C68 00000000
buffer list lock    (buf_lock)  :@F1000F0009507C70 00000000
flag bits            (devflags) : E0000000
nextfree             (nextfree) : 00000000
max phys Xlation ent (maxphys)  : 00000020
pageout down limit   (minpout)  : FFFFFFFF
pageout up limit     (maxpout)  : FFFFFFFF
base buf structs  (base_nbufs)  : 00C8
free buf structs  (free_nbufs)  : 00C8
refundable pile      (pile)     : 0000000000000000
last_io_done    (last_io_done)  : 0000000000000000
buf chain ptr           (bplist): 0000000000000000
next pdt on iolist (nextiostart): FFFFFFFF
thrpginfox ptr(thrpginfopx)     : F1000A0400581380
RAS block (pdt_rasb) : F1000A040058D300

 

 

(0)> lke -s F1000000C0906178                                            
    ADDRESS          FILE             FILESIZE FLAGS    MODULE NAME     
.                                                                       
15
  1 F1000000A05FA900 F1000000C0781000 0018A000 00090242                 
                                   /usr/lib/drivers/oracleacfs.ext      
  le_flags....... TEXT DATA DATAEXISTS NOTKEYSAFE 64                    

 

Resolution

 AIX APAR IV39497  has been created to rectifiy this issue. Please contact IBM support to obtain this fix.


Issue/Introduction

 On AIX a vxfs filesystem fails to mount with an out of memory error (ENOMEM) even though the system  has plenty of free memory.