System panic in vx_softcnt_flush() in vxfs module after an unsuccessful attempt to unmount the file system
Article ID: 100006232
Updated On:
Description
Error Message
*** Panic stack from crash dump would show:
crash> bt
PID: 3974 TASK: ffff8104a4613820 CPU: 3 COMMAND: "vx_worklist_thr"
#0 [ffff8104981a7a60] crash_kexec at ffffffff800ad9ce
#1 [ffff8104981a7b20] __die at ffffffff80065157
#2 [ffff8104981a7b60] do_page_fault at ffffffff80066dd7
[...]
#4 [ffff8104981a7d08] generic_drop_inode at ffffffff80039ca6
#5 [ffff8104981a7d28] vx_softcnt_flush at ffffffff885a7cc4
#6 [ffff8104981a7d58] vx_ireuse_clean at ffffffff8856fb70
#7 [ffff8104981a7d88] vx_ilist_chunkclean at ffffffff88570660
#8 [ffff8104981a7df8] vx_inode_free_list at ffffffff88571edf
#9 [ffff8104981a7e38] vx_ifree_scan_list at ffffffff8856123e
#10 [ffff8104981a7e48] vx_workitem_process at ffffffff88560461
#11 [ffff8104981a7e58] vx_worklist_process at ffffffff88560656
#12 [ffff8104981a7ed8] vx_worklist_thread at ffffffff885634bd
#13 [ffff8104981a7ee8] vx_kthread_init at ffffffff885a5d19
#14 [ffff8104981a7f48] kernel_thread at ffffffff8005dfb1
*** Syslog messages prior to system panic:
Jul 10 10:06:40 sys31 kernel: vxfs: msgcnt 11 vx_detach_fset error 16
Jul 10 10:06:40 sys31 kernel:
Jul 10 10:06:40 sys31 kernel: vxfs: msgcnt 12 mesg 096: V-2-96: vx_setfsflags - /dev/vx/dsk/testdg/vol01 file system fullfsck flag set - vx_unmount
Jul 10 10:06:40 sys31 kernel: vxfs: msgcnt 13 mesg 023: V-2-23: vx_unmountroot - /dev/vx/dsk/testdg/vol01 file system is busy and can't be unmounted cleanly
Jul 10 10:06:40 sys31 kernel: vxfs: msgcnt 14 mesg 031: V-2-31: vx_disable - /dev/vx/dsk/testdg/vol01 file system disabled
Cause
This issue is tracked via the etrack incidents mentioned in the Supplemental Materials section.
Panic occured in iput() (vx_softcnt_flush()) which was called to drop the softcount hold (i_count) held by VxFS. Panic thread is cleaning an inode on freelist. Inode being cleaned belongs to the file system (FS) which is already unmounted. FS superblock structure is freed after vxfs unmount returns irrespective of whether unmount was successful or failed as linux always expects umount to succeed.
In this case, unmount of this FS was not clean i.e. detach of this FS failed with EBUSY error. Detach of FS can fail with EBUSY error if there are any busy inodes i.e. having pending operations. During an unmount operation, all the inodes belonging to that file system are cleaned. But as unmount of FS was not successful, inode processing of FS didn't complete. When background thread picks the inodes of such unmounted FS, it can result in panic when accessing superblock structure which was already freed.
Resolution
Code changes were done to prevent such panic. The fix is available in the following patch releases.
- VxFS 5.0MP3RP2HF11
- VxFS 5.1SP1RP2
Please visit SORT website https://sort.Veritas.com/patch/matrix for latest available patches.
Applies To
This issue is applicable to Linux platform only.
Issue/Introduction
Additional Information
