Server panics when a NULL pointer is dereferenced during a lazy unmount

book

Article ID: 100031528

calendar_today

Updated On:

Description

Error Message

crash> bt PID: 14199

TASK: ffff880650d4eaa0 CPU: 0 COMMAND: "bash" #0 [ffff880febb13340] machine_kexec at ffffffff81035b7b #1 [ffff880febb133a0] crash_kexec at ffffffff810c0db2 #2 [ffff880febb13470] oops_end
    at ffffffff815111d0 #3 [ffff880febb134a0] no_context at ffffffff81046bfb #4 [ffff880febb134f0] __bad_area_nosemaphore
    at ffffffff81046e85 #5 [ffff880febb13540] bad_area_nosemaphore
    at ffffffff81046f53 #6 [ffff880febb13550] __do_page_fault at ffffffff810476b1 #7 [ffff880febb13670] do_page_fault
    at ffffffff8151311e #8 [ffff880febb136a0] page_fault
    at ffffffff815104d5 [exception RIP: _read_lock+9] RIP: ffffffff8150fe79 RSP: ffff880febb13758 RFLAGS: 00010246 RAX: ffff880650d4eaa0 RBX: ffff880febb13828 RCX: ffff880febb13828 RDX: 0000000000000101 RSI: ffff880febb13948 RDI: 0000000000000004 RBP: ffff880febb13758 R8: 0000000000000067 R9: 0000000000000000 R10: 0000000000000055 R11: 0000000000000000 R12: 0000000000000000 R13: 0000000000000024 R14: ffff880febb13828 R15: ffffffffffffffe9 ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018 #9 [ffff880febb13760] path_init
    at ffffffff8118ee1a #10 [ffff880febb137a0] do_path_lookup at ffffffff811918db #11 [ffff880febb137d0] do_filp_open at ffffffff8119285b #12 [ffff880febb13920] filp_open
    at ffffffff8119354d #13 [ffff880febb13930] vx_imc_init_module at ffffffffa0b151a1 [vxfs] #14 [ffff880febb139d0] vx_imc_import
    at ffffffffa0b15701 [vxfs] #15 [ffff880febb13a20] vx_detach_fset at ffffffffa0cb990b [vxfs] #16 [ffff880febb13b40] vx_unmount
    at ffffffffa0d4276d [vxfs] #17 [ffff880febb13bc0] generic_shutdown_super
    at ffffffff8118326b #18 [ffff880febb13be0] kill_block_super
    at ffffffff81183321 #19 [ffff880febb13c00] vx_kill_sb
    at ffffffffa0d40918 [vxfs] #20 [ffff880febb13c20] deactivate_super
    at ffffffff81183af7 #21 [ffff880febb13c40] mntput_no_expire at ffffffff811a1b6f #22 [ffff880febb13c70] path_put
    at ffffffff8118f141 #23 [ffff880febb13c90] free_fs_struct at ffffffff811b2fa3 #24 [ffff880febb13cb0] exit_fs
    at ffffffff811b31b8 #25 [ffff880febb13ce0] do_exit
    at ffffffff81073415 #26 [ffff880febb13d60] do_group_exit
    at ffffffff81073b48 #27 [ffff880febb13d90] get_signal_to_deliver
    at ffffffff81088e16 #28 [ffff880febb13e30] do_signal at ffffffff8100a265 #29 [ffff880febb13f30] do_notify_resume
    at ffffffff8100aa80 #30 [ffff880febb13f50] int_signal
    at ffffffff8100b341

 

Cause

The panic is caused when an existing thread drops the last reference to a "lazy unmounted" Veritas File System (VxFS) if it was the last VxFS mount. The exiting thread performs an unmount. It then calls into Volume Manager (VxVM) to "de-initialize" the private FS-VM API, because it is the last VxFS mounted file system.

The function to be called in VxVM is "looked-up" via the files under /proc. This requires an opening of a file, but the exit processing has removed the structs that are needed by the thread to open a file.
 
Inspection of the failure indicates that the module is attempting to open a file pointer when the "fs" member of the task_struct is NULL. This results in the read_lock() function call encountering a NULL pointer dereference, due to the spinlock_t member of the fs_struct being uninitialized.

More information is available from the Redhat website.
https://access.redhat.com/solutions/1386703

Resolution

Cache the "de-init" function when the VxFS/VxVM API is initialized. This means that no function look-up is needed during an unmount operation. The cached function pointer can then be called during the last unmount. This bypasses the need to open the file by the exiting thread.

A fix for this issue is included in the following packages. Patches and hot fixes are normally available from SORT: https://sort.veritas.com/patch/finder
 
Product Version Platform Type Package
Veritas Storage Foundation HA 6.2 Linux PA sfha-sles11sp4_x86_64-Patch-6.2.1.100
InfoScale 7 Linux PA infoscale-rhel6.7_x86_64-Patch-7.0.0.100  
Veritas File System 7 Linux PA fs-sles12_x86_64-Patch-7.0.0.100
Veritas File System 7 Linux PA fs-sles11_x86_64-Patch-7.0.0.100
Veritas File System 6.2 Linux PA fs-sles11_x86_64-Patch-6.2.1.100
Veritas File System 7 Linux PA fs-rhel7_x86_64-Patch-7.0.0.100
Veritas File System 6.2 Linux PA fs-rhel7_x86_64-Patch-6.2.1.100
Veritas File System 7 Linux PA fs-rhel6_x86_64-Patch-7.0.0.100
Veritas File System 6.2 Linux PA fs-rhel6_x86_64-Patch-6.2.1.100
Veritas File System 6.2.1 Linux PA fs-sles12_x86_64-Patch-6.2.1.100
Veritas File System 6.0.1 Linux PHF fs-sles11_x86_64-HotFix-6.0.5.110
Veritas File System 6.0.1 Linux PHF fs-sles10_x86_64-HotFix-6.0.5.110
Veritas File System 6.0.1 Linux PHF fs-rhel6_x86_64-HotFix-6.0.5.205
Veritas File System 6.0.1 Linux PHF fs-rhel6_x86_64-HotFix-6.0.5.204
Veritas File System 6.0.1 Linux PHF fs-rhel6_x86_64-HotFix-6.0.5.110
Veritas File System 6.0.1 Linux PHF fs-rhel6_x86_64-HotFix-6.0.3.305
Veritas File System 6.0.1 Linux PHF fs-rhel5_x86_64-HotFix-6.0.5.110
Veritas File System 6.0.1 Linux PA fs-rhel5_x86_64-Patch-6.0.5.400
Veritas File System 6.0.1 Linux PA fs-rhel6_x86_64-Patch-6.0.5.400   
Veritas File System 6.0.1 Linux PA fs-sles10_x86_64-Patch-6.0.5.400 
Veritas File System 6.0.1 Linux PA fs-sles11_x86_64-Patch-6.0.5.400 
 

Issue/Introduction

The server panics when a NULL pointer is dereferenced during a lazy unmount.

Additional Information

ETrack: 3736398
Powered by Wolken Software