PANIC: "Kernel panic - not syncing: VxVM vxio V-5-3-2090 bulk_cleanup_verification

book

Article ID: 100054094

calendar_today

Updated On:

Description

Error Message

The following is reported in the vmcore file:

      KERNEL: vmlinux.3.10.0-1160.59.1.el7.x86_64
    DUMPFILE: vmcore-xxxxxx.  [PARTIAL DUMP]
        CPUS: 32
        DATE: Tue May 24 13:45:49 2022
      UPTIME: 3 days, 21:14:57
LOAD AVERAGE: 0.08, 0.31, 0.44
       TASKS: 1106
    NODENAME: xxxxxxx.xxxxx.com
     RELEASE: 3.10.0-1160.59.1.el7.x86_64
     VERSION: #1 SMP Wed Feb 16 12:17:35 UTC 2022
     MACHINE: x86_64  (2593 Mhz)
      MEMORY: 144 GB
       PANIC: "Kernel panic - not syncing: VxVM vxio V-5-3-2090 bulk_cleanup_verification: bad upd"
         PID: 19769
     COMMAND: "vxiod"
        TASK: ffff8fd2f619b180  [THREAD_INFO: ffff8fcef1968000]
         CPU: 31
       STATE: TASK_RUNNING (PANIC)

crash> bt
PID: 19769  TASK: ffff8fd2f619b180  CPU: 31  COMMAND: "vxiod"
 #0 [ffff8fcef196bbf0] machine_kexec at ffffffffbb2662f4
 #1 [ffff8fcef196bc50] __crash_kexec at ffffffffbb322a32
 #2 [ffff8fcef196bd20] panic at ffffffffbb9802cc
 #3 [ffff8fcef196bda0] volrv_seclog_bulk_cleanup_verification at ffffffffc09f099a [vxio]
 #4 [ffff8fcef196be18] volrv_seclog_write1_done at ffffffffc09f0a41 [vxio]
 #5 [ffff8fcef196be48] voliod_iohandle at ffffffffc0827688 [vxio]
 #6 [ffff8fcef196be88] voliod_loop at ffffffffc082787c [vxio]
 #7 [ffff8fcef196bec8] kthread at ffffffffbb2c5e61

Cause

Network failures cause packets to arrive at the secondary in the incorrect order.

Resolution

Enable VVR nmcom checksum. Up to and including Infoscale 8.x this is disabled by default, but will be enabled in future releases;

Confirm current setting:

# vxtune vol_nmcom_disable_checksum
Tunable                                        Current Value                Default Value Reboot Clusterwide
-------------------------------                -------------                ------------- ------ -----------
vol_nmcom_disable_checksum                                 1                            1    N        N

To Enable:

# vxtune vol_nmcom_disable_checksum 0

 

Please note that this change is dynamic and persistent - it will take effect immediately and retain the new setting after reboots.

 

 

Issue/Introduction

In an environment with intermittent Networking failures the Secondary (DR) node in a VVR configuration can panic and reboot.