Analysis of resulting crash dump shows the stack, like below:
crash> bt
PID: 6542 TASK: ffff91c2440d8000 CPU: 3 COMMAND: "llt_ioship_recv"
#0 [ffffafb5096a3808] machine_kexec at ffffffff8166564e
#1 [ffffafb5096a3860] __crash_kexec at ffffffff817a576d
#2 [ffffafb5096a3928] crash_kexec at ffffffff817a665d
#3 [ffffafb5096a3940] oops_end at ffffffff816264cd
#4 [ffffafb5096a3960] no_context at ffffffff816763bf
#5 [ffffafb5096a39b8] __bad_area_nosemaphore at ffffffff8167671c
#6 [ffffafb5096a3a00] do_page_fault at ffffffff81676fb7
#7 [ffffafb5096a3a30] page_fault at ffffffff8200111e
[exception RIP: memcpy_erms+6]
RIP: ffffffff81f93a16 RSP: ffffafb5096a3ae0 RFLAGS: 00010202
RAX: ffff91bf83791000 RBX: 0000000000000200 RCX: 0000000000000030
RDX: 0000000000000030 RSI: 00000000000034c0 RDI: ffff91bf83791000
RBP: 00000000000034c0 R8: ffffafb5096a3a78 R9: ffff91bec13bfa40
R10: ffff91bf83791000 R11: 0000000000000246 R12: ffff91c244034220
R13: 0000000000000030 R14: 0000000000000000 R15: 00000000000001d0
ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
#8 [ffffafb5096a3ae0] vol_ioship_copy_frags_to_ioship_buf at ffffffffc2cdda6b [vxio]
#9 [ffffafb5096a3b38] vol_disk_rhandle_read_resp at ffffffffc2ce7d75 [vxio]
#10 [ffffafb5096a3c68] vol_ioship_rrecv at ffffffffc2ce9fdc [vxio]
#11 [ffffafb5096a3e38] gab_lrrecv at ffffffffc32b8f52 [gab]
#12 [ffffafb5096a3e78] vx_ioship_process_data at ffffffffc0c7ab69 [llt]
#13 [ffffafb5096a3ed0] vx_ioship_recv_data at ffffffffc0c7b083 [llt]
#14 [ffffafb5096a3f10] kthread at ffffffff81712b3a
#15 [ffffafb5096a3f50] ret_from_fork at ffffffff82000255
crash>
i.e. a panic in IO Shipping related code
After consulting output from ifconfig -a, the network's Maximum Transmission Unit (MTU) was set at a mixture of 8000 and 9000 MTU values across the various nodes' networks. This is against recommendations for network links used for IO Shipping.
The recommendation is to set the networks' MTU setting to 9000 for those configured with IO Shipping.
There are no plans to address this issue by way of a patch or hotfix in earlier versions of the software at the present time. However, the issue has been addressed in the revision of the product specified at the end of this article.
Please contact your Veritas Sales representative or the Veritas Sales group for upgrade information including upgrade eligibility to the release containing the resolution for this issue.
crash> bt
PID: 6542 TASK: ffff91c2440d8000 CPU: 3 COMMAND: "llt_ioship_recv"
#0 [ffffafb5096a3808] machine_kexec at ffffffff8166564e
#1 [ffffafb5096a3860] __crash_kexec at ffffffff817a576d
#2 [ffffafb5096a3928] crash_kexec at ffffffff817a665d
#3 [ffffafb5096a3940] oops_end at ffffffff816264cd
#4 [ffffafb5096a3960] no_context at ffffffff816763bf
#5 [ffffafb5096a39b8] __bad_area_nosemaphore at ffffffff8167671c
#6 [ffffafb5096a3a00] do_page_fault at ffffffff81676fb7
#7 [ffffafb5096a3a30] page_fault at ffffffff8200111e
[exception RIP: memcpy_erms+6]
RIP: ffffffff81f93a16 RSP: ffffafb5096a3ae0 RFLAGS: 00010202
RAX: ffff91bf83791000 RBX: 0000000000000200 RCX: 0000000000000030
RDX: 0000000000000030 RSI: 00000000000034c0 RDI: ffff91bf83791000
RBP: 00000000000034c0 R8: ffffafb5096a3a78 R9: ffff91bec13bfa40
R10: ffff91bf83791000 R11: 0000000000000246 R12: ffff91c244034220
R13: 0000000000000030 R14: 0000000000000000 R15: 00000000000001d0
ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
#8 [ffffafb5096a3ae0] vol_ioship_copy_frags_to_ioship_buf at ffffffffc2cdda6b [vxio]
#9 [ffffafb5096a3b38] vol_disk_rhandle_read_resp at ffffffffc2ce7d75 [vxio]
#10 [ffffafb5096a3c68] vol_ioship_rrecv at ffffffffc2ce9fdc [vxio]
#11 [ffffafb5096a3e38] gab_lrrecv at ffffffffc32b8f52 [gab]
#12 [ffffafb5096a3e78] vx_ioship_process_data at ffffffffc0c7ab69 [llt]
#13 [ffffafb5096a3ed0] vx_ioship_recv_data at ffffffffc0c7b083 [llt]
#14 [ffffafb5096a3f10] kthread at ffffffff81712b3a
#15 [ffffafb5096a3f50] ret_from_fork at ffffffff82000255
crash>