Solaris NVME devices not visible (claimed by libvxnvme_sol ASL) to Veritas Volume Manager (VxVM) following Solaris 11.4 SRU 42 update
Article ID: 100052956
Updated On:
Description
Error Message
The Device Discovery Layer (DDL) is unable to claim the previously discovered & claimed NVME devices since upgrading to Solaris SRU 42.
# vxddladm list devices | grep -i error
c12t1d0 - Online ERROR (libvxnvme_sol.so)
c14t1d0 - Online ERROR (libvxnvme_sol.so)
c15t1d0 - Online ERROR (libvxnvme_sol.so)
c13t1d0 - Online ERROR (libvxnvme_sol.so)
Sample SCSI inquiry command outputs
/dev/rdsk/c12t1d0s2
ioctl failed: Inappropriate ioctl for device
VxVM vxscsiinq ERROR V-5-1-8853 /usr/lib/vxvm/diag.d/vxscsiinq for /dev/rdsk/c12t1d0s2. ioctl failed., evpd 0 page code 0. Inappropriate ioctl for device
#exit=1
ioctl failed: Inappropriate ioctl for device
VxVM vxscsiinq ERROR V-5-1-8853 /usr/lib/vxvm/diag.d/vxscsiinq for /dev/rdsk/c12t1d0s2. ioctl failed., evpd 1 page code 80. Inappropriate ioctl for device
#exit=1
ioctl failed: Inappropriate ioctl for device
VxVM vxscsiinq ERROR V-5-1-8853 /usr/lib/vxvm/diag.d/vxscsiinq for /dev/rdsk/c12t1d0s2. ioctl failed., evpd 1 page code 83. Inappropriate ioctl for device
#exit=1
##############################################################
/dev/rdsk/c13t1d0s2
ioctl failed: Inappropriate ioctl for device
VxVM vxscsiinq ERROR V-5-1-8853 /usr/lib/vxvm/diag.d/vxscsiinq for /dev/rdsk/c13t1d0s2. ioctl failed., evpd 0 page code 0. Inappropriate ioctl for device
#exit=1
ioctl failed: Inappropriate ioctl for device
VxVM vxscsiinq ERROR V-5-1-8853 /usr/lib/vxvm/diag.d/vxscsiinq for /dev/rdsk/c13t1d0s2. ioctl failed., evpd 1 page code 80. Inappropriate ioctl for device
#exit=1
ioctl failed: Inappropriate ioctl for device
VxVM vxscsiinq ERROR V-5-1-8853 /usr/lib/vxvm/diag.d/vxscsiinq for /dev/rdsk/c13t1d0s2. ioctl failed., evpd 1 page code 83. Inappropriate ioctl for device
#exit=1
##############################################################
/dev/rdsk/c14t1d0s2
ioctl failed: Inappropriate ioctl for device
VxVM vxscsiinq ERROR V-5-1-8853 /usr/lib/vxvm/diag.d/vxscsiinq for /dev/rdsk/c14t1d0s2. ioctl failed., evpd 0 page code 0. Inappropriate ioctl for device
#exit=1
ioctl failed: Inappropriate ioctl for device
VxVM vxscsiinq ERROR V-5-1-8853 /usr/lib/vxvm/diag.d/vxscsiinq for /dev/rdsk/c14t1d0s2. ioctl failed., evpd 1 page code 80. Inappropriate ioctl for device
#exit=1
ioctl failed: Inappropriate ioctl for device
VxVM vxscsiinq ERROR V-5-1-8853 /usr/lib/vxvm/diag.d/vxscsiinq for /dev/rdsk/c14t1d0s2. ioctl failed., evpd 1 page code 83. Inappropriate ioctl for device
#exit=1
##############################################################
/dev/rdsk/c15t1d0s2
ioctl failed: Inappropriate ioctl for device
VxVM vxscsiinq ERROR V-5-1-8853 /usr/lib/vxvm/diag.d/vxscsiinq for /dev/rdsk/c15t1d0s2. ioctl failed., evpd 0 page code 0. Inappropriate ioctl for device
#exit=1
ioctl failed: Inappropriate ioctl for device
VxVM vxscsiinq ERROR V-5-1-8853 /usr/lib/vxvm/diag.d/vxscsiinq for /dev/rdsk/c15t1d0s2. ioctl failed., evpd 1 page code 80. Inappropriate ioctl for device
#exit=1
ioctl failed: Inappropriate ioctl for device
VxVM vxscsiinq ERROR V-5-1-8853 /usr/lib/vxvm/diag.d/vxscsiinq for /dev/rdsk/c15t1d0s2. ioctl failed., evpd 1 page code 83. Inappropriate ioctl for device
#exit=1
##############################################################
Cause
The issue is unknown at this time. Veritas is working closely with Oracle Support to troubleshoot the issue.
A change in Solaris 11.4 SRU 42 appears to prevent SCSI IOCTL calls on NVME related devices.
No change has been introduced in Veritas.
The Solaris nvmeadm command can be used to display NVMe related device information:
# nvmeadm list -v
Resolution
Veritas engineering has provided an updated ASLAPM package for InfoScale 7.41 & 8.0.which will now work with SRU 42 onwards and below.
InfoScale 7.4.1 environments: download and apply
ASLAPM PATCH ID: 7.4.1.3401
Link: https://www.veritas.com/content/support/en_US/downloads/update.UPD184578
Incident ASL/APM Abstract
4076838 libvxnvme_sol Existing configured NVME devices no longer visible (libvxnvme_sol.so :ERROR state) after SRU 42 update
NOTE: Veritas is also working on a revised ASLAPM patch for InfoScale 7.4.2, where Solaris 11.4 SRU 42 or higher is deployed.
InfoScale 8.0 environments: download and apply
ASLAPM PATCH ID: 8.0.0.1601
Link: https://www.veritas.com/content/support/en_US/downloads/update.UPD113073
Incident ASL/APM Abstract
4076021 libvxnvme_sol configured NVME devices no longer visible (libvxnvme_sol.so :ERROR state) after SRU 42 update
NOTE: Solaris 11.4 SRU 41 and lower is fine and stable.
To see the existing NVME devices, revert back to the previous patch level (11.4-11.4.41.0.1.107.2 or earlier) until a revised ASLAPM can be released for the specific InfoScale product release.
The previously discovered NVME devices should now be visible with VxVM once again.
Veritas recommends creating Alternate Boot Environments (ABEs) when upgrading the Solaris patch level or InfoScale versions.
See related article:
How to create and manage Solaris Alternate Boot Environment (ABE) using the beadm utility
https://www.veritas.com/support/en_US/article.100047977
Issue/Introduction
Solaris NVME devices previously visible to Veritas Volume Manager (VxVM) with Solaris 11.4 SRU 41 or lower, are no longer visible to VxVM following Solaris 11.4 SRU 42 update
InfoScale (IS) 7.3.1 introduced support for NVMe (Non-Volatile Memory Express) devices with Solaris Sparc servers. Uses a servers high-speed Peripheral Component Interconnect Express (PCIe) interface.
The Solaris "echo | format" command is able to see the NVME related devices, however, VxVM is not once upgraded to Solaris 11.4 SRU 42.
Sample output # echo | format | grep -i nvme
/pci@306/pci@1/pci@0/pci@6/nvme@0/disk@1
/dev/chassis/SYS/DBP/NVME0/disk
/pci@306/pci@1/pci@0/pci@7/nvme@0/disk@1
/dev/chassis/SYS/DBP/NVME1/disk
/pci@307/pci@1/pci@0/pci@6/nvme@0/disk@1
/dev/chassis/SYS/DBP/NVME2/disk
/pci@307/pci@1/pci@0/pci@7/nvme@0/disk@1
/dev/chassis/SYS/DBP/NVME3/diskSolaris 11 update (SRU 41) 11.4-11.4.41.0.1.107.2 & below is fine both Solaris & Veritas Solaris 11 update (SRU 42) 11.4-11.4.42.0.1.113.1 & higher breaks VxVM's ability to claim NVME devices
Additional Information
