Solaris 11 Sparc EnhancementSolaris 11.3 SRU 18.0.6 provides the timeout-noretry-list parameter, which is not currently available with Solaris 10.File location: /platform/sun4v/kernel/drv/vdc.conf

The vdc.conf file needs to be modified in the Solaris 11 GUEST domain (i.e. scooby) to reduce the chances of specific commands hanging in the logical domain when access to the Control/Service I/O domain is lost.
 

Sample vdc.conf update:

scooby # cat /platform/sun4v/kernel/drv/vdc.conf
#
# Copyright (c) 2011, Oracle and/or its affiliates. All rights reserved.
#
#
# Associate the driver with devid resolution.
#
ddi-devid-registrant=1;
timeout-noretry-list=0,1,2,3,4,5;
 

NOTE:

The “timeout-noretry-list” setting has been added to the vdc.conf file shown above. The Virtual Disk Client (VDC) for virtual disk instances 0,1,2,3,4 & 5 will NOW return an I/O error immediately (for each I/O), after the defined VDC timeout from the time the I/O domain is not accessible.

Oracle decided not to backport this functionality to Solaris 10 at this time. 

VDC TimeoutIn Oracle VM Server configurations the Virtual Disk Client (VDC) driver timeout is set to zero by default, which signifies infinity.

The VDC driver timeout must be defined in all cases to further minimize delays with returning a failure to the VDC client.

The VDC timeout can be set manually at a Virtual disk level (vdisk) using the “ldm” commands shown below:

Sample commands:

How to set the VDC timeout when creating the Virtual disk:

# ldm add-vdisk timeout=20 scoobydisk scoobyboot@primary-vds0 scooby

How to change the VDC timeout at a Virtual disk level:

# ldm set-vdisk timeout=30  scoobydisk scooby

NOTE: The ldm commands are executed from the Control (primary) domain.
 

The VDC timeout must be defined for all virtual disk instances to enable the timeout-noretry-list parameter to work correctly in the Solaris 11 LDOM Guest.

If not defined for Solaris 11 LDOM GUESTs, the failed I/O may not to return to the guest domain in the event that either the Control (primary) or Service I/O domain crashes (reboots) unexpectedly.

The VDC timeout can be defined in the /etc/system file in the LDOM GUEST domain (system wide globally parameter).

Sample file output:

# cat /etc/system
.
.
set vdc:vdc_timeout=30
set vdc:vdc_read_timeout=30
 

Even when the VDC (Virtual Disk Client) is defined, the Solaris 10 LDOM GUEST  may be unable to provide the required responses back upstream to the Veritas DMP (multi-pathing) interface. Preventing DMP from acting on SCSI failures returned by the lower layers up into the GUEST.

In the event that the Solaris 10 GUEST domain cannot get the failed I/O(s) back, the failed I/O cannot be routed through an alternate operational I/O domain. This can result in commands hanging, especially when an I/O domain is rebooted or it is panics.
 
 

To overcome the various interoperability limitations with Solaris 10 LDOM Guests, Veritas agreed to support Veritas DMP in connection with Solaris MPGROUPs for Solaris 10 LDOM GUESTs  only, caveats apply for specific Veritas product solutions.

Solaris 10: LDOM Guests configured with Clustered Volume Manager (CVM) & Clustered File System (CFS).

The presented Virtual disks must be exported using the DMP node path (/dev/vx/dmp/<dmpnode-name>s2) via a MPGROUP (for I/O domain redundancy) up into the Solaris 10 based GUEST.

Solaris 11: MPGROUPs is not supported in any capacity or configuration with Solaris 11 LDOM Guests.

NOTE: In order to exported Veritas dmpnodes, Veritas must be installed in the I/O Domains. MPxIO is not supported.

Solaris 10: How to configure Backend & Virtual disks with MPGROUPs:

Commands issued from the Control (Primary) domain.

# ldm add-vdsdev mpgroup=scrappyboot /dev/vx/dmp/emc_clariion0_47s2 scrappyboot@primary-vds0
# ldm add-vdsdev mpgroup=scrappyboot /dev/vx/dmp/emc_clariion0_47s2 scrappyboot@altio-vds0
# ldm add-vdisk timeout=30 scrappydisk scrappyboot@primary-vds0 scrappy

NOTE: Only a single Virtual disk resource is required when using MPGROUPs. The boot & data disks should only be managed by MPGROUPs with Solaris 10 GUEST deployments.
 

For Solaris 10 LDOM Guests presented with ZFS devices, dmp_native_support should not be enabled, as the disks will appear with a single path in the Solaris 10 LDOM Guest.

# vxdmpadm gettune dmp_native_support

# vxdmpadm settune dmp_native_support=off

With Solaris 11 LDOM GUESTs the exported disks must NOT be configured in connection with MPGROUPs.


Solaris 11: How to configure Backend & Virtual disks without MPGROUPs:

Commands issued from the Control (Primary) domain.

Exported from Primary VDS (Virtual Disk Server) I/O domain:

# ldm add-vdsdev /dev/vx/dmp/emc_clariion1_177s2 scrappy177@primary-vds0
# ldm add-vdsdev /dev/vx/dmp/emc_clariion1_253s2 scrappy253@primary-vds0

Exported from Alternate/Secondary VDS (Virtual Disk Server) I/O domain: 

# ldm add-vdsdev /dev/vx/dmp/emc_clariion1_177s2 scrappy177@altio-vds0
# ldm add-vdsdev /dev/vx/dmp/emc_clariion1_253s2 scrappy253@altio-vds0

Virtual disk resources exported from the Primary VDS to the GUEST domain scrappy:

# ldm add-vdisk timeout=30 scrappy-177-pri scrappy177@primary-vds0 scrappy
# ldm add-vdisk timeout=30 scrappy-253-pri scrappy253@primary-vds0 scrappy

Virtual disk resources exported from the Alternate (Secondary) VDS to the GUEST domain scrappy:

# ldm add-vdisk timeout=30 scrappy-177-alt scrappy177@altio-vds0 scrappy
# ldm add-vdisk timeout=30 scrappy-253-alt scrappy253@altio-vds0 scrappy

NOTE: The Virtual disk names need to be unique when not using MPGROUPs.
 

To support the handling of Solaris ZFS devices with Solaris 11 LDOM Guests, the DMP tunable dmp_native support must be enabled in Solaris 11 LDOM Guests (no MPGROUPs).
 

# vxdmpadm gettune dmp_native_support

# vxdmpadm settune dmp_native_support=on

 

The ZFS pool should then be imported using:

# zpool -d import /dev/vx/dmp

LDOMs known as Oracle VM Server for Sparc provides a Solaris feature called MPGROUPs. Traditionally MPGROUPs (multipathing group) were not supported/certified with the Veritas product stack in connection with LDOM environments. Figure 1.0

MPxIO is not supported with Solaris 10 or 11 LDOM configurations.

SCSI-3 based Fencing is NOT supported with MPGROUPs, so CVM & other configurations must use an alternate Fencing solutions, like Veritas CPS Servers.

See Article:
Veritas SCSI-3 Fencing disks are not supported with Solaris 10 LDOM Guest with or without Solaris MPGROUPs
https://www.veritas.com/support/en_US/article.100052433
Solaris 11.3 Development with Veritas DMP

Oracle released an interoperability enhancement for Solaris 11.3 with SRU 18.0.6 and higher.

This interoperability enhancement enabled Veritas Dynamic Multi-pathing (DMP) to better handle the loss of Primary (Control) and Service I/O Domains. Enabling Veritas Fencing to work once again (without MPGROUPs) with Solaris 11 LDOM guests onwards.

Solaris 11 Sparc Enhancement


Solaris 11.3 SRU 18.0.6 provides the timeout-noretry-list parameter, which is not currently available with Solaris 10.


File location: /platform/sun4v/kernel/drv/vdc.conf

The vdc.conf file needs to be modified in the Solaris 11 GUEST domain (i.e. scooby) to reduce the chances of specific commands hanging in the logical domain when access to the Control/Service I/O domain is lost.
Sample vdc.conf update:

scooby # cat /platform/sun4v/kernel/drv/vdc.conf
#
# Copyright (c) 2011, Oracle and/or its affiliates. All rights reserved.
#
#
# Associate the driver with devid resolution.
#
ddi-devid-registrant=1;
timeout-noretry-list=0,1,2,3,4,5;
NOTE:

The “timeout-noretry-list” setting has been added to the vdc.conf file shown above. The Virtual Disk Client (VDC) for virtual disk instances 0,1,2,3,4 & 5 will NOW return an I/O error immediately (for each I/O), after the defined VDC timeout from the time the I/O domain is not accessible.

Oracle decided not to backport this functionality to Solaris 10 at this time.

VDC Timeout

In Oracle VM Server configurations the Virtual Disk Client (VDC) driver timeout is set to zero by default, which signifies infinity.

The VDC driver timeout must be defined in all cases to further minimize delays with returning a failure to the VDC client.

The VDC timeout can be set manually at a Virtual disk level (vdisk) using the “ldm” commands shown below: Sample commands:

How to set the VDC timeout when creating the Virtual disk:

# ldm add-vdisk timeout=20 scoobydisk scoobyboot@primary-vds0 scooby

How to change the VDC timeout at a Virtual disk level:

# ldm set-vdisk timeout=30 scoobydisk scooby

NOTE: The ldm commands are executed from the Control (primary) domain.
The VDC timeout must be defined for all virtual disk instances to enable the timeout-noretry-list parameter to work correctly in the Solaris 11 LDOM Guest. If not defined for Solaris 11 LDOM GUESTs, the failed I/O may not to return to the guest domain in the event that either the Control (primary) or Service I/O domain crashes (reboots) unexpectedly. The VDC timeout can be defined in the /etc/system file in the LDOM GUEST domain (system wide globally parameter). Sample file output: # cat /etc/system
.
.
set vdc:vdc_timeout=30
set vdc:vdc_read_timeout=30