Error Message

Sample messages

From: /var/log/messages

Jun 20 21:41:53 fred kernel: VxVM vxdmp V-5-0-34 [Info] added disk array 000297600070, datype = EMC
Jun 20 21:41:53 fred kernel: 
Jun 20 21:41:53 fred kernel: VxVM vxdmp V-5-3-2379 dmp_path_add_element: port_id: 000297600072, path_latency: 165
Jun 20 21:41:53 fred kernel: 
Jun 20 21:41:53 fred kernel: VxVM vxdmp V-5-3-2376 dmp_get_path_latency: Latency for LocationID 000297600072 = 165 microsecs, enclr_threshold = 200
Jun 20 21:41:53 fred kernel: 
Jun 20 21:41:53 fred kernel: VxVM vxdmp V-5-3-2379 dmp_path_add_element: port_id: 000297600070, path_latency: 374
Jun 20 21:41:53 fred kernel: 
Jun 20 21:41:53 fred kernel: VxVM vxdmp V-5-3-2376 dmp_get_path_latency: Latency for LocationID 000297600070 = 374 microsecs, enclr_threshold = 200

Metro/Geo configuration serves I/O through the local path, if the remote paths have higher latency than the defined latency_threshold_difference tunable.

InfoScale 8.0 development
 

New ASL (Array Support LIbrary) & APM (Array Policy Module) were required to support for SRDF metro/geo cluster array configurations.

This enables Dynamic Multi-Pathing (DMP) with an I/O array policy of A/A (active/active), to evenly distribute I/O is  across both local and remote array paths. 

As PMAX & VMAX arrays are both A/A arrays, DMP will send I/O against all paths.
 

Veritas needed to develop intelligence to differentiate between local and remote paths.

The new APM has been enhanced to identify local and remote array paths and set the "STANDBY" flag on remote path for better IO throughput where latency is a concern.

If path latency is detected for any path greater than the latency threshold, DMP will consider it as a remote path and mark those paths with a "STANDBY" flag.

To address any potential performance impact and improve I/O throughput, DMP will adapt and selectively distrubute I/O across the local paths only.

For high performance, it is beneficial for DMP to send I/O on local paths only, especially where paths coming from remote locations may have higher latency

SRDF Details:

Symmetrix Remote Data Facility (SRDF) solutions provide disaster recovery and data mobility solutions for Dell EMC™ PowerMax, VMAX, VMAX3, and VMAX All Flash arrays. 

SRDF services are provided by the following operating environments: 

• PowerMax OS for PowerMax 2000 and PowerMax 8000 
• HYPERMAX OS for VMAX All Flash VMAX 250F, VMAX 250FX, VMAX 450F, VMAX 450 FX, VMAX 850F, and VMAX 850 FX 

SRDF synchronous (SRDF/S) maintains a real-time copy at arrays located within 200 kilometers. 
Writes from the production host are acknowledged from the local array when they are written to cache at the remote array. 

SRDF asynchronous (SRDF/A) maintains a dependent-write consistent copy at arrays located at unlimited distances. 
Writes from the production host are acknowledge immediately by the local array, thus replication has no impact on host performance. 
Data at the remote array is typically only seconds behind the primary site. 

SRDF/Metro

With both set of devices being accessible to the host or hosts (in the case of a cluster), read and write operations can be performed to both the primary and secondary devices.

SRDF/Metro ensures that each copy remains current, consistent, and addresses any write conflicts which may occur between the paired SRDF devices. 

The DMP "latency_threshold_difference" enclosure tunable lets you manage the parameter values of the latency threshold for specific enclosures.

The default value of the latency_threshold_difference tunable parameter is 200 µs. 

The latency_threshold_difference tunable is valid only for the METRO type arrays.

The following is the syntax for the tuning latency_threshold_difference:

# vxdmpadm getattr enclosure {enclosure-name} latency_threshold_difference

Using the latency_threshold_difference tunable option, you can increase or decrease the value for a specified enclosure:

# vxdmpadm getattr enclosure emc0
ENCLR_NAME      ATTR_NAME                     DEFAULT        CURRENT
============================================================================
emc0           iopolicy                      MinimumQ       MinimumQ
emc0           partitionsize                 512            512           
emc0           use_all_paths                 -              -             
emc0           recoveryoption[throttle]      Nothrottle[0]  Nothrottle[0]
emc0           recoveryoption[errorretry]    Timebound[300] Timebound[300] 
emc0           redundancy                    0              0
emc0           dmp_lun_retry_timeout         0              0
emc0           failovermode                  -              -              
emc0           latency_threshold_difference  200            200

NOTE: Different threshold values can be configured for specific enclosures.

The local EMC array can be seen with an Enclosure Serial No (ENCLR_SNO) of "000297600070"

#vxdmpadm listenclosure all output
ENCLR_NAME        ENCLR_TYPE     ENCLR_SNO      STATUS       ARRAY_TYPE     LUN_COUNT    FIRMWARE 
===================================================================================================
disk              Disk           DISKS                CONNECTED    Disk        5         00-0
emc0              EMC            000297600070         CONNECTED    VMAX-A/A    5         5978

The "/etc/vx/diag.d/vxdmpdump arrayport enclosure=" CLI command can be used to determine the Location IDs for local and remote ports.
 

Sample output

# /etc/vx/diag.d/vxdmpdump arrayport enclosure=emc0
    ARRAY PORT - 1 
    portid            = 4d-000297600070
    enclr_cabsno        = 000297600070
    portwwn            = #############################
    ARRAY PORT - 2 
    portid            = 13d-000297600070
    enclr_cabsno        = 000297600070
    portwwn            = ############################# 
    ARRAY PORT - 3 
    portid            = 4d-000297600072
    enclr_cabsno        = 000297600070
    portwwn            = ############################# 
    ARRAY PORT - 4 
    portid            = 13d-000297600072
    enclr_cabsno        = 000297600070
    portwwn            = ############################## 

The "/etc/vx/diag.d/vxdmpdump arrayport path=###" CLI command can be used to determine the Location IDs for local and remote paths.

# /etc/vx/diag.d/vxdmpdump arrayport path=###

How to clear STANDBY flag using vxdmpadm for DMP paths where DMP latency_threshold_difference was reached
https://www.veritas.com/support/en_US/article.100053363