EMC Metro/Geo Cluster: DMP STANDBY flag is set for remotely attached EMC array paths where the DMP latency_threshold_difference is reached
Article ID: 100053347
Updated On:
Description
Error Message
Sample messages
From: /var/log/messages
Jun 20 21:41:53 fred kernel: VxVM vxdmp V-5-0-34 [Info] added disk array 000297600070, datype = EMC
Jun 20 21:41:53 fred kernel:
Jun 20 21:41:53 fred kernel: VxVM vxdmp V-5-3-2379 dmp_path_add_element: port_id: 000297600072, path_latency: 165
Jun 20 21:41:53 fred kernel:
Jun 20 21:41:53 fred kernel: VxVM vxdmp V-5-3-2376 dmp_get_path_latency: Latency for LocationID 000297600072 = 165 microsecs, enclr_threshold = 200
Jun 20 21:41:53 fred kernel:
Jun 20 21:41:53 fred kernel: VxVM vxdmp V-5-3-2379 dmp_path_add_element: port_id: 000297600070, path_latency: 374
Jun 20 21:41:53 fred kernel:
Jun 20 21:41:53 fred kernel: VxVM vxdmp V-5-3-2376 dmp_get_path_latency: Latency for LocationID 000297600070 = 374 microsecs, enclr_threshold = 200
Metro/Geo configuration serves I/O through the local path, if the remote paths have higher latency than the defined latency_threshold_difference tunable.
Cause
InfoScale 8.0 development
New ASL (Array Support LIbrary) & APM (Array Policy Module) were required to support for SRDF metro/geo cluster array configurations.
This enables Dynamic Multi-Pathing (DMP) with an I/O array policy of A/A (active/active), to evenly distribute I/O is across both local and remote array paths.
As PMAX & VMAX arrays are both A/A arrays, DMP will send I/O against all paths.
Infoscale needed to develop intelligence to differentiate between local and remote paths.
The new APM has been enhanced to identify local and remote array paths and set the "STANDBY" flag on remote path for better IO throughput where latency is a concern.
If path latency is detected for any path greater than the latency threshold, DMP will consider it as a remote path and mark those paths with a "STANDBY" flag.
To address any potential performance impact and improve I/O throughput, DMP will adapt and selectively distrubute I/O across the local paths only.
For high performance, it is beneficial for DMP to send I/O on local paths only, especially where paths coming from remote locations may have higher latency
SRDF Details:
Symmetrix Remote Data Facility (SRDF) solutions provide disaster recovery and data mobility solutions for Dell EMC™ PowerMax, VMAX, VMAX3, and VMAX All Flash arrays.
SRDF services are provided by the following operating environments:
• PowerMax OS for PowerMax 2000 and PowerMax 8000
• HYPERMAX OS for VMAX All Flash VMAX 250F, VMAX 250FX, VMAX 450F, VMAX 450 FX, VMAX 850F, and VMAX 850 FX
SRDF synchronous (SRDF/S) maintains a real-time copy at arrays located within 200 kilometers.
Writes from the production host are acknowledged from the local array when they are written to cache at the remote array.
SRDF asynchronous (SRDF/A) maintains a dependent-write consistent copy at arrays located at unlimited distances.
Writes from the production host are acknowledge immediately by the local array, thus replication has no impact on host performance.
Data at the remote array is typically only seconds behind the primary site.
SRDF/Metro
With both set of devices being accessible to the host or hosts (in the case of a cluster), read and write operations can be performed to both the primary and secondary devices.
SRDF/Metro ensures that each copy remains current, consistent, and addresses any write conflicts which may occur between the paired SRDF devices.
Resolution
The DMP "latency_threshold_difference" enclosure tunable lets you manage the parameter values of the latency threshold for specific enclosures.
The default value of the latency_threshold_difference tunable parameter is 200 µs.
The latency_threshold_difference tunable is valid only for the METRO type arrays.
The following is the syntax for the tuning latency_threshold_difference:
# vxdmpadm getattr enclosure {enclosure-name} latency_threshold_difference
Using the latency_threshold_difference tunable option, you can increase or decrease the value for a specified enclosure:
# vxdmpadm getattr enclosure emc0
ENCLR_NAME ATTR_NAME DEFAULT CURRENT
============================================================================
emc0 iopolicy MinimumQ MinimumQ
emc0 partitionsize 512 512
emc0 use_all_paths - -
emc0 recoveryoption[throttle] Nothrottle[0] Nothrottle[0]
emc0 recoveryoption[errorretry] Timebound[300] Timebound[300]
emc0 redundancy 0 0
emc0 dmp_lun_retry_timeout 0 0
emc0 failovermode - -
emc0 latency_threshold_difference 200 200
NOTE: Different threshold values can be configured for specific enclosures.
The local EMC array can be seen with an Enclosure Serial No (ENCLR_SNO) of "000297600070"
#vxdmpadm listenclosure all outputENCLR_NAME ENCLR_TYPE ENCLR_SNO STATUS ARRAY_TYPE LUN_COUNT FIRMWARE
===================================================================================================
disk Disk DISKS CONNECTED Disk 5 00-0
emc0 EMC 000297600070 CONNECTED VMAX-A/A 5 5978
The "/etc/vx/diag.d/vxdmpdump arrayport enclosure=" CLI command can be used to determine the Location IDs for local and remote ports.
Sample output
# /etc/vx/diag.d/vxdmpdump arrayport enclosure=emc0 ARRAY PORT - 1
portid = 4d-000297600070
enclr_cabsno = 000297600070
portwwn = #############################
ARRAY PORT - 2
portid = 13d-000297600070
enclr_cabsno = 000297600070
portwwn = #############################
ARRAY PORT - 3
portid = 4d-000297600072
enclr_cabsno = 000297600070
portwwn = #############################
ARRAY PORT - 4
portid = 13d-000297600072
enclr_cabsno = 000297600070
portwwn = ##############################
The "/etc/vx/diag.d/vxdmpdump arrayport path=###" CLI command can be used to determine the Location IDs for local and remote paths.
# /etc/vx/diag.d/vxdmpdump arrayport path=###
How to clear STANDBY flag using vxdmpadm for DMP paths where DMP latency_threshold_difference was reached
Article : 100053363
Issue/Introduction
The DMP "STANDBY" flag is set for remotely attached EMC array paths where the DMP "latency_threshold_difference" is reached. EMC Metro/Geo cluster configurations have visibility of paths from both local and remote arrays. • This feature is supported on latest VMAX/PMAX arrays with revision 5977 onwards
Sample output # vxdmpadm getsubpaths ctlr=c16NAME STATE[A] PATH-TYPE[M] DMPNODENAME ENCLR-TYPE ENCLR-NAME ATTRS PRIORITY
===================================================================================================
sdp ENABLED(A) - emc0_129e EMC emc0 - -
sdu ENABLED - emc0_129e EMC emc0 STANDBY -
sdq ENABLED(A) - emc0_129f EMC emc0 - -
sdv ENABLED - emc0_129f EMC emc0 STANDBY -
sdr ENABLED(A) - emc0_12a0 EMC emc0 - -
sdw ENABLED - emc0_12a0 EMC emc0 STANDBY -
sds ENABLED(A) - emc0_12a1 EMC emc0 - -
sdx ENABLED - emc0_12a1 EMC emc0 STANDBY -
sdt ENABLED(A) - emc0_12a2 EMC emc0 - -
sdy ENABLED - emc0_12a2 EMC emc0 STANDBY -
# vxdmpadm getsubpaths ctlr=c7NAME STATE[A] PATH-TYPE[M] DMPNODENAME ENCLR-TYPE ENCLR-NAME ATTRS PRIORITY
===================================================================================================
sdf ENABLED(A) - emc0_129e EMC emc0 - -
sdk ENABLED - emc0_129e EMC emc0 STANDBY -
sdg ENABLED(A) - emc0_129f EMC emc0 - -
sdl ENABLED - emc0_129f EMC emc0 STANDBY -
sdh ENABLED(A) - emc0_12a0 EMC emc0 - -
sdm ENABLED - emc0_12a0 EMC emc0 STANDBY -
sdi ENABLED(A) - emc0_12a1 EMC emc0 - -
sdn ENABLED - emc0_12a1 EMC emc0 STANDBY -
sdj ENABLED(A) - emc0_12a2 EMC emc0 - -
sdo ENABLED - emc0_12a2 EMC emc0 STANDBY -
