Description

This document attempts to explain the steps required to failover the EMC SRDF role when using Veritas Volume Manager (VxVM) and DMP.

EMC SRDF logically pairs a device or a group of devices from each array and replicates data from one to the other synchronously or asynchronously. An established pair of devices can be split, so that separate hosts can access the same data independently (maybe for backup), and then resynchronised.

In this instance, the configuration consists of "4" EMC SRDF "R1" devices presented to production server "Dopey" from EMC Symmetrix id (SID) "822".
An additional "4" EMC SRDF "R2" devices have been presented to DR server "Bashful" from EMC Symmetrix ID (SID) "414".

During the SRDF failover operation the R1 devices are write-disabled, the SRDF link is set to Not-Ready (NR), then the R2 devices are write-enabled.

In synchronous mode (SRDF/S), the primary array waits until the secondary array has acknowledged each write before the next write is accepted, ensuring that the replicated copy of the data is always as current as the primary. However, the latency due to propagation increases significantly with distance.

In asynchronous mode (SRDF/A) transfers the changes to the secondary array in units called delta sets, which are transferred at defined intervals. Although the remote copy of the data will never be as current as the primary copy, this method can replicate data over considerable distances and with reduced bandwidth requirements and mimimal impact on host performance.


Figure 1.0






Note: As of 5.0 MP3 onwards, the VxVM CLI command "vxdisk -e list" is capable of reporting the extended attribute based content for EMC BCV, BCV-NR, MIRROR, SRDF-R1, SRDF-R2, TDEV device states and EMC lun types.


Example


In this instance, the local EMC SYMDEVs are 008C, 008D, 008E and 008F. The remote EMC SYMDEVs are 0072, 0073, 0074 and 0075


The local VxVM Diskgroup named "DopeyDg", consists of four disks replicated to the remote host "Bashful".


1.] The VxVM disks at the local site (Dopey in this instance) are write-enabled.


Local:

Dopey # vxdisk -eg DopeyDg list
DEVICE       TYPE           DISK        GROUP        STATUS               OS_NATIVE_NAME   ATTR
emc0_008c    auto:cdsdisk   r1-008C      DopeyDg     online               c1t5006048C5368E580d266s2 srdf-r1
emc0_008d    auto:cdsdisk   r1-008D      DopeyDg     online               c1t5006048C5368E5A0d223s2 srdf-r1
emc0_008e    auto:cdsdisk   r1-008E      DopeyDg     online               c1t5006048C5368E5A0d229s2 srdf-r1
emc0_008f    auto:cdsdisk   r1-008f      DopeyDg     online               c1t5006048C5368E580d269s2 srdf-r1



2.] The VxVM disks at the remote site (Bashful in this instance) are write-disabled at this time.


Remote:


Bashful # vxdisk -eo alldgs list | grep DopeyDg
emc1_0072    auto:cdsdisk   -            (DopeyDg)   online               c1t5006048C536979A0d65s2 srdf-r2
emc1_0073    auto:cdsdisk   -            (DopeyDg)   online               c1t5006048C536979A0d66s2 srdf-r2
emc1_0074    auto:cdsdisk   -            (DopeyDg)   online               c1t5006048C536979A0d67s2 srdf-r2
emc1_0075    auto:cdsdisk   -            (DopeyDg)   online               c1t5006048C536979A0d68s2 srdf-r2


Note: The remote copy of the diskgroup "DopeyDg" cannot be imported at this time.


Bashful # vxdg import DopeyDg
VxVM vxdg ERROR V-5-1-10978 Disk group DopeyDg: import failed:
Disk is in use by another host


Attempting to force import the diskgroup on the remote host will result in the disks being marked by VxVM as "error".


Bashful #  vxdg -Cf import DopeyDg
VxVM vxdg ERROR V-5-1-10978 Disk group DopeyDg: import failed:
No valid disk found containing disk group


3.] The previous replicated state was "synchronized" as shown by "symrdf query".


Local:


Dopey # symrdf -g DopeyDg query


Device Group (DG) Name             : DopeyDg
DG's Type                          : RDF1
DG's Symmetrix ID                  : 000290300822    (Microcode Version: 5773)
Remote Symmetrix ID                : 000290301414    (Microcode Version: 5773)
RDF (RA) Group Number              :  11 (0A)


       Source (R1) View                 Target (R2) View     MODES
--------------------------------    ------------------------ ----- ------------
             ST                  LI      ST
Standard      A                   N       A
Logical       T  R1 Inv   R2 Inv  K       T  R1 Inv   R2 Inv       RDF Pair
Device  Dev   E  Tracks   Tracks  S Dev   E  Tracks   Tracks MDA   STATE
-------------------------------- -- ------------------------ ----- ------------

DEV08C  008C RW       0        0 RW 0072 WD       0        0 S..   Synchronized
DEV08D  008D RW       0        0 RW 0073 WD       0        0 S..   Synchronized
DEV08E  008E RW       0        0 RW 0074 WD       0        0 S..   Synchronized
DEV08F  008F RW       0        0 RW 0075 WD       0        0 S..   Synchronized

Total          -------- --------           -------- --------
  Track(s)            0        0                  0        0
  MB(s)             0.0      0.0                0.0      0.0

Legend for MODES:

 M(ode of Operation): A = Async, S = Sync, E = Semi-sync, C = Adaptive Copy
 D(omino)           : X = Enabled, . = Disabled
 A(daptive Copy)    : D = Disk Mode, W = WP Mode, . = ACp off



4.] To failover the "write-enabled" status to the remote host, use "symrdf failover".


Note: Prior to failing over the SRDF role, ensure the VxVM Diskgroup is not imported on the local host.
In most circumstances the local host will not be accessible/available at the time of the failover.


Remote:


Bashful # symrdf -g DopeyDg failover

Execute an RDF 'Failover' operation for device group 'DopeyDg' (y/[n]) ? y

An RDF 'Failover' operation execution is in progress for device group 'DopeyDg'. Please wait...

    Write Disable device(s) on SA at source (R1)..............Done.
    Suspend RDF link(s).......................................Done.
    Read/Write Enable device(s) on RA at target (R2)..........Done.

The RDF 'Failover' operation successfully executed for device group 'DopeyDg'.


Note: Once complete, the replicated state will report "Failed Over" as shown by "symrdf query"


Bashful # symrdf -g DopeyDg query


Device Group (DG) Name             : DopeyDg
DG's Type                          : RDF2
DG's Symmetrix ID                  : 000290301414    (Microcode Version: 5773)
Remote Symmetrix ID                : 000290300822    (Microcode Version: 5773)
RDF (RA) Group Number              :  11 (0A)


       Target (R2) View                 Source (R1) View     MODES
--------------------------------    ------------------------ ----- ------------
             ST                  LI      ST
Standard      A                   N       A
Logical       T  R1 Inv   R2 Inv  K       T  R1 Inv   R2 Inv       RDF Pair
Device  Dev   E  Tracks   Tracks  S Dev   E  Tracks   Tracks MDA   STATE
-------------------------------- -- ------------------------ ----- ------------

DEV072  0072 RW       0        0 NR 008C WD       0        0 S..   Failed Over
DEV073  0073 RW       0        0 NR 008D WD       0        0 S..   Failed Over
DEV074  0074 RW       0        0 NR 008E WD       0        0 S..   Failed Over
DEV075  0075 RW       0        0 NR 008F WD       0        0 S..   Failed Over

Total          -------- --------           -------- --------
  Track(s)            0        0                  0        0
  MB(s)             0.0      0.0                0.0      0.0

Legend for MODES:

 M(ode of Operation): A = Async, S = Sync, E = Semi-sync, C = Adaptive Copy
 D(omino)           : X = Enabled, . = Disabled
 A(daptive Copy)    : D = Disk Mode, W = WP Mode, . = ACp off


5.] Refresh the VxVM content prior to importing the VxVM Diskgroup on the remote server.


Remote:


Bashful # vxdisk scandisks

Bashful # vxdisk -eo alldgs list | grep DopeyDg
emc1_0072    auto:cdsdisk   -            (DopeyDg)   online               c1t5006048C536979A0d65s2 srdf-r2
emc1_0073    auto:cdsdisk   -            (DopeyDg)   online               c1t5006048C536979A0d66s2 srdf-r2
emc1_0074    auto:cdsdisk   -            (DopeyDg)   online               c1t5006048C536979A0d67s2 srdf-r2
emc1_0075    auto:cdsdisk   -            (DopeyDg)   online               c1t5006048C536979A0d68s2 srdf-r2

Bashful # vxdg -Cf import DopeyDg

Bashful # vxdisk -eo alldgs list | grep -w DopeyDg
emc1_0072    auto:cdsdisk   r1-008C      DopeyDg     online               c1t5006048C536979A0d65s2 srdf-r2
emc1_0073    auto:cdsdisk   r1-008D      DopeyDg     online               c1t5006048C536979A0d66s2 srdf-r2
emc1_0074    auto:cdsdisk   r1-008E      DopeyDg     online               c1t5006048C536979A0d67s2 srdf-r2
emc1_0075    auto:cdsdisk   r1-008f      DopeyDg     online               c1t5006048C536979A0d68s2 srdf-r2



6.] Recover the VxVM volume(s) on the remote server.


Remote:


Bashful #  vxprint -qhtg DopeyDg
dg DopeyDg      default      default  36000    1298306534.57.Dopey
dm r1-008C      emc1_0072    auto     65536    4058368  -
dm r1-008D      emc1_0073    auto     65536    4058368  -
dm r1-008E      emc1_0074    auto     65536    4058368  -
dm r1-008f      emc1_0075    auto     65536    4058368  -

v  vol01        -            DISABLED ACTIVE   2097152  SELECT    -        fsgen
pl vol01-01     vol01        DISABLED ACTIVE   2097152  CONCAT    -        RW
sd r1-008C-01   vol01-01     r1-008C  0        2097152  0         emc1_0072 ENA


Bashful # vxrecover -g DopeyDg -bs


Bashful # fsck -F vxfs /dev/vx/rdsk/DopeyDg/vol01 /vol01
log replay in progress
replay complete - marking super-block as CLEAN


Bashful #  mount -F vxfs /dev/vx/dsk/DopeyDg/vol01 /vol01


Bashful # ls /vol01
hosts       lost+found  vfstab



Process complete.