How to fix a rlink with disconnected resync_paused flags set?

book

Article ID: 100021052

calendar_today

Updated On:

Description

Error Message

 

flags: write enabled attached consistent disconnected asynchronous autosync resync_paused

 

Resolution

 

Note: For unix and linux platforms only

vxprint -ht outputs on both the primary and the secondary VVR servers show that everything is in enabled and active state however, the rlink is not in "CONNECT" state (see example output below).

VVR Primary:

rvrvg          1            ENABLED  ACTIVE  primary  1         srlvol
rl rlk_jung-v240_rvgrvg      
rvrvg          1            ENABLED  ACTIVE  primary  1         srlvol
rl rlk_jung-v240_rvgrvg     CONNECT  ACTIVE   jung-v240 harry_vvrdgrlk_fan-v240_rvg
v  vvrvol2      rvg          ENABLED  ACTIVE  204800   SELECT    -        fsgen
plvvrvol2-01   vvrvol2      ENABLED  ACTIVE  204800   CONCAT    -        RW
sdvvr3-01      vvrvol2-01   vvr3    0        204800   0        c2t50060E80004372C0d0 ENA
pl vvrvol2-03  vvrvol2      ENABLED  ACTIVE  LOGONLY  CONCAT    -        RW
sdvvr2-02      vvrvol2-03   vvr2     204800  32       LOG       c2t50060E80004372C0d5 ENA
plvvrvol2-04   vvrvol2      ENABLED  ACTIVE  LOGONLY  CONCAT    -        RW
sdvvr3-02      vvrvol2-04   vvr3     204800  32       LOG       c2t50060E80004372C0d0 ENA
dcvvrvol2_dco  vvrvol2      vvrvol2_dcl
v  vvrvol2_dcl  -            ENABLED  ACTIVE  544      SELECT    -        gen
plvvrvol2_dcl-01 vvrvol2_dcl ENABLED ACTIVE  544      CONCAT    -        RW
sdvvr3-04      vvrvol2_dcl-01 vvr3   204832  544      0         c2t50060E80004372C0d0ENA
pl vvrvol2_dcl-02 vvrvol2_dcl ENABLED ACTIVE  544      CONCAT    -        RW
sdvvr1-04      vvrvol2_dcl-02 vvr1   717184  544      0         c2t50060E80004372C0d4ENA
v  srlvol      rvg          ENABLED  ACTIVE   500016  SELECT    -        SRL
plsrlvol-01    srlvol       ENABLED  ACTIVE  500016   CONCAT    -        RW
sdvvr1-01      srlvol-01    vvr1    0        409632   0        c2t50060E80004372C0d4 ENA
sdvvr3-03      srlvol-01    vvr3    616064   90384    409632    c2t50060E80004372C0d0ENA
 ACTIVE   jung-v240 harry_vvrdgrlk_fan-v240_rvg
v  vvrvol2      rvg          ENABLED  ACTIVE  204800   SELECT    -        fsgen
plvvrvol2-01   vvrvol2      ENABLED  ACTIVE  204800   CONCAT    -        RW
sdvvr3-01      vvrvol2-01   vvr3    0        204800   0        c2t50060E80004372C0d0 ENA
pl vvrvol2-03  vvrvol2      ENABLED  ACTIVE  LOGONLY  CONCAT    -        RW
sdvvr2-02      vvrvol2-03   vvr2     204800  32       LOG       c2t50060E80004372C0d5 ENA
plvvrvol2-04   vvrvol2      ENABLED  ACTIVE  LOGONLY  CONCAT    -        RW
sdvvr3-02      vvrvol2-04   vvr3     204800  32       LOG       c2t50060E80004372C0d0 ENA
dcvvrvol2_dco  vvrvol2      vvrvol2_dcl
v  vvrvol2_dcl  -            ENABLED  ACTIVE  544      SELECT    -        gen
plvvrvol2_dcl-01 vvrvol2_dcl ENABLED ACTIVE  544      CONCAT    -        RW
sdvvr3-04      vvrvol2_dcl-01 vvr3   204832  544      0         c2t50060E80004372C0d0ENA
pl vvrvol2_dcl-02 vvrvol2_dcl ENABLED ACTIVE  544      CONCAT    -        RW
sdvvr1-04      vvrvol2_dcl-02 vvr1   717184  544      0         c2t50060E80004372C0d4ENA
v  srlvol      rvg          ENABLED  ACTIVE   500016  SELECT    -        SRL
plsrlvol-01    srlvol       ENABLED  ACTIVE  500016   CONCAT    -        RW
sdvvr1-01      srlvol-01    vvr1    0        409632   0        c2t50060E80004372C0d4 ENA
sdvvr3-03      srlvol-01    vvr3    616064   90384    409632    c2t50060E80004372C0d0ENA

VVR secondary:

rvrvg          1            ENABLED  ACTIVE  secondary 1        srlvol
rl rlk_fan-v240_rvgrvg      ENABLED  ACTIVE   fan-v240 frank_vvrdgrlk_jung-v240_rvg
v  vvrvol2      rvg          ENABLED  ACTIVE  204800   SELECT    -        fsgen
plvvrvol2-01   vvrvol2      ENABLED  ACTIVE  204800   CONCAT    -        RW
sdDisk_8-01    vvrvol2-01   Disk_8  0        204800   0        c2t50060E80004372C0d3 ENA
pl vvrvol2-02  vvrvol2      ENABLED  ACTIVE  LOGONLY  CONCAT    -        RW
sdDisk_8-05    vvrvol2-02   Disk_8   716832  32       LOG       c2t50060E80004372C0d3 ENA
dcvvrvol2_dco  vvrvol2      vvrvol2_dcl
v  vvrvol2_dcl  -            ENABLED  ACTIVE  544      SELECT    -        gen
plvvrvol2_dcl-01 vvrvol2_dcl ENABLED ACTIVE  544      CONCAT    -        RW
sdDisk_7-02    vvrvol2_dcl-01 Disk_7544      544      0        c2t50060E80004372C0d6 ENA
pl vvrvol2_dcl-02 vvrvol2_dcl ENABLED ACTIVE  544      CONCAT    -        RW
sdtest-02      vvrvol2_dcl-02 test   204800  544      0         c2t50060E80004372C0d1ENA
sp VD0108225128-vvrvol2_snp vvrvol2vvrvol2_dco
v  srlvol      rvg          ENABLED  ACTIVE   500016  SELECT    -        SRL
plsrlvol-01    srlvol       ENABLED  ACTIVE  500016   CONCAT    -        RW
sdDisk_10-01  srlvol-01    Disk_10  0        500016  0         c2t50060E80004372C0d7 ENA

vradminprintrvg shows no error:

root@fan-v240 /var/vx/vras/log [35091]:vradminprintrvg
Replicated Data Set:rvg
Primary:
       HostName:fan-v240      
       RvgName:rvg
       DgName:frank_vvrdg
Secondary:
       HostName:jung-v240
       RvgName:rvg
       DgName: harry_vvrdg

But vxprint -lP shows that the rlink is not connected with disconnected resync_paused flags set

VVR Primary.

oot@fan-v240 /var/vx/vras/log [35076]:vxprint-lP
Disk group:frank_vvrdg

Rlink:    rlk_jung-v240_rvg
info:    timeout=500 packet_size=8400rid=0.1596
         latency_high_mark=10000latency_low_mark=9950
         bandwidth_limit=none
state:    state=ACTIVE
         synchronous=offlatencyprot=offsrlprot=autodcm
assoc:    rvg=rvg
         remote_host=jung-v240IP_addr=10.251.56.165port=4145
         remote_dg=harry_vvrdg
         remote_dg_dgid=1165878502.389.jung-v240
         remote_rvg_version=30
         remote_rlink=rlk_fan-v240_rvg
         remote_rlink_rid=0.2389
         local_host=fan-v240IP_addr=10.251.56.164 port= 4145
protocol:UDP/IP
flags:    write enabled attached consistent disconnected asynchronous autosync resync_paused

and vrport output on the primary shows the heartbeat port is not correct:

root@fan-v240 /var/vx/vras/log[35077]:vrport
heartbeat= 4146
vradmind=8199
vxrsyncd=8989
data=Anonymous-Ports

Please note that the resync_paused flag set on the rlink is not a result of any VVR pause command run. In fact, it is a result caused by the rlink disconnect due to the incorrect VVR heartbeat port number on the primary.

To change the VVR heartbeat port, run vrport heartbeat .

root@fan-v240/var/vx/vras/log [35081]: vrport heartbeat 4145

root@fan-v240/var/vx/vras/log[35082] :vrport
heartbeat= 4145
vradmind=8199
vxrsyncd=8989
data=Anonymous-Ports
root@fan-v240/var/vx/vras/log [35083]:vxprint -lP
Disk group:frank_vvrdg

Rlink:    rlk_jung-v240_rvg
info:    timeout=500 packet_size=8400rid=0.1596
         latency_high_mark=10000latency_low_mark=9950
         bandwidth_limit=none
state:    state=ACTIVE
         synchronous=offlatencyprot=offsrlprot=autodcm
assoc:    rvg=rvg
         remote_host=jung-v240IP_addr=10.251.56.165port=4145
         remote_dg=harry_vvrdg
         remote_dg_dgid=1165878502.389.jung-v240
         remote_rvg_version=30
         remote_rlink=rlk_fan-v240_rvg
         remote_rlink_rid=0.2389
         local_host=fan-v240IP_addr=10.251.56.164 port= 4145
protocol:UDP/IP
flags:    write enabled attached consistent disconnected asynchronous autosync resync_paused

After fixing the VVR heartbeat port issue, the replication would not auto restart and the outstanding writes in the rlink remains unchanged.

root@fan-v240 /var/vx/vras/log[35084]:vxrlink -g frank_vvrdg -i 2 status rlk_jung-v240_rvg

Wed 20 May2009 08:04:13 PM EST
VxVM VVR vxrlink INFO V-5-1-4464 Rlink rlk_jung-v240_rvgis in AUTOSYNC. 102400 Kbytes remaining.
VxVM VVR vxrlink INFO V-5-1-4464Rlink rlk_jung-v240_rvg is in AUTOSYNC. 102400 Kbytes remaining.
VxVM VVRvxrlink INFO V-5-1-4464 Rlink rlk_jung-v240_rvg is in AUTOSYNC. 102400 Kbytesremaining.
VxVM VVR vxrlink INFO V-5-1-4464 Rlink rlk_jung-v240_rvg is inAUTOSYNC. 102400 Kbytes remaining.
VxVM VVR vxrlink INFO V-5-1-4464 Rlinkrlk_jung-v240_rvg is in AUTOSYNC. 102400 Kbytes remaining.
VxVM VVR vxrlinkINFO V-5-1-4464 Rlink rlk_jung-v240_rvg is in AUTOSYNC. 102400 Kbytesremaining.
VxVM VVR vxrlink INFO V-5-1-4464 Rlink rlk_jung-v240_rvg is inAUTOSYNC. 102400 Kbytes remaining.
... snip ...

To restart replication, run:
root@fan-v240 /var/vx/vras/log[35085]:/usr/sbin/vxstart_vvr stop
root@fan-v240 /var/vx/vras/log[35086]:/usr/sbin/vxstart_vvr start

Now VVR starts replicating.

root@fan-v240 /var/vx/vras/log [35087]:vxrlink -gfrank_vvrdg -i 2 status rlk_jung-v240_rvg

VxVM VVR vxrlink INFO V-5-1-4464 Rlink rlk_jung-v240_rvg is in AUTOSYNC.43520 Kbytes remaining.
VxVM VVR vxrlink INFO V-5-1-4464 Rlinkrlk_jung-v240_rvg is in AUTOSYNC. 23808 Kbytes remaining.
VxVM VVR vxrlinkINFO V-5-1-4464 Rlink rlk_jung-v240_rvg is in AUTOSYNC. 2048 Kbytesremaining.

... snip ...

and the rlink flags are changed to connected.

root@fan-v240 /var/vx/vras/log [35092]: vxprint -lP | grepflags
flags:    write enabled attached consistent connected asynchronous
 

Issue/Introduction

How to fix a rlink with disconnected resync_paused flags set?