How to successfully recover and enable a 'vxmake built' VxVM Data Volume that has an attached DCO/DCL.

book

Article ID: 100021114

calendar_today

Updated On:

Description

Problem

How to successfully recover and enable a 'vxmake built' VxVM Data Volume that has an attached DCO/DCL.

 

Solution

Typically, a VERITAS Volume Manager (VxVM) Volume would look like this in a vxprint -ht output:

This example shows a volume named'vol1', that is part of the Diskgroup 'tt':

#vxprint -htqg tt vol1
v  vol1        -            DISABLED EMPTY    204800  SELECT    -        fsgen
pl vol1-01      vol1         DISABLED EMPTY    204800  CONCAT    -        RW
sd EMC_CLARiiON0_4-01 vol1-01 EMC_CLARiiON0_4 0204800  0         EMC_CLARiiON0_4ENA
dc vol1_dco    vol1        vol1_dcl
v  vol1_dcl    -            DISABLED EMPTY    544      SELECT    -        gen
pl vol1_dcl-01  vol1_dcl     DISABLED EMPTY    544      CONCAT    -        RW
sd EMC_CLARiiON0_9-01 vol1_dcl-01 EMC_CLARiiON0_9 0 5440         EMC_CLARiiON0_9 ENA

We try to start the freshly recreated (EMPTY) Volume:

# vxvol -g start vol1
VxVM vxvol ERROR V-5-1-10128  Unexpected kernel error in configuration update

We try to unprepare it, in an attempt to remove the DCO Log:

# vxsnap -g unprepare vol1
VxVM vxassist ERRORV-5-1-6169  Volume vol1 has drl attach to it, use -f option to remove drl

# vxsnap -g -f unprepare vol1
VxVM vxassist ERROR V-5-1-2149  Volume vol1_dcl has non initialized plex

# vxplex -g -o rm dis vol1_dcl-01
VxVM vxplex ERRORV-5-1-10128  Unexpected kernel error in configuration update

At this point ...it appear that we can't do anything... except we can do something!

The process will vary depending on whether or not you have the ability to deport all of the diskgroups (including rootdg).
If the machine has an encapsulated boot disk, the machine will need to be rebooted to clear up the "V-5-1-10128' error condition.

PROCESS TO ENABLE VOLUME:

First we have to recover the cause for the errors "V-5-1-10128", for which we have two methods:

1. If possible deport all diskgroups and restart vxconfigd with reset option:
- deport all diskgroups (can't be done with encapsulated root)
- Check the diskgroups are all deported
#  [ `vxdg -q list | wc -l` -eq 0 ]&& echo "Diskgroups all deported"
Diskgroups all deported
- Run vxconfigd -k -r reset
# vxconfigd -k -r reset
-Run the following commands
# vxplex -g -o rm dis vol1_dcl-01
# vxsnap -g
-f unprepare vol1
 
2. Reboot the box so that the on disk and in-kernel vxconfig copies are in sync:
- After reboot, run these commands to allow removal of DCL/DCO objects.
#vxplex -g -o rm dis vol1_dcl-01
#vxsnap -g
-f unprepare vol1


After running these commands, the volume being recovered should look like this:

v  vol1        -            DISABLED EMPTY    204800  SELECT    -        fsgen
plvol1-01      vol1         DISABLED EMPTY    204800  CONCAT    -        RW
sd EMC_CLARiiON0_4-01 vol1-01 EMC_CLARiiON0_4 0204800  0         EMC_CLARiiON0_4ENA

Note that we've removed the DCO Volume, and our next step is to recreate the DCO Log and attach it to the Data Volume.

Start the volume and add a FMR3 (DCO/DCL logver=20) log with 'vxsnap prepare'.
# vxvol -g start vol1
# vxsnap -g
prepare vol1


The volume is recovered and now ready for use:

v  vol1        -            ENABLED  ACTIVE  204800  SELECT    -        fsgen
pl vol1-01      vol1        ENABLED  ACTIVE   204800  CONCAT    -        RW
sd EMC_CLARiiON0_4-01 vol1-01 EMC_CLARiiON0_4 0204800  0         EMC_CLARiiON0_4ENA
dc vol1_dco    vol1        vol1_dcl
v  vol1_dcl    -            ENABLED  ACTIVE  544      SELECT    -        gen
pl vol1_dcl-01  vol1_dcl     ENABLED  ACTIVE  544      CONCAT    -        RW
sd EMC_CLARiiON0_9-01 vol1_dcl-01 EMC_CLARiiON0_9 0 5440         EMC_CLARiiON0_9ENA


SECOND METHOD TO RECOVER, THAT RETAINS THE DCO LOG VOLUME AND SUBDISKS:

First, let's assume that a Volume that contains a DCO LOG Volume was recovered using vxmake:

Example method for saving a vxmake 'description file' :
# vxprint -g -hmpvsc vol1 > /tmp/vol1b.out

Example of recovering vol1 using 'vxmake -d' description file:
# vxmake -g -d /tmp/vol1b.out

Check the state of Volume 'vol1':

# vxprint -htqg
dg tt          default      default  46000    1243550172.40.sydfire04

dm EMC_CLARiiON0_4 EMC_CLARiiON0_4 auto65536  2031232  -
dm EMC_CLARiiON0_9 EMC_CLARiiON0_9 auto65536  2031232  -
dm EMC_CLARiiON0_10 EMC_CLARiiON0_10 auto 65536 2031232-

v  vol1        -            DISABLEDEMPTY    204800  SELECT    -        fsgen
pl vol1-01      vol1         DISABLEDEMPTY    204800  CONCAT    -        RW
sd EMC_CLARiiON0_4-01 vol1-01 EMC_CLARiiON0_4 0204800  0         EMC_CLARiiON0_4ENA
dc vol1_dco    vol1        vol1_dcl
v  vol1_dcl    -            DISABLEDEMPTY    544      SELECT    -        gen
pl vol1_dcl-01  vol1_dcl     DISABLEDEMPTY    544      CONCAT    -        RW
sd EMC_CLARiiON0_9-01 vol1_dcl-01 EMC_CLARiiON0_9 0 5440         EMC_CLARiiON0_9ENA

# vxvol -g start vol1
VxVM vxvol ERRORV-5-1-10128  Unexpected kernel error in configuration update

Once this error is seen, the machine either needs to be rebooted, or the vxconfigd daemon restart with "-r reset" to recover after this V-5-1-10128 error.

SO HOW TO OVERCOME THIS?

When starting a volume, after creating it using vxmake, you need to recover the log volume before recovering the regular volume.Trying to recover the Data Volume first will create the requirement for us to reboot or restart vxconfigd with -r reset.

# vxrecover -g tt -sbE vol1_dcl
# vxprint -htqg tt
dg tt          default      default  46000    1243550172.40.sydfire04

dm EMC_CLARiiON0_4 EMC_CLARiiON0_4 auto 65536  2031232  -
dm EMC_CLARiiON0_9 EMC_CLARiiON0_9 auto 65536  2031232  -
dm EMC_CLARiiON0_10 EMC_CLARiiON0_10 auto 65536 2031232-

v  vol1        -            DISABLED EMPTY    204800  SELECT    -        fsgen
pl vol1-01      vol1         DISABLE DEMPTY    204800  CONCAT    -        RW
sd EMC_CLARiiON0_4-01vol1-01 EMC_CLARiiON0_4 0 204800  0        EMC_CLARiiON0_4 ENA
dc vol1_dco    vol1         vol1_dcl
v  vol1_dcl    -            ENABLED  ACTIVE  544      SELECT    -        gen
p lvol1_dcl-01  vol1_dcl     ENABLED  ACTIVE  544      CONCAT    -        RW
sd EMC_CLARiiON0_9-01 vol1_dcl-01 EMC_CLARiiON0_9 0 5440         EMC_CLARiiON0_9 ENA


Next, recover the data volume:

# vxrecover -g tt -sbE vol1
# vxprint -htqg tt

dgtt          default      default  46000    1243550172.40.sydfire04

dm EMC_CLARiiON0_4 EMC_CLARiiON0_4 auto 65536  2031232  -
dm EMC_CLARiiON0_9 EMC_CLARiiON0_9 auto 65536  2031232  -
dm EMC_CLARiiON0_10 EMC_CLARiiON0_10 auto 65536 2031232-

v  vol1        -            ENABLED  ACTIVE  204800   SELECT    -        fsgen
pl vol1-01      vol1        ENABLED  ACTIVE   204800  CONCAT    -        RW
sd EMC_CLARiiON0_4-01vol1-01 EMC_CLARiiON0_4 0 204800  0        EMC_CLARiiON0_4 ENA
dc vol1_dco    vol1         vol1_dcl
v  vol1_dcl    -            ENABLED  ACTIVE  544      SELECT    -        gen
pl vol1_dcl-01  vol1_dcl     ENABLED  ACTIVE  544      CONCAT    -        RW
sd EMC_CLARiiON0_9-01 vol1_dcl-01 EMC_CLARiiON0_9 0 5440         EMC_CLARiiON0_9 ENA



This process should be repeated for all of the Volumes that were created using vxmake, until all of the volumes and DCO Volumes are in an ENABLED ACTIVE state.


After this point.. there should be no more errors like :
VxVM vxvol ERRORV-5-1-10128  Unexpected kernel error in configuration update


Typically, after restarting a volume created using vxmake, we run 'fstyp' or 'fsck -n' on the VxVM device to ensure that the expected filesystem is still OK. For raw volume situations, the application will need to be started to ensure the data is still intact.
 

 

Issue/Introduction

How to successfully recover and enable a 'vxmake built' VxVM Data Volume that has an attached DCO/DCL.