VxVM vxdg ERROR V-5-1-552 Disk <dm-name> is used by one or more subdisks when removing a thin reclamation disk

book

Article ID: 100037902

calendar_today

Updated On:

Description

Description

This document attempts to explain the process required to remove a disk from a given diskgroup following thin reclamation activity. In this instance, even after reclaiming the space, the related disk still cannot be released from the corresponding diskgroup. 

The diskgroup name is named "dg01disk01", and the related disk is Veritas disk media (dm) name dg01disk01 linked to O/S native device name sdg


# vxdisk list | grep dg01disk01
sdg          auto:cdsdisk    dg01disk01   dg01         online thinrclm


When attempting to remove the disk from the diskgroup, the vxdg command fails with the following message:

# vxdg -g dg01 rmdisk dg01disk01
VxVM vxdg ERROR V-5-1-552 Disk dg01disk01 is used by one or more subdisks.
        Use -k to remove device assignment.


From the standard vxprint output, no Veritas Volume Manager (VxVM) objects are reported or associated with Veritas disk media (dm) name dg01disk01


# vxprint -g dg01 -qhr dg01disk01
dm dg01disk01   sdg          -        104758320 -       -        -       -


Even though the vxdisk reclaim process was completed without issues, the space has not been released. The vxdg free command implies the space is still allocated:


# vxdg -g dg01 free | grep dg01disk01
dg01disk01   sdg          sdg          62914560  41843760  -

 
The vxdefault command can be used to obtain the configurable Storage Foundation tunables.
 

# vxdefault list
KEYWORD                        CURRENT-VALUE   DEFAULT-VALUE
usefssmartmove                 thinonly        thinonly
fssmartmovethreshold           100             100
sharedminorstart               33000           33000
reclaim_on_delete_wait_period  1               1        
<<<<<< currently set to "1"
reclaim_on_delete_start_time   22:10           22:10
usesmartmovewithvvr            on              on

 
 
Note:   The  reclaim_on_delete_wait_period tunable refers to the number of days to wait before starting to reclaim space on a thin LUN, after a volume using that LUN is deleted. Specified as an integer from -1 to 366,  where  -1 specifies immediately and 366 specifies never.

 
# vxdisk list
DEVICE       TYPE            DISK         GROUP        STATUS
cciss/c0d0   auto:none       -            -            online invalid
sda          auto:cdsdisk    dgoraclex01_XX_01  dgoraclex01  online
sdb          auto:cdsdisk    dgoraclex01_XX_02  dgoraclex01  online
sdc          auto:cdsdisk    dg01_XX_01   dg01         online
sdd          auto:cdsdisk    dg01_XX_02   dg01         online
sde          auto:cdsdisk    dg01_XX_03   dg01         online
sdf          auto:cdsdisk    dg01_XX_04   dg01         online
sdg          auto:cdsdisk    dg01disk01   dg01         online thinrclm   
<<<<<<<<
sdh          auto:cdsdisk    -            -            online thinrclm
sdi          auto:cdsdisk    -            -            online thinrclm
sdj          auto:cdsdisk    -            -            online thinrclm
sdk          auto:cdsdisk    dgoraclex01disk01  dgoraclex01  online thinrclm
sdl          auto:cdsdisk    dgoraclex01disk02  dgoraclex01  online thinrclm



The -z option with the vxprint command reports the subdisks to be reclaimed as configured by vxdefault.

# vxprint -g dg01 -z
TY NAME         ASSOC        KSTATE   LENGTH   PLOFFS   STATE    TUTIL0  PUTIL0
dg dg01         dg01         -        -        -        -        -       -
 
dm dg01_XX_01   sdc          -        41767456 -        -        -       -
dm dg01_XX_02   sdd          -        41767456 -        -        -       -
dm dg01_XX_03   sde          -        41767456 -        -        -       -
dm dg01_XX_04   sdf          -        104758320 -       -        -       -
dm dg01disk01   sdg          -        104758320 -       -        -       -
 
sd dg01disk01-01 -           ENABLED  20971520 -        RECLAIM  -       -     <<<<<
sd dg01disk01-02 -           ENABLED  41943040 -        RECLAIM  -       -     <<<<<
 
v  vol01        fsgen        ENABLED  41943040 -        ACTIVE   -       -
pl XX_vol01-01  vol01        ENABLED  41943040 -        ACTIVE   -       -
sd dg01_XX_04-01 XX_vol01-01 ENABLED  41943040 0        -        -       -
 
v  vol02        fsgen        ENABLED  20971520 -        ACTIVE   -       -
pl XX_vol02-01  vol02        ENABLED  20971520 -        ACTIVE   -       -
sd dg01_XX_01-01 XX_vol02-01 ENABLED  20971520 0        -        -     
  -

 
How to change the reclaim_on_delete_wait_period


# POSIXLY_CORRECT=1 vxdefault set reclaim_on_delete_wait_period -1


# vxdefault list reclaim_on_delete_wait_period              
KEYWORD                        CURRENT-VALUE   DEFAULT-VALUE
reclaim_on_delete_wait_period  -1              1


# vxprint -g dg01 -htz
dg dg01         default      default  27000    fully.qualified.obscured.example.domain.com
 
dm dg01_XX_01   sdc          auto     65536    41767456 -
dm dg01_XX_02   sdd          auto     65536    41767456 -
dm dg01_XX_03   sde          auto     65536    41767456 -
dm dg01_XX_04   sdf          auto     65536    104758320 -
dm dg01disk01   sdg          auto     65536    104758320 -
 
sd dg01disk01-01 -           dg01disk01 0      20971520 -         sdg      RCLM   <<<<<<
sd dg01disk01-02 -           dg01disk01 20971520 41943040 -       sdg      RCLM   <<<<<<
 
v  vol01        -            ENABLED  ACTIVE   41943040 SELECT    -        fsgen
pl XX_vol01-01  vol01        ENABLED  ACTIVE   41943040 CONCAT    -        RW
sd dg01_XX_04-01 XX_vol01-01 dg01_XX_04 0      41943040 0         sdf      ENA
 
v  vol02        -            ENABLED  ACTIVE   20971520 SELECT    -        fsgen
pl XX_vol02-01  vol02        ENABLED  ACTIVE   20971520 CONCAT    -        RW
sd dg01_XX_01-01 XX_vol02-01 dg01_XX_01 0      20971520 0         sdc      ENA
 

How to mark the problematic subdisks as "reclaim_done"


# vxdisk reclaim sdg
Disk sdg : Skipped. No VxFS file system found.
 
 
# vxprint -g dg01 -mz dg01disk01-01 | grep recl
        reclaim_pnd=on
        reclaim_done=off


# vxedit -g dg01 set reclaim_done=on dg01disk01-01
 
# vxprint -g dg01 -mz dg01disk01-01 | grep recl
        reclaim_pnd=on
        reclaim_done=on


# vxprint -g dg01 -htz | less
dg dg01         default      default  27000    fully.qualified.obscured.example.domain.com
 
dm dg01_XX_01   sdc          auto     65536    41767456 -
dm dg01_XX_02   sdd          auto     65536    41767456 -
dm dg01_XX_03   sde          auto     65536    41767456 -
dm dg01_XX_04   sdf          auto     65536    104758320 -
dm dg01disk01   sdg          auto     65536    104758320 -
 
sd dg01disk01-01 -           dg01disk01 0      20971520 -         sdg      RCLM    <<<<<<
sd dg01disk01-02 -           dg01disk01 20971520 41943040 -       sdg      RCLM    <<<<<<
 
v  vol01        -            ENABLED  ACTIVE   41943040 SELECT    -        fsgen
pl XX_vol01-01  vol01        ENABLED  ACTIVE   41943040 CONCAT    -        RW
sd dg01_XX_04-01 XX_vol01-01 dg01_XX_04 0      41943040 0         sdf      ENA
 
v  vol02        -            ENABLED  ACTIVE   20971520 SELECT    -        fsgen
pl XX_vol02-01  vol02        ENABLED  ACTIVE   20971520 CONCAT    -        RW
sd dg01_XX_01-01 XX_vol02-01 dg01_XX_01 0      20971520 0         sdc      ENA


# vxedit -g dg01 -fr rm dg01disk01-01
 
# vxprint -g dg01 -htz
dg dg01         default      default  27000    fully.qualified.obscured.example.domain.com
 
dm dg01_XX_01   sdc          auto     65536    41767456 -
dm dg01_XX_02   sdd          auto     65536    41767456 -
dm dg01_XX_03   sde          auto     65536    41767456 -
dm dg01_XX_04   sdf          auto     65536    104758320 -
dm dg01disk01   sdg          auto     65536    104758320 -
 
sd dg01disk01-02 -           dg01disk01 20971520 41943040 -       sdg      RCLM        <<<<<
 
v  vol01        -            ENABLED  ACTIVE   41943040 SELECT    -        fsgen
pl XX_vol01-01  vol01        ENABLED  ACTIVE   41943040 CONCAT    -        RW
sd dg01_XX_04-01 XX_vol01-01 dg01_XX_04 0      41943040 0         sdf      ENA
 
v  vol02        -            ENABLED  ACTIVE   20971520 SELECT    -        fsgen
pl XX_vol02-01  vol02        ENABLED  ACTIVE   20971520 CONCAT    -        RW
sd dg01_XX_01-01 XX_vol02-01 dg01_XX_01 0      20971520 0         sdc      ENA

 
Repeat the above process for remaining subdisks, such as dg01disk01-02 in this instance.
 
 Following the above process, the vxdg free command reports the free space for the disks to be removed dg01disk01:

 
# vxdg free
GROUP        DISK         DEVICE       TAG          OFFSET    LENGTH    FLAGS
dgoraclex01  dgoraclex01_XX_01 sda          sda          10246400  1536      -
dgoraclex01  dgoraclex01disk01 sdk          sdk          10246400  1536      -
dg01         dg01_XX_01   sdc          sdc          20971520  20795936  -
dg01         dg01_XX_02   sdd          sdd          0         41767456  -
dg01         dg01_XX_03   sde          sde          0         41767456  -
dg01         dg01_XX_04   sdf          sdf          41943040  62815280  -
dg01         dg01disk01   sdg          sdg          0         104758320
-  <<<<<<<<

 

The disk can now be removed from the corresponding diskgroup.

# vxdg -g dg01 rmdisk dg01disk01
 
# vxdisk list
DEVICE       TYPE            DISK         GROUP        STATUS
cciss/c0d0   auto:none       -            -            online invalid
sda          auto:cdsdisk    dgoraclex01_XX_01  dgoraclex01  online
sdb          auto:cdsdisk    dgoraclex01_XX_02  dgoraclex01  online
sdc          auto:cdsdisk    dg01_XX_01   dg01         online
sdd          auto:cdsdisk    dg01_XX_02   dg01         online
sde          auto:cdsdisk    dg01_XX_03   dg01         online
sdf          auto:cdsdisk    dg01_XX_04   dg01         online

sdg          auto:cdsdisk    -            -            online thinrclm   <<<<<<<<
sdh          auto:cdsdisk    -            -            online thinrclm
sdi          auto:cdsdisk    -            -            online thinrclm
sdj          auto:cdsdisk    -            -            online thinrclm
sdk          auto:cdsdisk    dgoraclex01disk01  dgoraclex01  online thinrclm
sdl          auto:cdsdisk    dgoraclex01disk02  dgoraclex01  online thinrclm

 

 Process complete.

 

Issue/Introduction

VxVM vxdg ERROR V-5-1-552 Disk is used by one or more subdisks when removing a thin reclamation disk