The error results because there is at least one active volume using the disk being removed. e.g.
dg testdg default default 28000 1641672206.15.localhost.localdomain
dm vrts_appliances0_4 vrts_appliances0_4 auto 65536 104791760 -
dm vrts_appliances0_5 vrts_appliances0_5 auto 65536 104791760 -
v testvol2 - ENABLED ACTIVE 204800 SELECT - fsgen
pl testvol2-01 testvol2 ENABLED ACTIVE 204800 CONCAT - RW
sd vrts_appliances0_5-01 testvol2-01 vrts_appliances0_5 0 204800 0 vrts_appliances0_5 ENA
pl testvol2-02 testvol2 ENABLED ACTIVE 204800 CONCAT - RW
sd vrts_appliances0_4-02 testvol2-02 vrts_appliances0_4 204800 204800 0 vrts_appliances0_4 ENA
vrts_appliances0_4 is occupied by a data plex by the testvol2 volume
With the given scenario, a decision was taken to use the -k option. This put the LUN in a removed state:
e.g.
# vxdg -g testdg -k rmdisk vrts_appliances0_4
dm vrts_appliances0_4 - - - - REMOVED
dm vrts_appliances0_5 vrts_appliances0_5 auto 65536 104791760 -
v testvol2 - ENABLED ACTIVE 204800 SELECT - fsgen
pl testvol2-01 testvol2 ENABLED ACTIVE 204800 CONCAT - RW
sd vrts_appliances0_5-01 testvol2-01 vrts_appliances0_5 0 204800 0 vrts_appliances0_5 ENA
pl testvol2-02 testvol2 DISABLED REMOVED 204800 CONCAT - RW
sd vrts_appliances0_4-02 testvol2-02 vrts_appliances0_4 204800 204800 0 - RMOV
The vrts_appliances0_4 LUN was now in REMOVED state. The volume was still ENABLED:ACTIVE as one plex was still enabled. It is not recommended that the -k option be used unless the data happens to have no importance.
Return the removed LUN back to its former state (and remove any unwanted data plexes first):
# vxdg -g testdg -k adddisk vrts_appliances0_4
dm vrts_appliances0_4 vrts_appliances0_4 auto 65536 104791760 -
dm vrts_appliances0_5 vrts_appliances0_5 auto 65536 104791760 -
v testvol2 - ENABLED ACTIVE 204800 SELECT - fsgen
pl testvol2-01 testvol2 ENABLED ACTIVE 204800 CONCAT - RW
sd vrts_appliances0_5-01 testvol2-01 vrts_appliances0_5 0 204800 0 vrts_appliances0_5 ENA
pl testvol2-02 testvol2 DISABLED RECOVER 204800 CONCAT - RW
sd vrts_appliances0_4-02 testvol2-02 vrts_appliances0_4 204800 204800 0 vrts_appliances0_4 ENA
The disk had been added back and plexvol1-01 was now in DISABLED RECOVER state. Running vxrecover resyncs the plex:
# vxrecover -g testdg testvol2
dm vrts_appliances0_4 vrts_appliances0_4 auto 65536 104791760 -
dm vrts_appliances0_5 vrts_appliances0_5 auto 65536 104791760 -
v testvol2 - ENABLED ACTIVE 204800 SELECT - fsgen
pl testvol2-01 testvol2 ENABLED ACTIVE 204800 CONCAT - RW
sd vrts_appliances0_5-01 testvol2-01 vrts_appliances0_5 0 204800 0 vrts_appliances0_5 ENA
pl testvol2-02 testvol2 ENABLED ACTIVE 204800 CONCAT - RW
sd vrts_appliances0_4-02 testvol2-02 vrts_appliances0_4 204800 204800 0 vrts_appliances0_4 ENA
The LUN can now be correctly removed by first disassociating and removing any plexes on the LUN.
# vxplex -g testdg -o rm dis testvol2-02
dm vrts_appliances0_4 vrts_appliances0_4 auto 65536 104791760 -
dm vrts_appliances0_5 vrts_appliances0_5 auto 65536 104791760 -
v testvol2 - ENABLED ACTIVE 204800 SELECT - fsgen
pl testvol2-01 testvol2 ENABLED ACTIVE 204800 CONCAT - RW
sd vrts_appliances0_5-01 testvol2-01 vrts_appliances0_5 0 204800 0 vrts_appliances0_5 ENA
# vxdg -g testdg rmdisk vrts_appliances0_4
dm vrts_appliances0_5 vrts_appliances0_5 auto 65536 104791760 -
v testvol2 - ENABLED ACTIVE 204800 SELECT - fsgen
pl testvol2-01 testvol2 ENABLED ACTIVE 204800 CONCAT - RW
sd vrts_appliances0_5-01 testvol2-01 vrts_appliances0_5 0 204800 0 vrts_appliances0_5 ENA