Failure to mount a filesystem as a result of shrinking a volume.
Article ID: 100020711
Updated On:
Description
Error Message
# mount-F vxfs /dev/vx/dsk/testdg/u100 /c1mnt
UX:vxfsmount: ERROR: V-3-24706: /dev/vx/dsk/testdg/u100 no such device or filesystem on it missing one or more devices .....
# fsck -Fvxfs /dev/vx/rdsk/testdg/u100
UX:vxfsfsck: ERROR: V-3-26248: could not read from block offset devid/blknum0/xxxxxx .
A device containing meta data may be missing in vset or device too big to be read on a 32 bit system.
UX:vxfsfsck: ERROR: V-3-20694: cannot initialize aggregate file system check failure,aborting .
Cause
The issue occurs when a volume has been forcefully reduced in size, leaving the underlying filesystem with a size larger than the volume.
1. Check the current status and size of filesystem and volume
# fstyp-v /dev/vx/dsk/testdg/u100
vxfs
magica501fcf5 version 6 ctime Sat Jan 28 01:27:37 2006
logstart0 logend 0
bsize1024 size 47185920 dsize 0 ninode 47185920 nau0...........
...
where
size offilesystem = (bsize*size)/512 sectors
=1024*47185920/512
=94371840 sectors (=45Gb)
# vxprint-g testdg -htv u100
v u100 - ENABLED ACTIVE 75497472 SELECT - fsgen
plu100-01 u100 ENABLED ACTIVE 75497472 CONCAT - RW
sdtestdg01-01 u100-01 testdg01 0 16777216 0 c1t2d0 ENA
sdtestdg01-08 u100-01 testdg01 167772160 16777216 16777216 c1t2d0ENA
sdtestdg01-12 u100-01 testdg01 146800640 20971520 33554432 c1t2d0ENA
sdtestdg01-13 u100-01 testdg01 234881024 20971520 54525952 c1t2d0ENA
where: filesystem= 94371840 sectors (=45Gb)
volumesize = 75497472 sectors, (=36Gb)
Differenceis volume size = 18874368 sectors (=9Gb)
So to resolve the mount issue, we need to grow the volume by 9Gb and back to the original size =45Gb
Optional:If the original volume is mirrored, it is recommended to break the mirror plex and keep it as backup copy
# vxplex -g dis
2. Using"vxdg free" first is to check what disks are available to be used
# vxdg -gtestdg free
GROUPDISK DEVICE TAG OFFSET LENGTH FLAGS
testdg testdg03 c1t4d0s2 c1t4d0285212672 18874368 -
testdg testdg04 c1t5d0s2 c1t5d0 285826704 869616 -
3. Check available data to determine the configuration of the volume prior to running the vxassist to shrink the volume.
(i) Copy of an old vxprint -g testdg -ht
(ii) Use backup directory: /etc/vx/cbg/bk/testdg.1224220109.123.xxx to recreate vxprint file by selecting the *cfrec. file created before the shrink command .
#cd /etc/vx/cbg/bk/testdg.1224220109.123.xxx
# cat1224220109.123.xxx.cfgrec. | vxprint -D - -ht
(iii) Use backup log : /var/adm/vx/cmdlog.* to check what commands were used
# cd/var/adm/vx
# cat cmdlog | grep shrinkto
# 4474,23451, Mon Apr 6 18:30:31 2009
/usr/sbin/vxassist-ttaskid_112 -f -g testdg shrinkto u100 75497472 alloc=testdg03
The log indicates testdg03 was used to shrink the volume.
Cause
The issue occurs when a volume has been forcefully reduced in size, leaving the underlying filesystem with a size larger than the volume.
Resolution
As per the diagnostic steps above, we need to grow the volume by 9Gb to the original size 45Gb
using # vxdg free output to determine the free space available. There are 3 possible options:
1) Selection is limited to only one sub-disk
# vxdg -g testdg free
GROUPDISK DEVICE TAG OFFSET LENGTH FLAGS
testdg testdg03 c1t4d0s2 c1t4d0 285212672 18874368 -
testdg testdg04 c1t5d0s2 c1t5d0 285826704 869616 -
In this case, the value under the "length" field = 18874368sectors=9Gb indicates there is only one sub-disk which matches the missing sectors and hence, it is the only choice to be used as the sub-disk to re-grow the volume.
#vxassist -g dg growby 18874368 u100 testdg03
# fsck -F vxfs /dev/vx/rdsk/dg/u100
#mount -F vxfs /dev/vx/dsk/testdg/u100 /u100
Note1: At this stage, if you manage to relocate the correct disk, there will be no missing data.
2) No likely candidate for the missing sub-disk and you are required to reallocate the missing space from an alternative source.
# vxdg -g testdg free
GROUPDISK DEVICE TAG OFFSET LENGTH FLAGS
testdg testdg03 c1t4d0s2 c1t4d0 285212672 144368 -
testdg testdg04 c1t5d0s2 c1t5d0 285826704 3416 -
There is not enough available disk to create the missing sub-disk, the sub-disk may have been reallocated to other volume (u1yy). To recover shrink an existing volume in the disk group (if possible, choose a volume using the same device) to free up the required space.
#vxresize -g testdg shrinkby u1yy 18874368 testdg03 ( free space fromtestdg03 disk)
#vxassist -g dg growto 18874368 u100 testdg03
# fsck -Fvxfs -o full /dev/vx/rdsk/dg/u100
#mount -F vxfs /dev/vx/dsk/testdg/u100 /u100
Alternatively,add a new disk
#vxdisksetup - i. cxtydz
# vxdg -gtestdg adddisk testdgxx=cxtydz
#vxassist -g dg growto 94371840 u100 testdgxx
# fsck -Fvxfs -o full /dev/vx/rdsk/dg/u100
#mount -F vxfs /dev/vx/dsk/testdg/u100 /u100
Note 2;Since you have reallocated or added a new disk to recreate the missing sub-disk,running fsck - o full will determine if you have lost any data as a result ofthis operation. Since it may be possible there was no "data" on the sub-disk in the first place.
3) Mutilple sub-disks are available to be concatenated to the required length of the missing subdisk
#vxdg -gtestdg free
GROUP DISK DEVICE TAG OFFSET LENGTH FLAGS
testdgtestdg01 c1t2d0s2 c1t2d0 50331648 16777216 -
testdgtestdg02 c1t3d0s2 c1t3d0 236978176 6291456 -
testdgtestdg03 c1t4d0s2 c1t4d0 50331648 16777216 -
testdgtestdg03 c1t4d0s2 c1t4d0 236978176 6291456 -
testdgtestdg03 c1t4d0s2 c1t4d0 285212672 2097152 -
testdgtestdg04 c1t5d0s2 c1t5d0 0 16777216 -
testdgtestdg04 c1t5d0s2 c1t5d0 163577856 12582912 -
testdgtestdg04 c1t5d0s2 c1t5d0 285826704 869616 -
Not easy to guess which of the free sub-disks can be used to make the total =18874368 sectors, as it could be any combination of 2 or more sub-disks. Using an old vxprint or equivalent output, we can use vxmake command to recreate the volume bottom-up.
From oldvxprint we have :
v u100 -ENABLED ACTIVE 94371840 SELECT - fsgen
plu100-01 u100 ENABLED ACTIVE 94371840 CONCAT - RW
sdtestdg01-01 u100-01 testdg01 0 16777216 0 c1t2d0 ENA
sdtestdg01-08 u100-01 testdg01 167772160 16777216 16777216 c1t2d0ENA
sdtestdg01-12 u100-01 testdg01 146800640 20971520 33554432 c1t2d0ENA
sdtestdg01-13 u100-01 testdg01 234881024 20971520 54525952 c1t2d0ENA
sdtestdg03-06 u100-01 testdg03 50331648 16777216 75497472 c1t4d0ENA
sdtestdg03-09 u100-01 testdg03 285212672 2097152 92274688 c1t4d0ENA
So we need to recreate the volume as follows :
(i)recreate the 2 missing subdisks
#vxmake -g testdg sd testdg03-06 disk=testdg03 offset=50331648length=16777216
#vxmake -g testdg sd testdg03-09 disk=testdg03 offset=285212672length=2097152
(ii) add subdisks to volume
# vxsd -gtestdg assoc u100-01 testdg03-06
# vxsd -gtestdg assoc u100-01 testdg03-09
(iii) check status
# vxprint -g testdg -ht
v u100 -ENABLED ACTIVE 75497472 SELECT - fsgen
plu100-01 u100 ENABLED ACTIVE 94371840 CONCAT - RW
sdtestdg01-01 u100-01 testdg01 0 16777216 0 c1t2d0 ENA
sdtestdg01-08 u100-01 testdg01 167772160 16777216 16777216 c1t2d0ENA
sdtestdg01-12 u100-01 testdg01 146800640 20971520 33554432 c1t2d0ENA
sdtestdg01-13 u100-01 testdg01 234881024 20971520 54525952 c1t2d0ENA
sdtestdg03-06 u100-01 testdg03 50331648 16777216 75497472 c1t4d0ENA
sdtestdg03-09 u100-01 testdg03 285212672 2097152 92274688 c1t4d0ENA
(iv) grow volume to match and then fsck and mount
# vxresize -g testdg growtou100 94371840
# fsck -F vxfs/dev/vx/rdsk/testdg/u100
#mount -F vxfs /dev/vx/dsk/testdg/u100 /u100
Note3: Again, after replacing the missing sub-disks based on an old configuration, running fsck - o full will determine if you have lost any data as a result of this operation.
Was this article helpful?
Yes
No
Powered by
