Data in a Veritas Volume Manager (VxVM) volume is shifted by 1 to 15 sectors because VxVM uses a dfferent effective Public Region offset

book

Article ID: 100007846

calendar_today

Updated On:

Description

Error Message

The problem can cause a Veritas File System (VxFS) to fail to mount:

# mount -t vxfs /dev/vx/dsk/test/vol01 /vol01

UX:vxfs mount.vxfs: ERROR: V-3-20012: not a valid vxfs file system

UX:vxfs mount.vxfs: ERROR: V-3-24996: Unable to get disk layout version

Cause

The problem is caused by the Etrack incident listed in the Supplemental Material section of this article.

Please refer to the related article TECH173882 for details on the Etrack incident.

When a diskgroup is created, the default subdisk alignment is set to 16. The diskgroup alignment can be checked with one of the following two commands.

# vxprint -g -Gl

For example:

# vxprint -g cdg -lG

Group:    cdg

info:     dgid=1325662784.54.rhel6vm1

version:  170

alignment: 8192 (bytes)           <<< default alignment is 16 sectors or 8KB

activation: read-write

detach-policy: global

dg-fail-policy: dgdisable

ioship: off

copies:   nconfig=default nlog=default

devices:  max=0 curr=2

minors:   >= 0

Or,

# vxprint -g -m

For example,

# vxprint -g cdg -m cdg | grep alignment

        alignment=16

The purpose of the 8KB (16 sectors) alignment is to facilitate the future conversion to Cross-Platform Disk Sharing (CDS) disk format, so that the diskgroup data can be migrated to different Operation System (OS) platforms.

When a disk is initialized as Sliced Disk format on Linux platform using DOS label, the public region partition may not fall exactly into 8KB boundary. For example, the following is a DOS format VxVM disk which has a public region partition not aligned to 8KB.

# vxdisk list virtio0_10

Device:    virtio0_10

devicetag: virtio0_10

type:      auto

hostid:    rhel6vm1

disk:      name=cdg01 id=1325662744.50.rhel6vm1

group:     name=cdg id=1325662784.54.rhel6vm1

info:      format=sliced,privoffset=1,pubslice=6,privslice=5

flags:     online ready private autoconfig autoimport imported

pubpaths:  block=/dev/vx/dmp/virtio0_10s6 char=/dev/vx/rdmp/virtio0_10s6

privpaths: block=/dev/vx/dmp/virtio0_10s5 char=/dev/vx/rdmp/virtio0_10s5

guid:      {885d4ed6-367c-11e1-b8b3-dca8360cd3a4}

udid:      QEMU%5FVIRTIO%5FVirtIO%5F%2Fdev%2Fvdk

site:      -

version:   2.1

iosize:    min=512 (bytes) max=1024 (blocks)

public:    slice=6 offset=0 len=4063042 disk_offset=131262            <<<< disk_offset is the disk offset of the public region partition

private:   slice=5 offset=1 len=131072 disk_offset=126

update:    time=1325662787 seqno=0.7

ssb:       actual_seqno=0.0

headers:   0 248

configs:   count=1 len=107282

logs:      count=1 len=4096

Defined regions:

 config   priv 000017-000247[000231]: copy=01 offset=000000 enabled

 config   priv 000249-107299[107051]: copy=01 offset=000231 enabled

 log      priv 107300-111395[004096]: copy=01 offset=000000 enabled

Multipathing information:

numpaths:   1

vdk             state=enabled

The above is called a Disk Access (DA) record.

# fdisk -lu /dev/vdk

Disk /dev/vdk: 2147 MB, 2147483648 bytes

128 heads, 32 sectors/track, 1024 cylinders, total 4194304 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0x5a40de4a

   Device Boot      Start         End      Blocks   Id  System

/dev/vdk4              63     4194303     2097120+   5  Extended

/dev/vdk5             126      131198       65536+  7f  Unknown

/dev/vdk6          131262     4194303     2031521   7e  Unknown     <<< public region partition starts at 131262



131262 sectors is not aligned to 8KB.

    131262 / 16 = 8203.875

In order to compensate for this misalignment, VxVM has two additional attributes for each Disk Media (DM) record in the diskgroup.

# vxprint -g cdg -m cdg01 | grep offset

        pub_offset=2              <<< alignment-compensating offset

        last_disk_offset=131262    <<< last effective public region offset

These two DM attributes are only used if the diskgroup alignment is 16 sectors.

In the above example, the effective public region of the disk can be calculated as Disk Access (DA) record "disk_offset" (131262) plus the Disk Media (DM) record "pub_offset" which should be the same as the DM last_disk_offset, if the disk partition table didn't change.

131262 + 2 = 131264

The sum of "DA disk_offset" and "DM pub_offset" is now divisible by 16.

131264 / 16 = 8204

This effective Public Region offset will be used to acess the data in the Public Region.

Due to the Etrack incident 2561012, VxVM can miscalculate the "DM pub_offset" and create subdisks not aligned to the 8KB boundary. The subdisk (and hence the volume) will continue to work until the system is rebooted. After the reboot, VxVM calculates the correct "DM pub_offset", but now the data in the volume will be shifted by 1 to 15 sectors.

For example, before the reboot, the wrong "DM pub_offset" is 0.

# vxprint -g cdg -m cdg01 | grep offset

        pub_offset=0               <<< incorrect pub_offset used initially             

        last_disk_offset=131262

After the reboot, the correct "DM pub_offset" is 2.

# vxprint -g cdg -m cdg01 | grep offset

        pub_offset=2             <<<< correct pub_offset after reboot

        last_disk_offset=131262

Now the volume data is shifted by 2 sectors. This can cause VxFS filesystems to fail to mount.

# mount -t vxfs /dev/vx/dsk/test/vol01 /vol01

UX:vxfs mount.vxfs: ERROR: V-3-20012: not a valid vxfs file system

UX:vxfs mount.vxfs: ERROR: V-3-24996: Unable to get disk layout version

=================================================
Side Note:
In the DA record, there is another offset value.   For example,

public:    slice=6 offset=32 len=4063010 disk_offset=131262          <<<< offset=32
private:   slice=5 offset=1 len=131072 disk_offset=126

This offset value is used as a minimum "DM pub_offset' when the diskgroup alignment is set to 16. The calculated "DM pub_offset' will always be bigger than this value. Normally this "DA public offset" is 0.    The "DA public offset" can be set to non-zero if it is deliberately specified in the vxdisk init command. For example,

vxdisk init format=sliced pub_offset=32

This offset value is not used in the calculation of the effective Public Region offset.

Resolution

In order to fix the problem without recreating the diskgroup, we can set the diskgroup alignment to 1. (Diskgroup alignment can be set to 1 for non-CDS diskgroup group.)

Use the following command to change the diskgroup alignment:

# vxdg -g set alignment=1

For example,

# vxdg -g cdg set alignment=1

After the diskgroup alignment is set to 1, the "DM pub_offset" will be set to 0.

rhel6vm1# vxprint -g cdg -m cdg01 | grep offset

        pub_offset=0

        last_disk_offset=131262

VxVM will adjust the disk offsets of the subdisks accordingly because VxvM does not know that the initial "DM pub_offset" was incorrect.

For example, after diskgroup alignment is set to 1, the subdisks in the above sample diskgroup will have an adjusted disk offset.

# vxprint -ht -g cdg DG NAME NCONFIG NLOG MINORS GROUP-ID ... DM NAME DEVICE TYPE PRIVLEN PUBLEN STATE ... V NAME RVG/VSET/CO KSTATE STATE LENGTH READPOL PREFPLEX UTYPE PL NAME VOLUME KSTATE STATE LENGTH LAYOUT NCOL/WID MODE SD NAME PLEX DISK DISKOFFS LENGTH [COL/]OFF DEVICE MODE <<< DISKOFFS ... dg cdg default default 0 1325653937.43.rhel6vm1 dm

cdg01 virtio0_10 auto 131072 4063010 -

v volvxfs - ENABLED ACTIVE 204800 SELECT - fsgen pl volvxfs-01 volvxfs ENABLED ACTIVE 204800 CONCAT - RW sd cdg01-01 volvxfs-01 cdg01 2 204800 0 virtio0_10 ENA <<< DISKOFFS is changed to 2 now With the above subdisk disk offset adjustment, the data will still be shifted. Now we need to change the public region partition offset and size to have the subdisk fall back to the correct data location.

We need to deport the diskgroup and temporarily remove the Disk Access (DA) records before changing the partition table:

# vxdg deport cdg

# vxdisk rm virtio0_10

Now use fdisk to change the public region partition by reducing the starting sectors by the number of shifted sectors but keep the end sector the same. That is, the public region partition will be grown a bit bigger from the beginning of the disk.

# fdisk -lu /dev/vdj

Disk /dev/vdj: 2147 MB, 2147483648 bytes

128 heads, 32 sectors/track, 1024 cylinders, total 4194304 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0xc67d860e

   Device Boot      Start         End      Blocks   Id  System

/dev/vdj4              63     4194303     2097120+   5  Extended

/dev/vdj5             126      131198       65536+  7f  Unknown

/dev/vdj6          131262     4194303     2031521   7e  Unknown    <<< change Start to 131260 (shifted by 2 sectors) but keep End at 4194303

# fdisk -u /dev/vdj

Command (m for help): d

Partition number (1-6): 6

Command (m for help): n

Command action

   l   logical (5 or over)

   p   primary partition (1-4)

l

First sector (95-4194303, default 95): 131260

Last sector, +sectors or +size{K,M,G} (131260-4194303, default 4194303): 4194303

Command (m for help): t

Partition number (1-6): 6

Hex code (type L to list codes): 7e

Changed system type of partition 6 to 7e (Unknown)

Command (m for help): p

Disk /dev/vdj: 2147 MB, 2147483648 bytes

128 heads, 32 sectors/track, 1024 cylinders, total 4194304 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0xc67d860e

  

Device Boot      Start         End      Blocks   Id  System

/dev/vdj4              63     4194303     2097120+   5  Extended

/dev/vdj5             126      131198       65536+  7f  Unknown

/dev/vdj6          131260     4194303     2031522   7e  Unknown

Command (m for help): w

The partition table has been altered!

Calling ioctl() to re-read partition table.

Syncing disks.

Now run "vxdctl enable" to enable the disk again

# vxdctl enable

# vxdisk list virtio0_9

Device:    virtio0_9

devicetag: virtio0_9

type:      auto

hostid:

disk:      name= id=1325653412.32.rhel6vm1

group:     name=bdg id=1325653449.34.rhel6vm1

info:      format=sliced,privoffset=1,pubslice=6,privslice=5

flags:     online ready private autoconfig

pubpaths:  block=/dev/vx/dmp/virtio0_9s6 char=/dev/vx/rdmp/virtio0_9s6

privpaths: block=/dev/vx/dmp/virtio0_9s5 char=/dev/vx/rdmp/virtio0_9s5

guid:      {84fd3a08-367c-11e1-94d0-519ba41e21bb}

udid:      QEMU%5FVIRTIO%5FVirtIO%5F%2Fdev%2Fvdj

site:      -

version:   2.1

iosize:    min=512 (bytes) max=1024 (blocks)

public:    slice=6 offset=0 len=4063042 disk_offset=131260     <<< New disk offset of the public region partition

private:   slice=5 offset=1 len=131072 disk_offset=126

update:    time=1325661626 seqno=0.9

ssb:       actual_seqno=0.0

headers:   0 248

configs:   count=1 len=107282

logs:      count=1 len=4096

Defined regions:

 config   priv 000017-000247[000231]: copy=01 offset=000000 enabled

 config   priv 000249-107299[107051]: copy=01 offset=000231 enabled

 log      priv 107300-111395[004096]: copy=01 offset=000000 enabled

Multipathing information:

numpaths:   1

vdj             state=enabled

Import the diskgroup and now the volume will have the data in the correct location.

# vxdg import cdg

# vxvol -g cdg startall

Etrack 2561012 is fixed in SF 5.1SP1RP3. Please download the required patch from the Veritas Operation Readiness Tools website.

https://sort.Veritas.com/patch/matrix

Applies To

Veritas Storage Foundation 5.1 on Linux

Issue/Introduction

Data in a Veritas Volume Manager (VxVM) volume is shifted by 1 to 15 sectors because VxVM uses a different effective Public Region offset. The effective Public Region offset is used to align the subdisks according to the diskgoup alignment setting.

Additional Information

ETrack: 2561012

Was this article helpful?

thumb_up Yes

thumb_down No

Welcome to "KB Articles"