Changes to device minor numbers cause NFS clients to fail

book

Article ID: 100027114

calendar_today

Updated On:

Description

Error Message

Apr 27 10:39:51 node1 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-14563 Disk group mydg: base minor was in private pool, will be change to shared pool.
Apr 27 10:39:51 node1 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-14562 Disk group mydg: base minor dynamically changed from 25000 to 49000.
Apr 27 10:39:51 node1 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-14565 Disk group mydg: device vol01 dynamically reminored from 25000 to 49000.
Apr 27 10:39:51 node1 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-14565 Disk group mydg: device vol02 dynamically reminored from 25001 to 49001.
Apr 27 10:39:51 node1 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-14565 Disk group mydg: device vol03 dynamically reminored from 25002 to 49002.
Apr 27 10:39:51 node1 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-14565 Disk group mydg: device vol04 dynamically reminored from 25003 to 49003.
Apr 27 10:39:51 node1 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-14565 Disk group mydg: device vol05 dynamically reminored from 25004 to 49004.

Cause

Starting in release 5.1, VxVM allocates separate pools of minor numbers for private and shared disk groups to avoid minor number conflicts. VxVM dynamically reminors the disk group and its objects if the minor numbers are not assigned from the correct pools. VxVM also dynamically reminors the disk group if a minor number conflict occurs for another reason.

NFS uses the minor number as part of the disk identifier during import. Therefore, the changes in the minor number cause NFS clients to fail, because the file handles become stale.

See also the incident mentioned in the supplemental materials section.

Resolution

The solution depends on the product version.

Solution for 5.1SP1RP3P1 or later patches
 
The behavior is changed in version 5.1SP1 RP3P1. During disk group import, VxVM checks to make sure the minor numbers do not conflict with the minor numbers for another disk group.  If VxVM does not detect a conflict, there is no need to reminor the disk group. The disk group import  succeeds, regardless of whether the minor numbers are from the correct pool.
 
If VxVM detects a minor number conflict, the conflict must be resolved (either by VxVM or by manual steps) before the disk group can be imported. A new tunable parameter autoreminor controls whether VxVM automatically assigns new minor numbers for the disk group.  The autoreminor tunable parameter has one of the following values:
  • on . When VxVM detects a minor number conflict, VxVM assigns new minor numbers for the disk group  from the correct pool.  VxVM uses the value of the tunable parameter sharedminorstart to determine where to start the minor numbers for shared disk groups. After the disk group is reminored, the disk group import succeeds. The default setting is on.
  • off . When VxVM detects a minor number conflict, the disk group import fails with an error. You must manually resolve the minor number conflict before importing the disk group.
  In an NFS environment, we recommend setting the autoreminor tunable parameter to off.  Use the following steps to set the autoreminor tunable to off.  

     1.       Set the autoreminor tunable to off on a NFS file server. If the NFS server is in a VCS cluster, make the same change on all nodes in the cluster.

     # vxdefault set autoreminor off

      2.        If required, change the starting point of the minor number pool for shared disk groups. If the NFS server is in a VCS cluster, make the same change on all nodes in the cluster.

3.       Run the following command on the system. If the system is part of a VCS cluster, run the command on each node.
 
# vxdctl enable
 
Note:  The tunable parameters autoreminor and sharedminorstart must have the same values on each node of the cluster.
 
When autoreminor is set to off, a disk group import fails if there is a minor number conflict.  Use the following steps to resolve the minor number conflict.
 
1. Import the disk group with a temporary reminor number:
 
# vxdg -t -o reminor import diskgroup
 
2. Reminor the disk group.
 
# vxdg reminor diskgroup
 
 
Solution for version 5.1   
 
Disable the dynamic reminoring between shared and private minor numbers.  You can do this before or after you upgrade VxVM.
 
To resolve before you upgrade VxVM from a release prior to 5.1 .
 
1. Add or modify the following entry in the /etc/default/vxsf file on a NFS file server. If the NFS server is in a VCS cluster, make similar changes on all nodes in the cluster.
 
The following entry disables the division between shared and private minor numbers:
  sharedminorstart=0
 
2. Proceed with product upgrade using the installer or other supported methods.
 
 
To resolve after upgrading VxVM from a release prior to 5.1
 
If you did not add above entry during update procedure, perform the following steps to resolve this issue.
 
1. Add the following entry in the /etc/default/vxsf file on a NFS file server. If the NFS server is in a VCS Cluster, make similar changes on all nodes in the   cluster.
sharedminorstart=0
 
2. Run the following command on the system. If the system is part of a VCS cluster, run the command on each node.
 
# vxdctl enable
 
3. Follow one of the following sets of steps.
 
For non-VCS environment
 
a. Un-share and unmount file systems on the NFS server.
 
b. Reminor the disk group using the old minor number.
 
# vxdg -g mydg -f reminor 25000
 
c. Mount the file systems and share again.
 
d. NFS client can now reconnect to the NFS server.
 
 
For VCS environment
 
a. Bring the resources offline. Share and mount the resources.
 
b. Reminor the disk group using the old minor number.
 
# vxdg -g mydg -f reminor 25000
 
c. Bring the resources online. Mount and share the resources.
 
d. The NFS client can now reconnect to the NFS server.

 


Applies To

VxVM 5.1 or later on all supported platforms

Issue/Introduction

Veritas Volume Manager (VxVM) assigns unique device minor numbers to each object (volume, plex, subdisk, disk, or disk group) that it controls. A device minor number  uniquely identifies some characteristic of a device to the device driver that controls that device. Starting in release 5.1, VxVM automatically reminors the disk group and its objects in certain cases. NFS clients can fail, because the file handles become stale. The issue is seen in the following situations:
  • When a private disk group is imported as shared, or a shared disk group is imported as private.
  • After upgrading from a version of VxVM prior to 5.1
The issue is resolved in VxVM 5.1SP1RP3P1.

Additional Information

ETrack: 2768458
Powered by Wolken Software