Encapsulated boot disk - Upgrade from SF 6.1 directly to 6.2.1.x00 hangs at "svcadm disable vxvm-sysboot" on Solaris platform

book

Article ID: 100041198

calendar_today

Updated On:

Description

Error Message

The Cross-Platform Installer (CPI) program "installmr" hangs with the following process tree when it tries to install the VxVM 6.2.1.400 patch. 

                18294 /space/sf62/mr/sol10_sparc/perl/SolSparc/bin/perl -x -S -I/space/sf62/mr/sol10_
                  1487  sh -c /usr/sbin/patchadd   /space/sf62/vm621_400/patches/151232-04 2>>/var/tmp/
                    1488  /usr/sbin/patchadd /space/sf62/vm621_400/patches/151232-04
                      1489  /bin/ksh -hp /usr/lib/patch/patchadd -M /space/sf62/vm621_400/patches -Y /tmp/.
                        2861  pkgadd -O patchPkgInstall -O nozones -O enable-hollow-package-support -G -S -n
                          2864  /usr/sadm/install/bin/pkginstall -O patchPkgInstall -O pkg-server-mode=timeout1
                            2957  /sbin/sh /var/sadm/pkg/VRTSvxvm/install/preinstall
                              3399  svcadm disable -s system/vxvm/vxvm-sysboot     <<< this process  hangs

Cause

In an environment with the boot disk encapsulated, during the upgrade from VxVM 6.1 to 6.2, the VxVM kernel modules (e.g. vxio, vxdmp and vxspec) cannot be unloaded because the volumes encapsulating the boot partitions are still being used.  The following progress messages will be logged.

    Stopping vxconfigd ..................................................................................... Skipped
    Stopping vxspec ........................................................................................ Skipped
    Stopping vxio .......................................................................................... Skipped
    Stopping vxdmp ......................................................................................... Skipped

Because the VxVM 6.1 kernel modules cannot be unloaded and reloaded to 6.2 version, when the upgrade to VxVM 6.2 GA version is finished, no VxVM command including the vxconfigd can be started.  

If it is only an upgrade to SF 6.2 GA version, it will not be a problem because when the upgrade to 6.2 finishes, the CPI installer will prompt the user to reboot the system.   After the system reboots, the VxvM 6.2 kernel modules will be loaded and everything will be fine.

It is also not a major issue when upgrading only to VxVM 6.2.1 by using the "-patch_path" option.  Although there will be one more minor issue where the Solaris Service Management Facitlity (SMF) service vxvm-sysboot will be stuck in the transition state with the following messages displayed on the console.

VxVM sysboot INFO V-5-2-3409 starting in boot mode...
Nov 22 01:14:19 sclt2000-30.labs.abc.com vxvm:vxconfigd: V-5-1-543 Differing version of vxconfigd installed
VxVM sysboot WARNING V-5-0-4817 Error starting vxconfigd
VxVM sysboot WARNING V-5-0-0 Errors were encountered in starting the boot disk group, as a result VxVM is unable to configure the root and/or /usr volumes.  If you have mirrored the root disk, you can try booting from that disk.
If you cannot boot from the root disk, you can try to repair the problem using a network-mounted root file system or some other alternate root file system.
Continue with system boot? [no]

This is because the VxVM 6.2.1 patch installation script will try to restart the VxVM SFM services including vxvm-sysboot.   In the vxvm-sysboot start method, these are the following codes.

-------------
if [ -n "$vold_must_succeed" ]; then       <<< $vol_must_succeed is set because the boot disk is encapsulated, we need VxVM to start the rootvol.
        msg="VxVM sysboot WARNING V-5-0-0
Errors were encountered in starting the boot disk group, as a result
.....
        $UMIECHO "$msg\n"
        if [ "$rel" != "5.10" -a "$rel" != "5.11" ]; then
                echo_console "Would you like a shell prompt right now? [no] \c"
                exec < /dev/console
                read yorn
                case $yorn in
                        y|yes)  /sbin/sh;;
                esac
        fi
        echo_console "Continue with system boot? [no] \c"
        exec < /dev/console
        read yorn                 <<< vxvm-sysboot start method is waiting here
        case $yorn in
                y|yes)  exit 0;;
        esac
        $UMIECHO "VxVM sysboot NOTICE V-5-2-3388 Halting system..."
        sync;sync;uadmin 2 0                <<< DON'T type ENTER to the console; otherwise, it will halt the system abruptly and kill the upgrade
fi
-----------------------------

The "svcs -a" command shows that vxvm-sysboot service is in the transition state.

From the svcs(1) manual page:
Absent or unrecognized states are denoted by a  question mark  (?)  character.  
An  asterisk  (*) is appended for instances in  transition, unless the NSTA or NSTATE column is also being displayed.

# svcs -a | grep vxvm-sysboot
offline*        1:14:18 svc:/system/vxvm/vxvm-sysboot:default     <<< in transition

This is not a major isssue because it will not block the CPI installer from continuing with the installation of other patches and the upgrade will eventually finish even if the vxvm-sysboot service is stuck in the transition state.   This may affect the shutdown of the system but if the system is rebooted, the VxVM 6.2.1 kernel modules will be successfully loaded after reboot.

**********
Please note that one should not press "Enter" or answer "no" to the prompt "Continue with system boot? [no]".  If one presses "Enter" (default answer is "no") or answers "no", the system will be halt immediately with the "udadmin 2 0" command and this may abort the upgrade because it cannot finish.
**********

This will be a problem when the VxVM 6.2.1.x00 patch is installed.   The installation of the VxVM 6.2.1.x00 patch will also try to restart the VxVM SMF services including vxvm-sysboot.  But since the "vxvm-sysboot" is now in the transition state waiting for the user to answer "yes" or "no" to the prompt "Continue with system boot? [no]".   The installation will now be stuck in the following process tree.

                18294 /space/sf62/mr/sol10_sparc/perl/SolSparc/bin/perl -x -S -I/space/sf62/mr/sol10_
                  1487  sh -c /usr/sbin/patchadd   /space/sf62/vm621_400/patches/151232-04 2>>/var/tmp/
                    1488  /usr/sbin/patchadd /space/sf62/vm621_400/patches/151232-04
                      1489  /bin/ksh -hp /usr/lib/patch/patchadd -M /space/sf62/vm621_400/patches -Y /tmp/.
                        2861  pkgadd -O patchPkgInstall -O nozones -O enable-hollow-package-support -G -S -n
                          2864  /usr/sadm/install/bin/pkginstall -O patchPkgInstall -O pkg-server-mode=timeout1
                            2957  /sbin/sh /var/sadm/pkg/VRTSvxvm/install/preinstall
                              3399  svcadm disable -s system/vxvm/vxvm-sysboot     <<< this process  hangs

The "svcadm disable" command is hung waiting for the vxvm-sysboot service to get out of the transition state.

The CPI installation will hang in the following step.

    Installing 151232-04 patch -        <<< this hangs !!!

Resolution

The solution to the above problem is that we need to answer "yes" to the prompt "Continue with system boot? [no]".  Once "yes" is selected, the vxvm-sysboot service will come out of the transition state and the system will remain running and the "svcadm disable" command will finish successfully. The installer will then be able to continue with the rest of the upgrade procedures. 

Continue with system boot? [no] yes          <<< Answer "yes" here

Issue/Introduction

In an environment with the boot disk encapsulated, upgrading from Storage Foundation (SF) 6.1 directly to 6.2.1.x00 hangs at the process "svcadm disable -s system/vxvm/vxvm-sysboot" on Solaris platform. For example, a system can be upgraded to 6.2.1.400 directly using the following single installmr command. /space/sf62/mr/sol10_sparc/installmr \
-base_path /space/sf62/ga/dvd1-sol_sparc/sol10_sparc \
-patch_path /space/sf62/vm621_400