Bug 1260551

Summary: Upgrade rhev-h from 7.1 to 7.2 via RHEV-M 3.5 failed because vdsmd does not come up
Product: Red Hat Enterprise Virtualization Manager Reporter: cshao <cshao>
Component: ovirt-nodeAssignee: Ryan Barry <rbarry>
Status: CLOSED ERRATA QA Contact: cshao <cshao>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 3.6.0CC: cwu, fdeutsch, gklein, huiwa, huzhao, leiwang, lsurette, rbarry, troels, yaniwang, ycui, ykaul
Target Milestone: ovirt-3.6.1Keywords: TestBlocker
Target Release: 3.6.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: ovirt-node-3.6.1-5.0.el7ev Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-03-09 14:37:03 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Node RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
/var/log/*.* + sosreport + engine.log
none
rhevm-failed.png none

Description cshao 2015-09-07 08:45:40 UTC
Created attachment 1070863 [details]
/var/log/*.*   +  sosreport  +  engine.log

Description of problem:
Upgrade rhev-h from 7.1 to 7.2 via RHEV-M failed.

Version-Release number of selected component (if applicable):
rhev-hypervisor-7-7.1-20150827.1
rhev-hypervisor-7-7.2-20150831.0
ovirt-node-3.3.0-0.2.20150722git7eba125.el7ev.noarch
vdsm-4.17.3-1.el7ev.noarch
RHEV-M vt16.9
Red Hat Enterprise Virtualization Manager Version: 3.5.4-1.3.el6ev

How reproducible:
100%

Steps to Reproduce:
1. Install rhevh 7.1 3.5.4GA build(rhev-hypervisor-7-7.1-20150827.1).
2. Register to RHEV-M 3.5.
3. Maintenance the host.
4. Upgrade to RHEV-H 7.2 via RHEVM Web UI.

Test result:
1. Upgrade failed via RHEV-M, more detail info please refer log.
2. RHEV-H host will not reboot after the upgrade.

Expected results:
Upgrade rhev-h from 7.1 to 7.2 via RHEV-M can successful.

Additional info:

Comment 1 cshao 2015-09-07 08:46:38 UTC
Created attachment 1070864 [details]
rhevm-failed.png

Comment 2 cshao 2015-09-07 08:55:38 UTC
Update vdsmd server info.

# systemctl status vdsmd
vdsmd.service - Virtual Desktop Server Manager
   Loaded: loaded (/usr/lib/systemd/system/vdsmd.service; enabled)
   Active: inactive (dead) since Mon 2015-09-07 07:32:20 UTC; 1h 20min ago
  Process: 17762 ExecStopPost=/usr/libexec/vdsm/vdsmd_init_common.sh --post-stop (code=exited, status=0/SUCCESS)
  Process: 17537 ExecStart=/usr/share/vdsm/daemonAdapter -0 /dev/null -1 /dev/null -2 /dev/null /usr/share/vdsm/vdsm (code=exited, status=0/SUCCESS)
  Process: 17425 ExecStartPre=/usr/libexec/vdsm/vdsmd_init_common.sh --pre-start (code=exited, status=0/SUCCESS)
 Main PID: 17537 (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/vdsmd.service

Sep 07 07:31:10 localhost vdsm[17537]: vdsm ovirt_hosted_engine_ha.env.config ERROR Configuration file '/etc/ovirt-hosted-engine/vm.conf' not available [[Errno 2...e/vm.conf']
Sep 07 07:31:11 localhost vdsm[17537]: vdsm ovirt_hosted_engine_ha.env.config ERROR Configuration file '/etc/ovirt-hosted-engine/hosted-engine.conf' not availabl...gine.conf']
Sep 07 07:31:11 localhost vdsm[17537]: vdsm ovirt_hosted_engine_ha.env.config ERROR Configuration file '/etc/ovirt-hosted-engine/vm.conf' not available [[Errno 2...e/vm.conf']
Sep 07 07:31:11 localhost vdsm[17537]: vdsm vds ERROR failed to retrieve Hosted Engine HA info
                                       Traceback (most recent call last):
                                         File "/usr/share/vdsm/API.py", line 1703, in _getHaInfo...
Sep 07 07:32:14 localhost systemd[1]: Stopping Virtual Desktop Server Manager...
Sep 07 07:32:20 localhost python[17537]: DIGEST-MD5 client mech dispose
Sep 07 07:32:20 localhost python[17537]: DIGEST-MD5 common mech dispose
Sep 07 07:32:20 localhost vdsmd_init_common.sh[17762]: vdsm: Running run_final_hooks
Sep 07 07:32:20 localhost systemd[1]: Stopped Virtual Desktop Server Manager.
Sep 07 07:32:23 localhost systemd[1]: Stopped Virtual Desktop Server Manager.
Hint: Some lines were ellipsized, use -l to show in full.
[root@localhost admin]#

Comment 3 Ying Cui 2015-09-07 09:09:24 UTC
# ovirt-node-upgrade.log
<snip>
2015-09-07 07:33:34,623 - ERROR    - ovirt-node-upgrade - Error: Upgrade Failed: Bootloader Installation Failed
Traceback (most recent call last):
  File "/usr/sbin/ovirt-node-upgrade", line 366, in run
    self._run_upgrade()
  File "/usr/sbin/ovirt-node-upgrade", line 256, in _run_upgrade
    raise RuntimeError("Bootloader Installation Failed")
RuntimeError: Bootloader Installation Failed
</snip>

Comment 6 cshao 2015-10-27 09:13:12 UTC
Test version:
rhev-hypervisor-7-7.1-20150827.1
rhev-hypervisor7-7.2-20151025.0.el7ev
ovirt-node-3.3.0-0.18.20151022git82dc52c.el7ev.noarch

Test steps:
1. Install rhevh 7.1 3.5.4GA build(rhev-hypervisor-7-7.1-20150827.1).
2. Register to RHEVM3.5
3. Maintenance the host.
4. Upgrade to 7.2-1025

Test result:
Still met issue like bug 1260559 #c9 show.
 An error occurs in the UI: IOError(2, ‘No such file or directory’)
Press ENTER to logout
or enter 's' to drop to shell

Hi rbarry,

  Seem this is new issue occurs, can I verify this bug and then report a new one?

Thanks!

Comment 7 Ryan Barry 2015-10-27 20:15:59 UTC
Yes, please verify both and report a new one.

I'm trying to reproduce, to see whether it's all upgrades from 7.1->7.2, or just 3.5->3.6 (as 1025 is 3.6, and 0827.1 is 3.5).

Comment 8 Ryan Barry 2015-10-27 23:05:13 UTC
Only upgrades to 7.2/3.6 are affected

Comment 9 cshao 2015-10-28 08:30:37 UTC
(In reply to Ryan Barry from comment #7)
> Yes, please verify both and report a new one.
> 
> I'm trying to reproduce, to see whether it's all upgrades from 7.1->7.2, or
> just 3.5->3.6 (as 1025 is 3.6, and 0827.1 is 3.5).

VERIFY this bug according #c6 & c7.

And the new bug is here:
Bug 1275956 - In Boot process show IOError info when upgrade via TUI/AUTO/RHEVM

Comment 11 errata-xmlrpc 2016-03-09 14:37:03 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-0378.html