Created attachment 1082836 [details] sos report Description of problem: RHEV-H upgrade from 3.5.4 to 3.5.5 failed, because vdsm not stopped Version-Release number of selected component (if applicable): From: Red Hat Enterprise Virtualization Hypervisor release 7.1 (20150911.0.el7ev) ovirt-hosted-engine-ha-1.2.6-3.el7ev.noarch ovirt-node-plugin-hosted-engine-0.2.0-18.0.el7ev.noarch ovirt-hosted-engine-setup-1.2.5.3-1.el7ev.noarch vdsm-4.16.26-1.el7ev.x86_64 To: rhev-hypervisor7-7.1-20151013.0.el7ev How reproducible: Always Steps to Reproduce: 1. Install rhev-hypervisor7-7.1-20150911.0.el7ev(https://brewweb.devel.redhat.com/buildinfo?buildID=456370) on two hosts 2. Deploy hosted-engine on both environments(not sure if it connect) 3. Put host without engine vm to maintenance, and try to upgrade host to rhev-hypervisor7-7.1-20151013.0.el7ev(https://brewweb.devel.redhat.com/buildinfo?buildID=460832) Actual results: Upgrade fail with error message under ovirt-node-upgrade.log 2015-10-14 07:42:39,705 - DEBUG - ovirt-node-upgrade - Failed to stop vdsdm: Error: ServiceOperationError: _systemctlStop failed Job for vdsmd.service canceled. 2015-10-14 07:42:39,705 - ERROR - ovirt-node-upgrade - Error: Upgrade Failed: Command Failed: '('/usr/libexec/ovirt-node/hooks/pre-upgrade/01-vdsm',)' [u'/usr/libexec/ovirt-node/hooks/pre-upgrade/01-vdsm: Stopping vdsmd to upgrade'] Traceback (most recent call last): File "/usr/sbin/ovirt-node-upgrade", line 365, in run self._run_hooks("pre-upgrade") File "/usr/sbin/ovirt-node-upgrade", line 197, in _run_hooks self._system(hook) File "/usr/sbin/ovirt-node-upgrade", line 145, in _system raise RuntimeError("Command Failed: '%s' %s" % (command, output)) RuntimeError: Command Failed: '('/usr/libexec/ovirt-node/hooks/pre-upgrade/01-vdsm',)' [u'/usr/libexec/ovirt-node/hooks/pre-upgrade/01-vdsm: Stopping vdsmd to upgrade'] Expected results: Upgrade succeed Additional info: The same problem exist in RHEV-H 6.7
*** Bug 1271706 has been marked as a duplicate of this bug. ***
After discussing offline with Fabian, moving to SLA for examination.
Raising the priority Can someone tell from the vdsm logs what failed while shutting down vdsm?
Raising the priority even more, because it's blocking the RHEV-M upgrade flow.
Currently we have 3.5.7 in the works and no similar reports. Please re-open if you can reproduce on 3.5.7.