Bug 1429594 - [downstream clone - 4.0.7] RHVH 4.0.7 cannot be up again in the side of RHEVM 4.0 after upgrade
Summary: [downstream clone - 4.0.7] RHVH 4.0.7 cannot be up again in the side of RHEVM...
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: rhev-hypervisor-ng
Version: 4.0.7
Hardware: Unspecified
OS: Unspecified
Target Milestone: ovirt-4.0.7
: ---
Assignee: Yuval Turgeman
QA Contact: jianwu
Depends On: 1429288
TreeView+ depends on / blocked
Reported: 2017-03-06 16:31 UTC by rhev-integ
Modified: 2017-03-16 15:40 UTC (History)
19 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: vdsm is not configured after an upgrade of RHVH Consequence: RHVH is not brought up Fix: Running vdsm-tool configure --force on boot sets up the different vdsm modules Result: vdsm is configured and RHVH is up
Clone Of: 1429288
Last Closed: 2017-03-16 15:40:34 UTC
oVirt Team: Node
Target Upstream Version:

Attachments (Terms of Use)

System ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2017:0549 normal SHIPPED_LIVE Moderate: redhat-virtualization-host security and bug fix update 2017-03-16 19:26:25 UTC
oVirt gerrit 73596 master MERGED Added systemd unit for running vdsm-tool configure 2017-03-06 16:33:08 UTC
oVirt gerrit 73597 master MERGED Enabling imgbase-config-vdsm unit 2017-03-06 16:33:08 UTC
oVirt gerrit 73621 ovirt-4.0 MERGED Added systemd unit for running vdsm-tool configure 2017-03-06 16:36:05 UTC
oVirt gerrit 73622 ovirt-4.1 MERGED Added systemd unit for running vdsm-tool configure 2017-03-06 16:33:08 UTC
oVirt gerrit 73623 ovirt-4.0 MERGED Enabling imgbase-config-vdsm unit 2017-03-06 16:35:53 UTC
oVirt gerrit 73624 ovirt-4.1 MERGED Enabling imgbase-config-vdsm unit 2017-03-06 16:33:08 UTC

Description rhev-integ 2017-03-06 16:31:44 UTC
+++ This bug is a downstream clone. The original bug is: +++
+++   bug 1429288 +++

Created attachment 1260242 [details]
4.0.7 related log about upgrade issue

Description of problem:

when register old bulid  RHVH 4.0 to rhevm 4.0, it cannot be up again after upgrade to RHVH 4.0.7 new build from engine side

Version-Release number of selected component (if applicable):
Before upgrade:
After upgrade:
redhat-virtualization-host-4.0-20170302.0.x86_64(new build)

How reproducible:

Steps to Reproduce:
1. Install RHVH 4.0 old build via anaconda GUI
2. Reboot and set local repo
3. Register to rhevm 4.0 and upgrade from rhevm side
4. Upgrade successfully and Reboot into new system
5. Check RHVH status in the side of rhevm
6. Check vdsm status # service vdsmd status

Actual results:
After step 5, RHVH 4.0.7 cannot be up in the side of rhevm 4.0
After step 6, 
#service vdsmd status
   vdsmd.service - Virtual Desktop Server Manager
   Loaded: loaded (/usr/lib/systemd/system/vdsmd.service; enabled; vendor preset: enabled)
   Active: failed (Result: start-limit) since Sun 2017-03-05 19:00:20 CST; 23min ago
  Process: 5023 ExecStartPre=/usr/libexec/vdsm/vdsmd_init_common.sh --pre-start (code=exited, status=1/FAILURE)

Expected results:
After step 5, RHVH 4.0.7 can be up in the side of rhevm 4.0

Additional info:
1. The same issue about upgrade via "yum update" way.
2. The RHVH 4.0.7 will be up in the rhevm side when install directly

(Originally by Jian Wu)

Comment 1 rhev-integ 2017-03-06 16:31:57 UTC
Created attachment 1260243 [details]
related picture about this issue

(Originally by Jian Wu)

Comment 3 rhev-integ 2017-03-06 16:32:05 UTC
Created attachment 1260244 [details]
another picture about vdsmd.service

(Originally by Jian Wu)

Comment 4 rhev-integ 2017-03-06 16:32:12 UTC
This bug blocks upgrade test in RHEVM side, and the same issue happens via "yum update" way ,so I could not check status in rhevm side and related test.

(Originally by Jian Wu)

Comment 5 rhev-integ 2017-03-06 16:32:20 UTC
Can you please provide journalctl output?

(Originally by Ryan Barry)

Comment 6 rhev-integ 2017-03-06 16:32:27 UTC
Created attachment 1260262 [details]
journalctl output

(Originally by Jian Wu)

Comment 7 rhev-integ 2017-03-06 16:32:35 UTC
(In reply to Ryan Barry from comment #4)
> Can you please provide journalctl output?

I have upload attachment 1260262 [details] for journalcl output, please check it


(Originally by Jian Wu)

Comment 8 rhev-integ 2017-03-06 16:32:44 UTC
Thanks jianwu.

I'm not sure this is unexpected behavior, though it's not nice.

We've carefully avoided adding a 'vdsm-tool configure --force' as part of NGN booting (which vintage RHV-H had).

The version of RHV-H used for upgrading was particularly old, but I'm not sure how vdsm handles this behind the scenes.

The version of lvmlocal.conf from the old version did not have the necessary configuration. In fact, it appears to have matched the installed version exactly:

# mount
/dev/mapper/rhvh_dhcp--10--229-rhvh--4.0--0.20160919.0+1 on /tmp/a type xfs (rw,relatime,seclabel,attr2,inode64,logbsize=256k,sunit=512,swidth=512,noquota)
# diff -u /tmp/a/etc/lvm/lvmlocal.conf /tmp/a/usr/share/factory/etc/lvm/lvmlocal.conf

In this case, we took the lvmlocal.conf from the new image, but this is also not what vdsm expects. Differences between /etc/lvm/lvmlocal.conf before and after "vdsm-tool configure --force" are significant.

I don't see anything in the relevant topic branch which looks like vdsm should keep/handle this on upgrades either. I'm not very familiar with how vdsm handles upgrades, though.

A quick glance at the specfile show:

if ! %{_bindir}/vdsm-tool is-configured >/dev/null 2>&1; then
   %{_bindir}/vdsm-tool configure --force >/dev/null 2>&1

I'd like to match this if possible.


Can you please add a method in plugins/osupdater which matches this?

(Originally by Ryan Barry)

Comment 10 jianwu 2017-03-09 03:28:33 UTC
I have test this bug on redhat-virtualization-host-4.0-20170307.1.x86_64

Test version:
Before upgrade:
After upgrade:
redhat-virtualization-host-4.0-20170307.1.x86_64(new build)

Test results:
Vdsmd.service works well and New RHVH could be up in rhevm side

SO I think this bug is fixed in this build.

Comment 12 cshao 2017-03-09 14:17:35 UTC
Verify this bug according #c10

Comment 14 errata-xmlrpc 2017-03-16 15:40:34 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.