Bug 1478641 - Failed to install the latest RHVH (redhat-virtualization-host-4.1-20170804.0)
Failed to install the latest RHVH (redhat-virtualization-host-4.1-20170804.0)
Status: CLOSED CURRENTRELEASE
Product: ovirt-node
Classification: oVirt
Component: Installation & Update (Show other bugs)
4.1
Unspecified Unspecified
urgent Severity urgent (vote)
: ovirt-4.1.4
: 4.1
Assigned To: Ryan Barry
Qin Yuan
: Regression, TestBlocker
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2017-08-05 08:43 EDT by Qin Yuan
Modified: 2017-11-08 11:39 EST (History)
15 users (show)

See Also:
Fixed In Version: imgbased-0.9.41-0.1.el7ev
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-08-23 04:04:52 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Node
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
rule-engine: ovirt‑4.1+
rule-engine: blocker+
rbarry: devel_ack+
ycui: testing_ack+


Attachments (Terms of Use)
logs under /tmp during installation (209.05 KB, application/x-gzip)
2017-08-05 08:43 EDT, Qin Yuan
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
oVirt gerrit 80215 ovirt-4.1 ABANDONED osupdater: fix a typo in setfiles 2017-08-07 11:49 EDT

  None (edit)
Description Qin Yuan 2017-08-05 08:43:51 EDT
Created attachment 1309393 [details]
logs under /tmp during installation

Description of problem:
Failed to install the latest RHVH, that is redhat-virtualization-host-4.1-20170804.0, there are error popped up at the end of installation:

2017-08-05 20:17:05,856 [INFO] (MainThread) Trying to create a manageable base from '/'
2017-08-05 20:17:06,656 [INFO] (MainThread) Initial base will be <Base rhvh-4.1-0.20170804.0 [] />
2017-08-05 20:17:06,656 [INFO] (MainThread) Initial layer will be <Layer rhvh-4.1-0.20170804.0+1 />
2017-08-05 20:17:06,657 [INFO] (MainThread) Creating an initial base <Base rhvh-4.1-0.20170804.0 [] /> for <LV 'rhvh_bootp-73-75-155/root' />
2017-08-05 20:17:08,002 [INFO] (MainThread) Creating initial layer <Layer rhvh-4.1-0.20170804.0+1 /> for initial base
2017-08-05 20:17:08,003 [INFO] (MainThread) Adding a new layer after <Base rhvh-4.1-0.20170804.0 [] />
2017-08-05 20:17:08,003 [INFO] (MainThread) Adding a new layer after <Base rhvh-4.1-0.20170804.0 [] />
2017-08-05 20:17:08,003 [INFO] (MainThread) New layer will be: <Layer rhvh-4.1-0.20170804.0+1 />
2017-08-05 20:17:09,789 [INFO] (migrate_etc) Verifying stream compatability
2017-08-05 20:17:09,871 [INFO] (migrate_etc) Migrating /etc (from <LV 'rhvh_bootp-73-75-155/rhvh-4.1-0.20170804.0+1' />)
2017-08-05 20:17:11,284 [INFO] (migrate_etc) Migrating /root
2017-08-05 20:17:11,287 [INFO] (fix_systemd_services) Syncing systemd levels
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/imgbased/utils.py", line 1473, in thread_group_handler
    t.join_with_exceptions()
  File "/usr/lib/python2.7/site-packages/imgbased/utils.py", line 1465, in join_with_exceptions
    raise exc[1]
TypeError: coercing to Unicode: need string or buffer, list found
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/imgbased/utils.py", line 1473, in thread_group_handler
    t.join_with_exceptions()
  File "/usr/lib/python2.7/site-packages/imgbased/utils.py", line 1465, in join_with_exceptions
    raise exc[1]
SystemExit: 1
2017-08-05 20:19:06,242 [ERROR] (MainThread) Failed to migrate etc
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/imgbased/plugins/osupdater.py", line 117, in on_new_layer
    thread_group_handler(threads)
  File "/usr/lib/python2.7/site-packages/imgbased/utils.py", line 1476, in thread_group_handler
    sys.exit(1)
SystemExit: 1
Traceback (most recent call last):
  File "/usr/lib64/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/lib/python2.7/site-packages/imgbased/__main__.py", line 53, in <module>
    CliApplication()
  File "/usr/lib/python2.7/site-packages/imgbased/__init__.py", line 82, in CliApplication
    app.hooks.emit("post-arg-parse", args)
  File "/usr/lib/python2.7/site-packages/imgbased/hooks.py", line 120, in emit
    cb(self.context, *args)
  File "/usr/lib/python2.7/site-packages/imgbased/plugins/core.py", line 173, in post_argparse
    layout.initialize(args.source, args.init_nvr)
  File "/usr/lib/python2.7/site-packages/imgbased/plugins/core.py", line 220, in initialize
    self.app.imgbase.init_layout_from(source, init_nvr)
  File "/usr/lib/python2.7/site-packages/imgbased/imgbase.py", line 292, in init_layout_from
    self.add_layer(initial_base, new_layer)
  File "/usr/lib/python2.7/site-packages/imgbased/imgbase.py", line 191, in add_layer
    self.hooks.emit("new-layer-added", prev_lv, new_lv)
  File "/usr/lib/python2.7/site-packages/imgbased/hooks.py", line 120, in emit
    cb(self.context, *args)
  File "/usr/lib/python2.7/site-packages/imgbased/plugins/osupdater.py", line 120, in on_new_layer
    raise ConfigMigrationError()
imgbased.plugins.osupdater.ConfigMigrationError 


Version-Release number of selected component (if applicable):
redhat-virtualization-host-4.1-20170804.0 
imgbased-0.9.39-0.1.el7ev.noarch


How reproducible:
100%


Steps to Reproduce:
1. Install redhat-virtualization-host-4.1-20170804.0 via PXE, and use the Anaconda GUI mode.
2. Choose auto partitioning, and finish other needed steps. 


Actual results:
1. After step2, there are errors popped up at the end of installation as above. 


Expected results:
1. After step2, installation could succeed without errors. 


Additional info:
Comment 1 Red Hat Bugzilla Rules Engine 2017-08-05 10:25:39 EDT
Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.
Comment 2 Red Hat Bugzilla Rules Engine 2017-08-05 10:25:39 EDT
This bug report has Keywords: Regression or TestBlocker.
Since no regressions or test blockers are allowed between releases, it is also being identified as a blocker for this release. Please resolve ASAP.
Comment 3 Qin Yuan 2017-08-10 01:38:46 EDT
Verify Versions:
redhat-virtualization-host-4.1-20170808.0
imgbased-0.9.41-0.1.el7ev.noarch

Verify Steps:
As described in comment #0

redhat-virtualization-host-4.1-20170808.0 can be installed successfully, so change the status to VERIFIED.
Comment 4 Mike Goodwin 2017-11-07 23:56:55 EST
This bug is present in ovirt-node-ng-installer-ovirt-4.1-2017110715.iso

Why does this keep happening?
Comment 5 cshao 2017-11-08 05:35:36 EST
(In reply to Mike Goodwin from comment #4)
> This bug is present in ovirt-node-ng-installer-ovirt-4.1-2017110715.iso
> 
> Why does this keep happening?

This is new issue.
see https://bugzilla.redhat.com/show_bug.cgi?id=1510858
Comment 6 Ryan Barry 2017-11-08 08:58:03 EST
(In reply to Mike Goodwin from comment #4)
> This bug is present in ovirt-node-ng-installer-ovirt-4.1-2017110715.iso
> 
> Why does this keep happening?

Mike -

Since the osupdater code handles both installs and updates, sometimes a change in one affects the other. The bugs you commented on are unrelated (this bug was never observable in a publicly available build).
Comment 7 Mike Goodwin 2017-11-08 11:18:22 EST
(In reply to Ryan Barry from comment #6)
> (In reply to Mike Goodwin from comment #4)
> > This bug is present in ovirt-node-ng-installer-ovirt-4.1-2017110715.iso
> > 
> > Why does this keep happening?
> 
> Mike -
> 
> Since the osupdater code handles both installs and updates, sometimes a
> change in one affects the other. The bugs you commented on are unrelated
> (this bug was never observable in a publicly available build).

Thanks for replying. I realized that later on (about the script being used for both upgrade and install) when I backed off of 4.1.7 install and installed with the 4.1.6 image from september and tried to upgrade via yum upgrade. The install worked, but the upgrade did not. 

It's another matter, but there's no documentation on upgrading ovirt node properly and I was under the impression that all that was needed was `yum upgrade` and what actually happens is that the new image LV is created, but nothing is switched to boot into it (again if I'm missing a step to switch to the new image it's not documented). That, or 4.1.7's install/upgrade is completely broken.

If it seems like I'm speaking off topic, I am, because I've been trying to reach out on IRC, as this is my first experience with oVirt, and it's a completely dead channel (despite the 400 or so idlers). So, there's clearly active development going on here, where's the community? Mailing list?
Comment 8 Ryan Barry 2017-11-08 11:39:42 EST
users@ovirt.org is a good way to respond.

Unfortunately, IRC is so-so. It's very active in European timezones, and less active in US 'prime time'. Most of us are on a lot of different IRC channels, and may miss conversations in #ovirt before the questioner logs off. Pinging is great if you know who to respond to (I'm there as rbarry, for example), but harder otherwise.

'yum upgrade' does a couple of things. Broadly, Node updates are shipped as a squashfs image inside the RPM, and the RPM injects a new PYTHONPATH to catch any changes in the installer code, then does the following:

mount the new squashfs

create a new LV and sync it with the new squashfs

create any new partitions needed

sync over /etc, /root, /var, and a couple of other paths (this particular problem is due to an assumption that /usr/share/rhn will exist for satellite, which it does not upstream)

create new bootloader entries

You're not missing a step. It's just likely the the upgrade process hit the same exception as the installer -- same code paths. yum should have told you something went wrong, or you can look at /var/log/imgbased.log (or /tmp/imgbased.log)

Note You need to log in before you can comment on or make changes to this bug.