Bug 1528468

Summary: oVirt NGN kickstart-based install fails during imgbased layout initialization after installing an rpm package in %post
Product: [oVirt] ovirt-node Reporter: Giuseppe Ragusa <giuseppe.ragusa>
Component: Installation & UpdateAssignee: Ryan Barry <rbarry>
Status: CLOSED CURRENTRELEASE QA Contact: Qin Yuan <qiyuan>
Severity: high Docs Contact:
Priority: high    
Version: 4.1CC: bugs, cshao, dguo, huzhao, lveyde, qiyuan, rbarry, sbonazzo, yaniwang, ycui, yisong, yzhao
Target Milestone: ovirt-4.2.1Flags: rule-engine: ovirt-4.2+
rbarry: devel_ack+
qiyuan: testing_ack+
Target Release: 4.2   
Hardware: x86_64   
OS: Unspecified   
Whiteboard:
Fixed In Version: imgbased-1.0.6 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-02-12 11:47:20 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Node RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
The test kickstart to exercise the installer crash.
none
Archive with all /tmp/*log files after the crash none

Description Giuseppe Ragusa 2017-12-21 23:37:54 UTC 
Created attachment 1371098 [details]
The test kickstart to exercise the installer crash.

Description of problem:
Installation stops with a Python stack trace in imgbased rpmpersistence phase if in previous %post an rpm package has been installed.

Version-Release number of selected component (if applicable):
4.1.8

How reproducible:
Installing from ISO with the attached sample kickstart crashes when installing a package in %post and completes successfully otherwise.

Steps to Reproduce:
1. Place the test kickstart on a web server
2. Boot from ISO adding on commandline: inst.ks=http:/server/test.ks crashme
3. Wait until crash message

Actual results:
Install crashes if an rpm package has been installed.

Expected results:
Install always completes successfully.

Additional info:
Verified also on 4.1.6 and 4.1.7
The installed rpm package gets correctly added under /var/imgbased/persisted-rpms inside the installed system.
The test kickstart crashes the installation both in autopart and with explicit partitioning mode.
Comment 1 Giuseppe Ragusa 2017-12-21 23:41:47 UTC 
Created attachment 1371099 [details]
Archive with all /tmp/*log files after the crash
Comment 2 Ryan Barry 2017-12-22 01:03:57 UTC 
We get an EAGAIN from the kernel when trying to spawn a container to reinstall the PRMs.

I can't even `systemd-nspawn -b -D /mnt/sysimage bash` inside Anaconda, so well work around this by checking whether or not we're in the middle of Anaconda (even ISO installs have a partial kickstart, so checking for inst.ks is reasonably safe) and emitting a message if so.
Comment 3 Qin Yuan 2018-01-02 15:17:17 UTC 
QE can reproduce this bug using ovirt-node-ng-installer-ovirt-4.1-2017122606.iso
Comment 4 Qin Yuan 2018-01-29 13:20:52 UTC 
Verify Versions:
ovirt-node-ng-installer-ovirt-4.2-pre-2018012909.iso
imgbased-1.0.8-1.el7.centos.noarch

Verify Steps:
1. Create a kickstart file with a %post section as below, and put it on a web server:
%post
yum -y --enablerepo=base install wget
%end
2. Boot from the iso, modify the boot option: inst.ks=http://server/test.ks
3. Wait the installation to finish
4. Log into system, check whether wget is isnalled using `rpm -qa |grep wget`
5. Check content under /var/imgbased/persisted-rpms

Verify Results:
1. The installation completed successfully.
2. wget was installed:
[root@dell-per730-34 ~]# rpm -qa |grep wget
wget-1.14-15.el7.x86_64
3. wget rpm was persisted under /var/imgbased/persisted-rpms:
[root@dell-per730-34 ~]# ls /var/imgbased/persisted-rpms/
wget-1.14-15.el7.x86_64.rpm

According to the above steps and results, the bug is fixed, but there is one concern that:
As the boot option ks= is not discarded yet, if users happen to use ks= to specify the path of kickstart file, then system installation will also fail if a rpm has been installed via `yum install`. 

I wonder which is the better way to solve my concern:
1. Just ignore it, as ks= is not the common use case.(ks= is the default boot option in downstream iso, such as RHVH-4.2-20180128.0-RHVH-x86_64-dvd1.iso)
2. Improve the patch to identify both inst.ks= and ks=
3. Publish a notification saying users must use inst.ks= if they want to use `yum install` in the kickstart file to install rpms.
Comment 5 Ryan Barry 2018-01-29 13:25:57 UTC 
Since ks= is deprecated, I would expect users to use inst.ks instead of inst.ks, but my preference would to to document this instead of changing the patch, since it's very possible that a future kernel argument (or current argument for some vendor module) may use ks=, which would prevent imgbased from persisting RPMs on upgrades.
Comment 6 Qin Yuan 2018-01-30 02:38:37 UTC 
According to comment #5, my concern in comment #4 is clarified, so change the bug status to VERIFIED.
Comment 7 Sandro Bonazzola 2018-02-12 11:47:20 UTC 
This bugzilla is included in oVirt 4.2.1 release, published on Feb 12th 2018.

Since the problem described in this bug report should be
resolved in oVirt 4.2.1 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.