Bug 1842484 - RHV-H "liveimg" breaks %post chroot [NEEDINFO]
Summary: RHV-H "liveimg" breaks %post chroot
Keywords:
Status: ASSIGNED
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: redhat-virtualization-host
Version: 4.3.0
Hardware: Unspecified
OS: Unspecified
unspecified
low
Target Milestone: ovirt-4.4.7
: ---
Assignee: Lev Veyde
QA Contact: cshao
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-06-01 11:32 UTC by Lukas Zapletal
Modified: 2021-04-22 07:47 UTC (History)
13 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-08-04 07:42:55 UTC
oVirt Team: Node
Target Upstream Version:
nlevy: needinfo? (mtessun)


Attachments (Terms of Use)
RHVH QE logs (1.25 MB, application/gzip)
2020-12-03 14:19 UTC, cshao
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 5603731 0 None None None 2020-11-26 10:27:09 UTC

Description Lukas Zapletal 2020-06-01 11:32:19 UTC
Hello, when installing RHV-H, anything that writes to the filesystem in %post script is erased after reboot.

Reproducer (any version of RHV-H or RHEL):

1) Create a kickstart with liveimg statement
2) Write a %post script: touch /test
3) Install and reboot

File /test is not present on the installed system.

Analysis:

It looks like the system is not written into /mnt/sysimage but a directory mounted under /mnt/sysimage/tmp/mnt*.

Comment 2 Jiri Konecny 2020-06-18 15:58:54 UTC
Hello,

I'm not able to reproduce this issue. Could you please try it on RHEL-8.3 nightly and provide us logs if the issue is still present there?

You are able to get logs from /tmp/*.log or /var/log/anaconda/*.log on the installed system.

Comment 3 Lukas Zapletal 2020-06-19 12:08:46 UTC
The customer is trying this with RHV 4.3 which is RHEL 7.8 based. Note the version I've filed this BZ against.

We will collect required logs.

Comment 9 Sandro Bonazzola 2020-08-04 07:42:55 UTC
I'm closing with insufficient data.
If you can provide the failing kickstart please reopen the bug.

Comment 11 Juan Orti 2020-11-26 10:16:58 UTC
Reopening. Easily testable with a simple kickstart file:

~~~
liveimg --url=http://bootserver.lan/redhat-virtualization-host-4.4.3-20201116.0.el8_3.squashfs.img
clearpart --all
autopart --type=thinp
rootpw --plaintext 1234
timezone --utc Europe/Madrid
keyboard es
zerombr
text

reboot

%post --log=/root/ks.post.log --erroronfail
nodectl init
echo "Hello World" > /root/foo
%end
~~~

I wonder if this problem is also causing the missing /root/anaconda-ks.cfg file.

Comment 12 cshao 2020-11-26 13:50:39 UTC
RHVH QE can reproduce this issue.

Comment 14 cshao 2020-12-03 14:19:19 UTC
Created attachment 1736104 [details]
RHVH QE logs

Comment 15 Nir Levy 2021-01-13 15:18:32 UTC
The right way to do it and the explanation on why this is the case

nodectl wrappes call to imgbased which is the install/upgrade package for rhvh/node-ng

imgbased responsible for getting the squashfs inside the liveimg and create the logical volume for it.
during the install the new layer is created, mounted and unmounted 
so after nodectl init the device is not mounted any longer.

so in order to modify the content of that layer mount should be done,

lvm layout consist something similar to 
ovirt-node-ng-4.4.4-0.20201221.0
 +- ovirt-node-ng-4.4.4-0.20201221.0+1

where the +1 layer is a r/w snapshot on top of the base layer.
we should mount that layer and than proceed as follows.
nsenter on opos to accessing /tmp/rw_layer/root/foo enables us to run executable commands.


Here is a full example:


%post --log=/root/ks.post.log --erroronfail
nodectl init

mkdir /tmp/rw_layer
layer=/dev/$(lvs -ofullname | grep -o "[^ ].*+1")
mount -onouuid "$layer" /tmp/rw_layer
nsenter --root=/tmp/rw_layer echo "Hello World" > /root/foo
umount /tmp/rw_layer

echo "Hello World" > /root/foo
%end


please see if you need anything else

Comment 16 Lukas Zapletal 2021-01-14 07:23:39 UTC
Thanks, is this something you guys are able to document or there are no intentions to support this for end customers? It looks fragile to be honest.

Comment 17 Nir Levy 2021-01-14 08:04:06 UTC
The question here is what do you expect to get from such actions,

If that is something you wish to execute on massive amount and you want some run-able file to be presence,
In such case I can raise this to our PM and find the right way to achieve that.

Comment 18 Sandro Bonazzola 2021-01-18 12:53:19 UTC
We need to understand why this is needed in order to decide if it's worth to support.

Comment 24 Lukas Zapletal 2021-01-21 09:27:01 UTC
I see.

Well then on behalf of RH Satellite team, I am raising a RFE - create some tool/command that enables us to write to disk during %post section. Red Hat Satellite cannot use Anaconda to configure network and subscription manager due to several technical limitations and we have to stick with %post. Requirements:

1) Ability to enter (chroot) environment where all files can be modified as root.
2) Ability to execute arbitrary amount of commands (not just a single command - our snippets are lengthy).
3) Ability to exit and commit the changes made on the disk.
4) Make this command stable, have QA plan to prevent changes and regressions.

Pseudo-example:

%post --log=/root/ks.post.log --erroronfail

nodectl init

rfe_command start

a_satellite_command
a_satellite_command
a_satellite_command

rfe_command end

%end

If you think that mount/nsenter is the way to go, just confirm and prepare some QA plan so we can rely on this, but it feels a little bit clunky.

Q: Has RHV-H installation been like that? We currently render the snippets normally, without any mount/nsenter commands, and this passed our QA which is weird.

Comment 28 Lukas Zapletal 2021-02-04 13:29:53 UTC
Q: Is it possible to issue customization BEFORE the nodectl command? Like:

%post

my_own_customization_commands

nodectl init
%end


Note You need to log in before you can comment on or make changes to this bug.