Bug 1802591

Summary: livemedia-creator --make-pxe-live doesn't work
Product: Red Hat Enterprise Linux 7 Reporter: Christophe Besson <cbesson>
Component: loraxAssignee: Brian Lane <bcl>
Status: CLOSED ERRATA QA Contact: Release Test Team <release-test-team-automation>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.7CC: atodorov, jrusz, sujagtap, tbowling
Target Milestone: rcKeywords: Extras, Patch, Reproducer
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: lorax-19.7.27-1 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-09-29 20:23:40 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Christophe Besson 2020-02-13 13:29:42 UTC
Description of problem:
Unable to use the --make-pxe-live feature. It fails after the anaconda step which installs successfully a disk image, but an error occurs while dealing with the loop device, and finally the expected output files are not created (squashfs, vmlinuz, initramfs, syslinux config...).

From livemedia.log:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Running post-installation scripts
.
losetup: /dev/: detach failed: Inappropriate ioctl for device
2020-02-10 09:58:35,470: Disk Image install successful
2020-02-10 09:58:35,472: working dir is /var/tmp/tmpizpZtc
mount: /dev/loop0 is write-protected, mounting read-only
mount: unknown filesystem type '(null)'
2020-02-10 09:58:40,488: Command '['mount', '-o', 'loop', '/var/tmp/diskjS6U6s.img', '/var/tmp/lorax.imgutils.rMMSw1']' returned non-zero exit status 32
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

It's a raw image file containing 2 partitions, one ext4 and one swap.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# file /var/tmp/diskjS6U6s.img
/var/tmp/diskjS6U6s.img: x86 boot sector; partition 1: ID=0x83, active, starthead 32, startsector 2048, 8192000 sectors; partition 2: ID=0x82, starthead 14, startsector 8194048, 2048000 sectors, code offset 0x63
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Indeed, by executing manually the last command done by live-media-creator:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# mount -o loop /var/tmp/diskjS6U6s.img /media
mount: /dev/loop1 is write-protected, mounting read-only
mount: unknown filesystem type '(null)'
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

We would need to a tool like kpartx to be able to mount the rootfs inside this image.


Version-Release number of selected component (if applicable):
lorax-composer-19.7.35-1.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
# livemedia-creator --make-pxe-live --no-virt --ks rhel7-minimal.ks

Actual results:
losetup: /dev/: detach failed: Inappropriate ioctl for device
2020-02-10 09:58:35,470: Disk Image install successful
2020-02-10 09:58:35,472: working dir is /var/tmp/tmpizpZtc
mount: /dev/loop0 is write-protected, mounting read-only
mount: unknown filesystem type '(null)'
2020-02-10 09:58:40,488: Command '['mount', '-o', 'loop', '/var/tmp/diskjS6U6s.img', '/var/tmp/lorax.imgutils.rMMSw1']' returned non-zero exit status 32

Expected results:
Squashfs and additional files are created.

Additional info:
Never used this mode before, I'm not sure about the usage. 
Additionally, customer reports he succeeded to make it working on RHEL 8.x, by adding "dracut-live" to the kickstart, as in this BZ:
https://bugzilla.redhat.com/show_bug.cgi?id=1467506

Comment 2 Christophe Besson 2020-02-13 16:13:35 UTC
I finally figure out why it does not work. It's about the combination options in 
/usr/lib/python2.7/site-packages/pylorax/creator.py:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
587     elif opts.make_pxe_live:
588         work_dir = tempfile.mkdtemp()
589         log.info("working dir is %s", work_dir)
590 
591         if (opts.fs_image or opts.no_virt) and not opts.disk_image:
592             # Create pxe live images from a filesystem image
593             disk_img = opts.fs_image or disk_img
594             with Mount(disk_img, opts="loop") as mnt_dir:
595                 result_dir = make_live_images(opts, work_dir, mnt_dir, rootfs_image=disk_img)
596         else:
597             # Create pxe live images from a partitioned disk image
598             disk_img = opts.disk_image or disk_img
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I don't know what option from the cmdline I missed, but the image generated with --no-virt leads to 
a disk image, which can be mounted with PartitionMount().

The following patch works for this case, I didn't check any possible regression:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# diff -u /usr/lib/python2.7/site-packages/pylorax/creator.py.orig /usr/lib/python2.7/site-packages/pylorax/creator.py
--- /usr/lib/python2.7/site-packages/pylorax/creator.py.orig	2020-02-13 15:46:13.012101563 +0100
+++ /usr/lib/python2.7/site-packages/pylorax/creator.py	2020-02-13 16:03:22.376465871 +0100
@@ -588,7 +588,7 @@
         work_dir = tempfile.mkdtemp()
         log.info("working dir is %s", work_dir)
 
-        elif (opts.fs_image or opts.no_virt) and not opts.disk_image:
+        if opts.fs_image and not opts.disk_image:
             # Create pxe live images from a filesystem image
             disk_img = opts.fs_image or disk_img
             with Mount(disk_img, opts="loop") as mnt_dir:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

It ends with:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
losetup: /dev/: detach failed: Inappropriate ioctl for device
2020-02-13 15:52:39,286: Disk Image install successful
2020-02-13 15:52:39,290: working dir is /var/tmp/tmpo9fFH7
2020-02-13 15:52:44,338: Partition mounted on /var/tmp/tmpUrvAoc size=4194304000
2020-02-13 15:52:44,339: Creating live rootfs image
---[cut]---
2020-02-13 15:55:41,366: Rebuilding initramfs for live
2020-02-13 15:55:41,366: dracut args = ['--xz', '--add', 'livenet dmsquash-live convertfs pollcdrom', '--omit', 'plymouth', '--no-hostonly', '--no-early-microcode']
2020-02-13 15:55:41,367: rebuilding boot/initramfs-3.10.0-1062.el7.x86_64.img.live
No '/dev/log' or 'logger' included for syslog logging
loop deleted : /dev/loop0
2020-02-13 15:56:42,347: SUMMARY
2020-02-13 15:56:42,348: -------
2020-02-13 15:56:42,348: Logs are in /root/pxe4
2020-02-13 15:56:42,349: Disk image is at /var/tmp/disk6Z4Idb.img
2020-02-13 15:56:42,349: Results are in /var/tmp/tmpo9fFH7

# ll /var/tmp/tmpo9fFH7/
total 484736
-rw-r--r--. 1 root root  37331228 Feb 13 15:56 initramfs-3.10.0-1062.el7.x86_64.img
-rw-r--r--. 1 root root 452296704 Feb 13 15:55 live-rootfs.squashfs.img
-rw-r--r--. 1 root root       211 Feb 13 15:56 PXE_CONFIG
-rwxr-xr-x. 1 root root   6730032 Jul 18  2019 vmlinuz-3.10.0-1062.el7.x86_64
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Comment 3 Brian Lane 2020-02-13 18:33:39 UTC
Thanks for the report, it looks like commit efb0cce9b85a157cbf526bc22a7bcc37a87b6500 needs to be backported, the problem is that novirt_install assumes it should make a partitioned disk. It needs to check for pxe_live and switch to a filesystem disk in that case.

It looks like the only way to work around this right now is to either run it with a VM instead of --no-virt, or to do it in two steps:

First create a filesystem image:
 livemedia-creator --make-fsimage --image-name=pxeboot.img --no-virt --ks ./rhel7-minimal.ks

And then use that image to make the pxe image:
livemedia-creator --make-pxe-live --fs-image=/var/tmp/pxeboot.img --ks ./rhel7-minimal.ks

I also added dracut-network to the kickstart to make sure the livenet module was installed.

The build process works for me in a RHEL 7.8 VM, but I don't currently have a PXE setup I can boot it with so I can't be 100% sure it works.

Comment 5 Brian Lane 2020-04-16 17:18:44 UTC
https://github.com/weldr/lorax/pull/992

Comment 11 errata-xmlrpc 2020-09-29 20:23:40 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (lorax bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:3989