Bug 2051332
| Summary: | supermin should ignore +debug kernels when choosing a kernel to boot | |||
|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 8 | Reporter: | mhou <mhou> | |
| Component: | supermin | Assignee: | Richard W.M. Jones <rjones> | |
| Status: | CLOSED ERRATA | QA Contact: | YongkuiGuo <yoguo> | |
| Severity: | high | Docs Contact: | ||
| Priority: | unspecified | |||
| Version: | 8.2 | CC: | bhu, jlelli, kkiwi, mhou, mtosatti, nilal, ppandit, rjones, tzheng, virt-maint, yoguo | |
| Target Milestone: | rc | Keywords: | Triaged, ZStream | |
| Target Release: | 8.7 | Flags: | pm-rhel:
mirror+
|
|
| Hardware: | x86_64 | |||
| OS: | Linux | |||
| Whiteboard: | ||||
| Fixed In Version: | supermin-5.2.1-2.el8 | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | ||
| Clone Of: | ||||
| : | 2059397 (view as bug list) | Environment: | ||
| Last Closed: | 2022-11-08 09:18:38 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | ||||
| Bug Blocks: | 1932086 | |||
|
Comment 2
Richard W.M. Jones
2022-02-07 09:26:48 UTC
Yes, I still meet this issue. Here is the package version: kernel: 4.18.0-193.71.1.rt13.121.el8_2.x86_64 qemu-kvm: qemu-kvm-4.2.0-29.module+el8.2.1+7297+a825794d.x86_64 seabios-bin: seabios-bin-1.13.0-2.module+el8.2.1+7284+aa32a2c4.noarch libguestfs: libguestfs-1.40.2-24.module+el8.2.1+7154+47ffd890.x86_64 libvirt: libvirt-daemon-driver-storage-iscsi-direct-6.0.0-25.module+el8.2.1+7154+47ffd890.x86_64 libvirt-daemon-driver-nwfilter-6.0.0-25.module+el8.2.1+7154+47ffd890.x86_64 libvirt-daemon-driver-storage-disk-6.0.0-25.module+el8.2.1+7154+47ffd890.x86_64 libvirt-daemon-driver-storage-6.0.0-25.module+el8.2.1+7154+47ffd890.x86_64 libvirt-daemon-driver-storage-mpath-6.0.0-25.module+el8.2.1+7154+47ffd890.x86_64 libvirt-daemon-6.0.0-25.module+el8.2.1+7154+47ffd890.x86_64 libvirt-daemon-driver-interface-6.0.0-25.module+el8.2.1+7154+47ffd890.x86_64 libvirt-daemon-driver-secret-6.0.0-25.module+el8.2.1+7154+47ffd890.x86_64 libvirt-daemon-driver-network-6.0.0-25.module+el8.2.1+7154+47ffd890.x86_64 libvirt-daemon-kvm-6.0.0-25.module+el8.2.1+7154+47ffd890.x86_64 libvirt-client-6.0.0-25.module+el8.2.1+7154+47ffd890.x86_64 libvirt-daemon-driver-storage-iscsi-6.0.0-25.module+el8.2.1+7154+47ffd890.x86_64 libvirt-libs-6.0.0-25.module+el8.2.1+7154+47ffd890.x86_64 libvirt-daemon-driver-storage-gluster-6.0.0-25.module+el8.2.1+7154+47ffd890.x86_64 libvirt-daemon-driver-storage-rbd-6.0.0-25.module+el8.2.1+7154+47ffd890.x86_64 libvirt-daemon-driver-nodedev-6.0.0-25.module+el8.2.1+7154+47ffd890.x86_64 libvirt-bash-completion-6.0.0-25.module+el8.2.1+7154+47ffd890.x86_64 libvirt-daemon-config-network-6.0.0-25.module+el8.2.1+7154+47ffd890.x86_64 libvirt-daemon-driver-storage-scsi-6.0.0-25.module+el8.2.1+7154+47ffd890.x86_64 libvirt-daemon-driver-storage-logical-6.0.0-25.module+el8.2.1+7154+47ffd890.x86_64 libvirt-daemon-driver-qemu-6.0.0-25.module+el8.2.1+7154+47ffd890.x86_64 libvirt-daemon-driver-storage-core-6.0.0-25.module+el8.2.1+7154+47ffd890.x86_64 > kernel: 4.18.0-193.71.1.rt13.121.el8_2.x86_64
It's probably going to be a problem with the real time kernel. I would
guess if you use the non-RT kernel it'll be OK.
As for exactly why is going to be a bit of a mystery, but the RT kernels
aren't designed for virtualization use AIUI.
Hello Richard You are right. Actually, If I use the corresponding stock kernel, this issue was disappeared. From a testing standpoint, this issue belongs to a released version. If there is no customer-related bug involved in this issue, I don't think this issue will affect my test. Or whether to allow RT-related development to assess the extent of the issue's impact？ I'm not sure to what extent we support libguestfs on the RT kernel. I *think* we've had this issue come up in the past, but my bugzilla skills are not allowing me to find any bugs at the moment. We might have Virt customers using the RT kernel on 8.2, so it might be a good idea to debug this. @Minxi, are you using CPU isolation? That is, can you reproduce this simply by installing the RT kernel and advanced Virt? Or do you install tuned and configure the nfv-guest/host profiles as well? @Minxi or @Richard, do we know what the error message is? What is actually failing? Hello Luiz This issue is very easy to reproduce. Don't do any CPU isolation. Just install RT kernel and advanced Virt, then run virt-customize can reproduce this issue. when I want to change a configuration on a guest image(such as change login password, copying a file from host to a guest image, or run some command on a guest image, not in a running VM), virt-customize is a good choice. On this issue, virt-customize can't use as well on RT kernel, so I can't modify some configuration on the guest image. (In reply to Richard W.M. Jones from comment #6) > I'm not sure to what extent we support libguestfs on the RT kernel. I > *think* > we've had this issue come up in the past, but my bugzilla skills are not > allowing > me to find any bugs at the moment. This one: https://bugzilla.redhat.com/show_bug.cgi?id=1795973 I can 'not' reproduce this issue. Steps: 1. Reserve a bare metal from beaker with distro 'RHEL-8.2.1-updates-20200901.0' 2. Install libguestfs related packages from the repo: http://download.eng.pek2.redhat.com/rhel-8/rel-eng/ADVANCED-VIRT-8/latest-ADVANCED-VIRT-8.2.1-RHEL-8/compose/Advanced-virt/x86_64/os 3. Install the latest kernel-rt and kernel-rt-debug packages from the reppo: http://download.eng.pek2.redhat.com/rhel-8/nightly/updates/RHEL-8/RHEL-8.2.1-20220207.n.0/compose/RT/x86_64/os 4. # rpm -q libguestfs qemu-kvm libvirt-daemon kernel-rt kernel-rt-debug seabios-bin libguestfs-1.40.2-24.module+el8.2.1+7154+47ffd890.x86_64 qemu-kvm-4.2.0-29.module+el8.2.1+7297+a825794d.x86_64 libvirt-daemon-6.0.0-25.module+el8.2.1+7154+47ffd890.x86_64 kernel-rt-4.18.0-193.71.1.rt13.121.el8_2.x86_64 kernel-rt-debug-4.18.0-193.71.1.rt13.121.el8_2.x86_64 seabios-bin-1.13.0-2.module+el8.2.1+7284+aa32a2c4.noarch 5. libguestfs-test-tool works well # libguestfs-test-tool ************************************************************ * IMPORTANT NOTICE * * When reporting bugs, include the COMPLETE, UNEDITED * output below in your bug report. * ************************************************************ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin XDG_RUNTIME_DIR=/run/user/0 SELinux: Enforcing guestfs_get_append: (null) guestfs_get_autosync: 1 guestfs_get_backend: libvirt guestfs_get_backend_settings: [] guestfs_get_cachedir: /var/tmp guestfs_get_hv: /usr/libexec/qemu-kvm guestfs_get_memsize: 768 guestfs_get_network: 0 guestfs_get_path: /usr/lib64/guestfs guestfs_get_pgroup: 0 guestfs_get_program: libguestfs-test-tool guestfs_get_recovery_proc: 1 guestfs_get_smp: 1 guestfs_get_sockdir: /tmp guestfs_get_tmpdir: /tmp guestfs_get_trace: 0 guestfs_get_verbose: 1 host_cpu: x86_64 Launching appliance, timeout set to 600 seconds. libguestfs: launch: program=libguestfs-test-tool libguestfs: launch: version=1.40.2rhel=8,release=24.module+el8.2.1+7154+47ffd890,libvirt libguestfs: launch: backend registered: unix libguestfs: launch: backend registered: uml libguestfs: launch: backend registered: libvirt libguestfs: launch: backend registered: direct libguestfs: launch: backend=libvirt libguestfs: launch: tmpdir=/tmp/libguestfswVcPkr libguestfs: launch: umask=0022 libguestfs: launch: euid=0 libguestfs: libvirt version = 6000000 (6.0.0) libguestfs: guest random name = guestfs-ixhmtokg4zekbgnn libguestfs: connect to libvirt libguestfs: opening libvirt handle: URI = qemu:///system, auth = default+wrapper, flags = 0 libguestfs: successfully opened libvirt handle: conn = 0x5621e50817d0 libguestfs: qemu version (reported by libvirt) = 4002000 (4.2.0) libguestfs: get libvirt capabilities libguestfs: parsing capabilities XML libguestfs: build appliance libguestfs: begin building supermin appliance libguestfs: run supermin libguestfs: command: run: /usr/bin/supermin libguestfs: command: run: \ --build libguestfs: command: run: \ --verbose libguestfs: command: run: \ --if-newer libguestfs: command: run: \ --lock /var/tmp/.guestfs-0/lock libguestfs: command: run: \ --copy-kernel libguestfs: command: run: \ -f ext2 libguestfs: command: run: \ --host-cpu x86_64 libguestfs: command: run: \ /usr/lib64/guestfs/supermin.d libguestfs: command: run: \ -o /var/tmp/.guestfs-0/appliance.d supermin: version: 5.1.19 supermin: rpm: detected RPM version 4.14 supermin: package handler: fedora/rpm supermin: acquiring lock on /var/tmp/.guestfs-0/lock supermin: build: /usr/lib64/guestfs/supermin.d supermin: reading the supermin appliance supermin: build: visiting /usr/lib64/guestfs/supermin.d/base.tar.gz type gzip base image (tar) supermin: build: visiting /usr/lib64/guestfs/supermin.d/daemon.tar.gz type gzip base image (tar) supermin: build: visiting /usr/lib64/guestfs/supermin.d/excludefiles type uncompressed excludefiles supermin: build: visiting /usr/lib64/guestfs/supermin.d/hostfiles type uncompressed hostfiles supermin: build: visiting /usr/lib64/guestfs/supermin.d/init.tar.gz type gzip base image (tar) supermin: build: visiting /usr/lib64/guestfs/supermin.d/packages type uncompressed packages supermin: build: visiting /usr/lib64/guestfs/supermin.d/udev-rules.tar.gz type gzip base image (tar) supermin: build: visiting /usr/lib64/guestfs/supermin.d/zz-packages-xfs type uncompressed packages supermin: mapping package names to installed packages supermin: resolving full list of package dependencies supermin: build: 184 packages, including dependencies supermin: build: 36481 files supermin: build: 10211 files, after matching excludefiles supermin: build: 10225 files, after adding hostfiles supermin: build: 10211 files, after removing unreadable files supermin: build: 10246 files, after munging supermin: kernel: looking for kernel using environment variables ... supermin: kernel: looking for kernels in /lib/modules/*/vmlinuz ... supermin: kernel: picked vmlinuz /lib/modules/4.18.0-193.71.1.rt13.121.el8_2.x86_64+debug/vmlinuz supermin: kernel: kernel_version 4.18.0-193.71.1.rt13.121.el8_2.x86_64+debug supermin: kernel: modpath /lib/modules/4.18.0-193.71.1.rt13.121.el8_2.x86_64+debug supermin: ext2: creating empty ext2 filesystem '/var/tmp/.guestfs-0/appliance.d.andl9r3n/root' supermin: ext2: populating from base image supermin: ext2: copying files from host filesystem supermin: ext2: copying kernel modules supermin: ext2: creating minimal initrd '/var/tmp/.guestfs-0/appliance.d.andl9r3n/initrd' supermin: ext2: wrote 28 modules to minimal initrd supermin: renaming /var/tmp/.guestfs-0/appliance.d.andl9r3n to /var/tmp/.guestfs-0/appliance.d libguestfs: finished building supermin appliance libguestfs: command: run: qemu-img libguestfs: command: run: \ create libguestfs: command: run: \ -f qcow2 libguestfs: command: run: \ -o backing_file=/var/tmp/.guestfs-0/appliance.d/root,backing_fmt=raw libguestfs: command: run: \ /tmp/libguestfswVcPkr/overlay2.qcow2 Formatting '/tmp/libguestfswVcPkr/overlay2.qcow2', fmt=qcow2 size=4294967296 backing_file=/var/tmp/.guestfs-0/appliance.d/root backing_fmt=raw cluster_size=65536 lazy_refcounts=off refcount_bits=16 libguestfs: create libvirt XML libguestfs: libvirt XML:\n<?xml version="1.0"?>\n<domain type="kvm" xmlns:qemu="http://libvirt.org/schemas/domain/qemu/1.0">\n <name>guestfs-ixhmtokg4zekbgnn</name>\n <memory unit="MiB">768</memory>\n <currentMemory unit="MiB">768</currentMemory>\n <cpu mode="host-passthrough">\n <model fallback="allow"/>\n </cpu>\n <vcpu>1</vcpu>\n <clock offset="utc">\n <timer name="rtc" tickpolicy="catchup"/>\n <timer name="pit" tickpolicy="delay"/>\n <timer name="hpet" present="no"/>\n </clock>\n <os>\n <type>hvm</type>\n <kernel>/var/tmp/.guestfs-0/appliance.d/kernel</kernel>\n <initrd>/var/tmp/.guestfs-0/appliance.d/initrd</initrd>\n <cmdline>panic=1 console=ttyS0 edd=off udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=/dev/sdb selinux=0 guestfs_verbose=1 TERM=xterm-256color</cmdline>\n <bios useserial="yes"/>\n </os>\n <on_reboot>destroy</on_reboot>\n <devices>\n <rng model="virtio">\n <backend model="random">/dev/urandom</backend>\n </rng>\n <controller type="scsi" index="0" model="virtio-scsi"/>\n <disk device="disk" type="file">\n <source file="/tmp/libguestfswVcPkr/scratch1.img"/>\n <target dev="sda" bus="scsi"/>\n <driver name="qemu" type="raw" cache="unsafe"/>\n <address type="drive" controller="0" bus="0" target="0" unit="0"/>\n </disk>\n <disk type="file" device="disk">\n <source file="/tmp/libguestfswVcPkr/overlay2.qcow2"/>\n <target dev="sdb" bus="scsi"/>\n <driver name="qemu" type="qcow2" cache="unsafe"/>\n <address type="drive" controller="0" bus="0" target="1" unit="0"/>\n </disk>\n <serial type="unix">\n <source mode="connect" path="/tmp/libguestfsyC9CEZ/console.sock"/>\n <target port="0"/>\n </serial>\n <channel type="unix">\n <source mode="connect" path="/tmp/libguestfsyC9CEZ/guestfsd.sock"/>\n <target type="virtio" name="org.libguestfs.channel.0"/>\n </channel>\n <controller type="usb" model="none"/>\n <memballoon model="none"/>\n </devices>\n <qemu:commandline>\n <qemu:env name="TMPDIR" value="/var/tmp"/>\n </qemu:commandline>\n</domain>\n libguestfs: command: run: ls libguestfs: command: run: \ -a libguestfs: command: run: \ -l libguestfs: command: run: \ -R libguestfs: command: run: \ -Z /var/tmp/.guestfs-0 libguestfs: /var/tmp/.guestfs-0: libguestfs: total 0 libguestfs: drwxr-xr-x. 3 root root unconfined_u:object_r:user_tmp_t:s0 37 Feb 8 02:00 . libguestfs: drwxrwxrwt. 6 root root system_u:object_r:tmp_t:s0 200 Feb 8 02:00 .. libguestfs: drwxr-xr-x. 2 root root unconfined_u:object_r:user_tmp_t:s0 46 Feb 8 02:00 appliance.d libguestfs: -rw-r--r--. 1 root root unconfined_u:object_r:user_tmp_t:s0 0 Feb 8 02:00 lock libguestfs: libguestfs: /var/tmp/.guestfs-0/appliance.d: libguestfs: total 390148 libguestfs: drwxr-xr-x. 2 root root unconfined_u:object_r:user_tmp_t:s0 46 Feb 8 02:00 . libguestfs: drwxr-xr-x. 3 root root unconfined_u:object_r:user_tmp_t:s0 37 Feb 8 02:00 .. libguestfs: -rw-r--r--. 1 root root unconfined_u:object_r:user_tmp_t:s0 10731520 Feb 8 02:00 initrd libguestfs: -rwxr-xr-x. 1 root root unconfined_u:object_r:user_tmp_t:s0 20286832 Feb 8 02:00 kernel libguestfs: -rw-r--r--. 1 root root unconfined_u:object_r:user_tmp_t:s0 4294967296 Feb 8 02:00 root libguestfs: command: run: ls libguestfs: command: run: \ -a libguestfs: command: run: \ -l libguestfs: command: run: \ -Z /tmp/libguestfsyC9CEZ libguestfs: total 4 libguestfs: drwxr-xr-x. 2 root root unconfined_u:object_r:user_tmp_t:s0 47 Feb 8 02:00 . libguestfs: drwxrwxrwt. 11 root root system_u:object_r:tmp_t:s0 4096 Feb 8 02:00 .. libguestfs: srw-rw----. 1 root qemu unconfined_u:object_r:user_tmp_t:s0 0 Feb 8 02:00 console.sock libguestfs: srw-rw----. 1 root qemu unconfined_u:object_r:user_tmp_t:s0 0 Feb 8 02:00 guestfsd.sock libguestfs: launch libvirt guest libguestfs: responding to serial console Device Status Report \x1b[1;256r\x1b[256;256H\x1b[6n Google, Inc. Serial Graphics Adapter 11/20/19 SGABIOS $Id$ (mockbuild@) Wed Nov 20 15:03:34 UTC 2019 Term: 80x24 4 0 SeaBIOS (version 1.13.0-2.module+el8.2.1+7284+aa32a2c4) Machine UUID c3fb41ef-8b18-456f-8eb6-d0af15668c6f Booting from ROM... \x1b[2J[ 0.000000] Linux version 4.18.0-193.71.1.rt13.121.el8_2.x86_64+debug (mockbuild.eng.bos.redhat.com) (gcc version 8.3.1 20191121 (Red Hat 8.3.1-5) (GCC)) #1 SMP PREEMPT RT Tue Dec 7 12:54:12 EST 2021 [ 0.000000] Command line: panic=1 console=ttyS0 edd=off udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=/dev/sdb selinux=0 guestfs_verbose=1 TERM=xterm-256color [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] BIOS-provided physical RAM map: ... ===== TEST FINISHED OK ===== 6. virt-customize also works # virt-customize -v -x -a rhel-guest-image-8.2-501.x86_64.qcow2 --root-password password:"redhat" [ 0.0] Examining the guest ... libguestfs: trace: set_verbose true libguestfs: trace: set_verbose = 0 libguestfs: trace: set_network true libguestfs: trace: set_network = 0 libguestfs: trace: add_drive "rhel-guest-image-8.2-501.x86_64.qcow2" "readonly:false" "protocol:file" "discard:besteffort" libguestfs: trace: add_drive = 0 libguestfs: trace: launch libguestfs: trace: max_disks libguestfs: trace: max_disks = 255 libguestfs: trace: get_tmpdir libguestfs: trace: get_tmpdir = "/tmp" libguestfs: trace: version libguestfs: trace: version = <struct guestfs_version = major: 1, minor: 40, release: 2, extra: rhel=8,release=24.module+el8.2.1+7154+47ffd890,libvirt, > libguestfs: trace: get_backend libguestfs: trace: get_backend = "libvirt" libguestfs: launch: program=virt-customize libguestfs: launch: version=1.40.2rhel=8,release=24.module+el8.2.1+7154+47ffd890,libvirt libguestfs: launch: backend registered: unix libguestfs: launch: backend registered: uml libguestfs: launch: backend registered: libvirt libguestfs: launch: backend registered: direct libguestfs: launch: backend=libvirt libguestfs: launch: tmpdir=/tmp/libguestfsN3VRtB libguestfs: launch: umask=0022 libguestfs: launch: euid=0 libguestfs: libvirt version = 6000000 (6.0.0) libguestfs: guest random name = guestfs-p2siu85ez955d50s libguestfs: connect to libvirt libguestfs: opening libvirt handle: URI = qemu:///system, auth = default+wrapper, flags = 0 libguestfs: successfully opened libvirt handle: conn = 0x560c3d4c3cc0 libguestfs: qemu version (reported by libvirt) = 4002000 (4.2.0) libguestfs: get libvirt capabilities libguestfs: parsing capabilities XML libguestfs: trace: get_backend_setting "force_tcg" libguestfs: trace: get_backend_setting = NULL (error) libguestfs: trace: get_backend_setting "internal_libvirt_label" libguestfs: trace: get_backend_setting = NULL (error) libguestfs: trace: get_backend_setting "internal_libvirt_imagelabel" libguestfs: trace: get_backend_setting = NULL (error) libguestfs: trace: get_backend_setting "internal_libvirt_norelabel_disks" libguestfs: trace: get_backend_setting = NULL (error) libguestfs: build appliance libguestfs: trace: get_cachedir libguestfs: trace: get_cachedir = "/var/tmp" libguestfs: begin building supermin appliance libguestfs: run supermin libguestfs: command: run: /usr/bin/supermin libguestfs: command: run: \ --build libguestfs: command: run: \ --verbose libguestfs: command: run: \ --if-newer libguestfs: command: run: \ --lock /var/tmp/.guestfs-0/lock libguestfs: command: run: \ --copy-kernel libguestfs: command: run: \ -f ext2 libguestfs: command: run: \ --host-cpu x86_64 libguestfs: command: run: \ /usr/lib64/guestfs/supermin.d libguestfs: command: run: \ -o /var/tmp/.guestfs-0/appliance.d supermin: version: 5.1.19 supermin: rpm: detected RPM version 4.14 supermin: package handler: fedora/rpm supermin: acquiring lock on /var/tmp/.guestfs-0/lock supermin: build: /usr/lib64/guestfs/supermin.d supermin: reading the supermin appliance supermin: build: visiting /usr/lib64/guestfs/supermin.d/base.tar.gz type gzip base image (tar) supermin: build: visiting /usr/lib64/guestfs/supermin.d/daemon.tar.gz type gzip base image (tar) supermin: build: visiting /usr/lib64/guestfs/supermin.d/excludefiles type uncompressed excludefiles supermin: build: visiting /usr/lib64/guestfs/supermin.d/hostfiles type uncompressed hostfiles supermin: build: visiting /usr/lib64/guestfs/supermin.d/init.tar.gz type gzip base image (tar) supermin: build: visiting /usr/lib64/guestfs/supermin.d/packages type uncompressed packages supermin: build: visiting /usr/lib64/guestfs/supermin.d/udev-rules.tar.gz type gzip base image (tar) supermin: build: visiting /usr/lib64/guestfs/supermin.d/zz-packages-xfs type uncompressed packages supermin: mapping package names to installed packages supermin: resolving full list of package dependencies supermin: build: 184 packages, including dependencies supermin: build: 36481 files supermin: build: 10211 files, after matching excludefiles supermin: build: 10225 files, after adding hostfiles supermin: build: 10211 files, after removing unreadable files supermin: build: 10246 files, after munging supermin: kernel: looking for kernel using environment variables ... supermin: kernel: looking for kernels in /lib/modules/*/vmlinuz ... supermin: kernel: picked vmlinuz /lib/modules/4.18.0-193.71.1.rt13.121.el8_2.x86_64+debug/vmlinuz supermin: kernel: kernel_version 4.18.0-193.71.1.rt13.121.el8_2.x86_64+debug supermin: kernel: modpath /lib/modules/4.18.0-193.71.1.rt13.121.el8_2.x86_64+debug supermin: ext2: creating empty ext2 filesystem '/var/tmp/.guestfs-0/appliance.d.fstt5zko/root' supermin: ext2: populating from base image supermin: ext2: copying files from host filesystem supermin: ext2: copying kernel modules supermin: ext2: creating minimal initrd '/var/tmp/.guestfs-0/appliance.d.fstt5zko/initrd' supermin: ext2: wrote 28 modules to minimal initrd supermin: renaming /var/tmp/.guestfs-0/appliance.d.fstt5zko to /var/tmp/.guestfs-0/appliance.d libguestfs: finished building supermin appliance libguestfs: trace: disk_create "/tmp/libguestfsN3VRtB/overlay1.qcow2" "qcow2" -1 "backingfile:/var/tmp/.guestfs-0/appliance.d/root" "backingformat:raw" libguestfs: command: run: qemu-img libguestfs: command: run: \ create libguestfs: command: run: \ -f qcow2 libguestfs: command: run: \ -o backing_file=/var/tmp/.guestfs-0/appliance.d/root,backing_fmt=raw libguestfs: command: run: \ /tmp/libguestfsN3VRtB/overlay1.qcow2 Formatting '/tmp/libguestfsN3VRtB/overlay1.qcow2', fmt=qcow2 size=4294967296 backing_file=/var/tmp/.guestfs-0/appliance.d/root backing_fmt=raw cluster_size=65536 lazy_refcounts=off refcount_bits=16 libguestfs: trace: disk_create = 0 libguestfs: trace: get_sockdir libguestfs: trace: get_sockdir = "/tmp" libguestfs: create libvirt XML libguestfs: trace: disk_format "rhel-guest-image-8.2-501.x86_64.qcow2" libguestfs: command: run: qemu-img --help | grep -sqE -- '\binfo\b.*-U\b' libguestfs: command: run: qemu-img libguestfs: command: run: \ info libguestfs: command: run: \ -U libguestfs: command: run: \ --output json libguestfs: command: run: \ ./rhel-guest-image-8.2-501.x86_64.qcow2 libguestfs: parse_json: qemu-img info JSON output:\n{\n "virtual-size": 10737418240,\n "filename": "./rhel-guest-image-8.2-501.x86_64.qcow2",\n "cluster-size": 65536,\n "format": "qcow2",\n "actual-size": 1199112192,\n "format-specific": {\n "type": "qcow2",\n "data": {\n "compat": "0.10",\n "refcount-bits": 16\n }\n },\n "dirty-flag": false\n}\n\n libguestfs: trace: disk_format = "qcow2" libguestfs: trace: get_cachedir libguestfs: trace: get_cachedir = "/var/tmp" libguestfs: libvirt XML:\n<?xml version="1.0"?>\n<domain type="kvm" xmlns:qemu="http://libvirt.org/schemas/domain/qemu/1.0">\n <name>guestfs-p2siu85ez955d50s</name>\n <memory unit="MiB">768</memory>\n <currentMemory unit="MiB">768</currentMemory>\n <cpu mode="host-passthrough">\n <model fallback="allow"/>\n </cpu>\n <vcpu>1</vcpu>\n <clock offset="utc">\n <timer name="rtc" tickpolicy="catchup"/>\n <timer name="pit" tickpolicy="delay"/>\n <timer name="hpet" present="no"/>\n </clock>\n <os>\n <type>hvm</type>\n <kernel>/var/tmp/.guestfs-0/appliance.d/kernel</kernel>\n <initrd>/var/tmp/.guestfs-0/appliance.d/initrd</initrd>\n <cmdline>panic=1 console=ttyS0 edd=off udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=/dev/sdb selinux=0 guestfs_verbose=1 guestfs_network=1 TERM=xterm-256color</cmdline>\n <bios useserial="yes"/>\n </os>\n <on_reboot>destroy</on_reboot>\n <devices>\n <rng model="virtio">\n <backend model="random">/dev/urandom</backend>\n </rng>\n <controller type="scsi" index="0" model="virtio-scsi"/>\n <disk device="disk" type="file">\n <source file="/home/rhel-guest-image-8.2-501.x86_64.qcow2"/>\n <target dev="sda" bus="scsi"/>\n <driver name="qemu" type="qcow2" cache="writeback" discard="unmap"/>\n <address type="drive" controller="0" bus="0" target="0" unit="0"/>\n </disk>\n <disk type="file" device="disk">\n <source file="/tmp/libguestfsN3VRtB/overlay1.qcow2"/>\n <target dev="sdb" bus="scsi"/>\n <driver name="qemu" type="qcow2" cache="unsafe"/>\n <address type="drive" controller="0" bus="0" target="1" unit="0"/>\n </disk>\n <serial type="unix">\n <source mode="connect" path="/tmp/libguestfs3WAdBH/console.sock"/>\n <target port="0"/>\n </serial>\n <channel type="unix">\n <source mode="connect" path="/tmp/libguestfs3WAdBH/guestfsd.sock"/>\n <target type="virtio" name="org.libguestfs.channel.0"/>\n </channel>\n <controller type="usb" model="none"/>\n <memballoon model="none"/>\n </devices>\n <qemu:commandline>\n <qemu:env name="TMPDIR" value="/var/tmp"/>\n <qemu:arg value="-netdev"/>\n <qemu:arg value="user,id=usernet,net=169.254.0.0/16"/>\n <qemu:arg value="-device"/>\n <qemu:arg value="virtio-net-pci,netdev=usernet"/>\n </qemu:commandline>\n</domain>\n libguestfs: trace: get_cachedir libguestfs: trace: get_cachedir = "/var/tmp" libguestfs: command: run: ls libguestfs: command: run: \ -a libguestfs: command: run: \ -l libguestfs: command: run: \ -R libguestfs: command: run: \ -Z /var/tmp/.guestfs-0 libguestfs: /var/tmp/.guestfs-0: libguestfs: total 0 libguestfs: drwxr-xr-x. 3 root root unconfined_u:object_r:user_tmp_t:s0 37 Feb 8 02:04 . libguestfs: drwxrwxrwt. 6 root root system_u:object_r:tmp_t:s0 200 Feb 8 02:04 .. libguestfs: drwxr-xr-x. 2 root root unconfined_u:object_r:user_tmp_t:s0 46 Feb 8 02:04 appliance.d libguestfs: -rw-r--r--. 1 root root unconfined_u:object_r:user_tmp_t:s0 0 Feb 8 02:04 lock libguestfs: libguestfs: /var/tmp/.guestfs-0/appliance.d: libguestfs: total 390148 libguestfs: drwxr-xr-x. 2 root root unconfined_u:object_r:user_tmp_t:s0 46 Feb 8 02:04 . libguestfs: drwxr-xr-x. 3 root root unconfined_u:object_r:user_tmp_t:s0 37 Feb 8 02:04 .. libguestfs: -rw-r--r--. 1 root root unconfined_u:object_r:user_tmp_t:s0 10731520 Feb 8 02:04 initrd libguestfs: -rwxr-xr-x. 1 root root unconfined_u:object_r:user_tmp_t:s0 20286832 Feb 8 02:04 kernel libguestfs: -rw-r--r--. 1 root root unconfined_u:object_r:user_tmp_t:s0 4294967296 Feb 8 02:04 root libguestfs: command: run: ls libguestfs: command: run: \ -a libguestfs: command: run: \ -l libguestfs: command: run: \ -Z /tmp/libguestfs3WAdBH libguestfs: total 4 libguestfs: drwxr-xr-x. 2 root root unconfined_u:object_r:user_tmp_t:s0 47 Feb 8 02:04 . libguestfs: drwxrwxrwt. 11 root root system_u:object_r:tmp_t:s0 4096 Feb 8 02:04 .. libguestfs: srw-rw----. 1 root qemu unconfined_u:object_r:user_tmp_t:s0 0 Feb 8 02:04 console.sock libguestfs: srw-rw----. 1 root qemu unconfined_u:object_r:user_tmp_t:s0 0 Feb 8 02:04 guestfsd.sock libguestfs: launch libvirt guest libguestfs: responding to serial console Device Status Report \x1b[1;256r\x1b[256;256H\x1b[6n Google, Inc. Serial Graphics Adapter 11/20/19 SGABIOS $Id$ (mockbuild@) Wed Nov 20 15:03:34 UTC 2019 Term: 80x24 4 0 SeaBIOS (version 1.13.0-2.module+el8.2.1+7284+aa32a2c4) Machine UUID 1af5efaa-c0ce-41dc-b8e8-eaabfa8597ef iPXE (http://ipxe.org) 00:05.0 C100 PCI2.10 PnP PMM+2FF90BD0+2FED0BD0 C100 Booting from ROM... \x1b[2J[ 0.000000] Linux version 4.18.0-193.71.1.rt13.121.el8_2.x86_64+debug (mockbuild.eng.bos.redhat.com) (gcc version 8.3.1 20191121 (Red Hat 8.3.1-5) (GCC)) #1 SMP PREEMPT RT Tue Dec 7 12:54:12 EST 2021 [ 0.000000] Command line: panic=1 console=ttyS0 edd=off udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=/dev/sdb selinux=0 guestfs_verbose=1 guestfs_network=1 TERM=xterm-256color [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 ... [ 25.4] Finishing off libguestfs: trace: shutdown libguestfs: trace: internal_autosync guestfsd: => umount_all (0x2f) took 1.37 secs guestfsd: <= internal_autosync (0x11a) request length 40 bytes umount-all: /proc/mounts: fsname=/dev/root dir=/ type=ext2 opts=rw,noatime freq=0 passno=0 umount-all: /proc/mounts: fsname=/proc dir=/proc type=proc opts=rw,relatime freq=0 passno=0 umount-all: /proc/mounts: fsname=/sys dir=/sys type=sysfs opts=rw,relatime freq=0 passno=0 umount-all: /proc/mounts: fsname=tmpfs dir=/run type=tmpfs opts=rw,nosuid,relatime,size=68200k,mode=755 freq=0 passno=0 umount-all: /proc/mounts: fsname=/dev dir=/dev type=devtmpfs opts=rw,relatime,size=158968k,nr_inodes=39742,mode=755 freq=0 passno=0 umount-all: /proc/mounts: fsname=/dev/pts dir=/dev/pts type=devpts opts=rw,relatime,mode=600,ptmxmode=000 freq=0 passno=0 umount-all: /proc/mounts: fsname=shmfs dir=/dev/shm type=tmpfs opts=rw,relatime freq=0 passno=0 commandrvf: stdout=n stderr=y flags=0x0 commandrvf: udevadm --debug settle -E /dev/sdb calling: settle commandrvf: stdout=n stderr=y flags=0x0 commandrvf: udevadm --debug settle -E /dev/sda calling: settle fsync /dev/sda libguestfs: trace: internal_autosync = 0 libguestfs: calling virDomainDestroy flags=VIR_DOMAIN_DESTROY_GRACEFUL libguestfs: trace: shutdown = 0 libguestfs: trace: close libguestfs: closing guestfs handle 0x560c3d4c20d0 (state 0) libguestfs: command: run: rm libguestfs: command: run: \ -rf /tmp/libguestfsN3VRtB libguestfs: command: run: rm libguestfs: command: run: \ -rf /tmp/libguestfs3WAdBH 7. virt-customize works with direct mode # LIBGUESTFS_BACKEND=direct virt-customize -v -x -a rhel-guest-image-8.2-501.x86_64.qcow2 --root-password password:"redhat" [ 0.0] Examining the guest ... libguestfs: trace: set_verbose true libguestfs: trace: set_verbose = 0 libguestfs: trace: set_network true libguestfs: trace: set_network = 0 libguestfs: trace: add_drive "rhel-guest-image-8.2-501.x86_64.qcow2" "readonly:false" "protocol:file" "discard:besteffort" libguestfs: trace: add_drive = 0 libguestfs: trace: launch libguestfs: trace: max_disks libguestfs: trace: max_disks = 255 libguestfs: trace: get_tmpdir libguestfs: trace: get_tmpdir = "/tmp" libguestfs: trace: version libguestfs: trace: version = <struct guestfs_version = major: 1, minor: 40, release: 2, extra: rhel=8,release=24.module+el8.2.1+7154+47ffd890,libvirt, > libguestfs: trace: get_backend libguestfs: trace: get_backend = "direct" libguestfs: launch: program=virt-customize libguestfs: launch: version=1.40.2rhel=8,release=24.module+el8.2.1+7154+47ffd890,libvirt libguestfs: launch: backend registered: unix libguestfs: launch: backend registered: uml libguestfs: launch: backend registered: libvirt libguestfs: launch: backend registered: direct libguestfs: launch: backend=direct libguestfs: launch: tmpdir=/tmp/libguestfst9Aqcf libguestfs: launch: umask=0022 libguestfs: launch: euid=0 libguestfs: trace: get_cachedir libguestfs: trace: get_cachedir = "/var/tmp" libguestfs: begin building supermin appliance libguestfs: run supermin libguestfs: command: run: /usr/bin/supermin libguestfs: command: run: \ --build libguestfs: command: run: \ --verbose libguestfs: command: run: \ --if-newer libguestfs: command: run: \ --lock /var/tmp/.guestfs-0/lock libguestfs: command: run: \ --copy-kernel libguestfs: command: run: \ -f ext2 libguestfs: command: run: \ --host-cpu x86_64 libguestfs: command: run: \ /usr/lib64/guestfs/supermin.d libguestfs: command: run: \ -o /var/tmp/.guestfs-0/appliance.d supermin: version: 5.1.19 supermin: rpm: detected RPM version 4.14 supermin: package handler: fedora/rpm supermin: acquiring lock on /var/tmp/.guestfs-0/lock supermin: build: /usr/lib64/guestfs/supermin.d supermin: reading the supermin appliance supermin: build: visiting /usr/lib64/guestfs/supermin.d/base.tar.gz type gzip base image (tar) supermin: build: visiting /usr/lib64/guestfs/supermin.d/daemon.tar.gz type gzip base image (tar) supermin: build: visiting /usr/lib64/guestfs/supermin.d/excludefiles type uncompressed excludefiles supermin: build: visiting /usr/lib64/guestfs/supermin.d/hostfiles type uncompressed hostfiles supermin: build: visiting /usr/lib64/guestfs/supermin.d/init.tar.gz type gzip base image (tar) supermin: build: visiting /usr/lib64/guestfs/supermin.d/packages type uncompressed packages supermin: build: visiting /usr/lib64/guestfs/supermin.d/udev-rules.tar.gz type gzip base image (tar) supermin: build: visiting /usr/lib64/guestfs/supermin.d/zz-packages-xfs type uncompressed packages supermin: mapping package names to installed packages supermin: resolving full list of package dependencies supermin: build: 184 packages, including dependencies supermin: build: 36481 files supermin: build: 10211 files, after matching excludefiles supermin: build: 10225 files, after adding hostfiles supermin: build: 10211 files, after removing unreadable files supermin: build: 10246 files, after munging supermin: kernel: looking for kernel using environment variables ... supermin: kernel: looking for kernels in /lib/modules/*/vmlinuz ... supermin: kernel: picked vmlinuz /lib/modules/4.18.0-193.71.1.rt13.121.el8_2.x86_64+debug/vmlinuz supermin: kernel: kernel_version 4.18.0-193.71.1.rt13.121.el8_2.x86_64+debug supermin: kernel: modpath /lib/modules/4.18.0-193.71.1.rt13.121.el8_2.x86_64+debug supermin: ext2: creating empty ext2 filesystem '/var/tmp/.guestfs-0/appliance.d.qelxddaf/root' supermin: ext2: populating from base image supermin: ext2: copying files from host filesystem supermin: ext2: copying kernel modules supermin: ext2: creating minimal initrd '/var/tmp/.guestfs-0/appliance.d.qelxddaf/initrd' supermin: ext2: wrote 28 modules to minimal initrd supermin: renaming /var/tmp/.guestfs-0/appliance.d.qelxddaf to /var/tmp/.guestfs-0/appliance.d libguestfs: finished building supermin appliance libguestfs: begin testing qemu features libguestfs: trace: get_cachedir libguestfs: trace: get_cachedir = "/var/tmp" libguestfs: checking for previously cached test results of /usr/libexec/qemu-kvm, in /var/tmp/.guestfs-0 libguestfs: command: run: /usr/libexec/qemu-kvm libguestfs: command: run: \ -display none libguestfs: command: run: \ -help libguestfs: command: run: /usr/libexec/qemu-kvm libguestfs: command: run: \ -display none libguestfs: command: run: \ -machine accel=kvm:tcg libguestfs: command: run: \ -device ? libguestfs: command: run: echo '{ "execute": "qmp_capabilities" }' '{ "execute": "query-qmp-schema" }' '{ "execute": "quit" }' | "/usr/libexec/qemu-kvm" -display none -machine "accel=kvm:tcg" -qmp stdio libguestfs: command: run: echo '{ "execute": "qmp_capabilities" }' '{ "execute": "query-kvm" }' '{ "execute": "quit" }' | "/usr/libexec/qemu-kvm" -display none -machine "accel=kvm:tcg" -qmp stdio libguestfs: saving test results libguestfs: qemu version: 4.2 libguestfs: qemu mandatory locking: yes libguestfs: qemu KVM: enabled libguestfs: trace: get_backend_setting "force_tcg" libguestfs: trace: get_backend_setting = NULL (error) libguestfs: trace: get_sockdir libguestfs: trace: get_sockdir = "/tmp" libguestfs: finished testing qemu features libguestfs: trace: get_backend_setting "gdb" libguestfs: trace: get_backend_setting = NULL (error) /usr/libexec/qemu-kvm \ -global virtio-blk-pci.scsi=off \ -no-user-config \ -enable-fips \ -nodefaults \ -display none \ -machine accel=kvm:tcg \ -cpu host \ -m 768 \ -no-reboot \ -rtc driftfix=slew \ -no-hpet \ -global kvm-pit.lost_tick_policy=discard \ -kernel /var/tmp/.guestfs-0/appliance.d/kernel \ -initrd /var/tmp/.guestfs-0/appliance.d/initrd \ -object rng-random,filename=/dev/urandom,id=rng0 \ -device virtio-rng-pci,rng=rng0 \ -device virtio-scsi-pci,id=scsi \ -drive file=/home/rhel-guest-image-8.2-501.x86_64.qcow2,cache=writeback,discard=unmap,id=hd0,if=none \ -device scsi-hd,drive=hd0 \ -drive file=/var/tmp/.guestfs-0/appliance.d/root,snapshot=on,id=appliance,cache=unsafe,if=none,format=raw \ -device scsi-hd,drive=appliance \ -device virtio-serial-pci \ -serial stdio \ -device sga \ -chardev socket,path=/tmp/libguestfsc9I26f/guestfsd.sock,id=channel0 \ -device virtserialport,chardev=channel0,name=org.libguestfs.channel.0 \ -netdev user,id=usernet,net=169.254.0.0/16 \ -device virtio-net-pci,netdev=usernet \ -append "panic=1 console=ttyS0 edd=off udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=/dev/sdb selinux=0 guestfs_verbose=1 guestfs_network=1 TERM=xterm-256color" libguestfs: responding to serial console Device Status Report \x1b[1;256r\x1b[256;256H\x1b[6n Google, Inc. Serial Graphics Adapter 11/20/19 SGABIOS $Id$ (mockbuild@) Wed Nov 20 15:03:34 UTC 2019 Term: 80x24 4 0 SeaBIOS (version 1.13.0-2.module+el8.2.1+7284+aa32a2c4) iPXE (http://ipxe.org) 00:05.0 C100 PCI2.10 PnP PMM+2FF907A0+2FED07A0 C100 Booting from ROM... \x1b[2J[ 0.000000] Linux version 4.18.0-193.71.1.rt13.121.el8_2.x86_64+debug (mockbuild.eng.bos.redhat.com) (gcc version 8.3.1 20191121 (Red Hat 8.3.1-5) (GCC)) #1 SMP PREEMPT RT Tue Dec 7 12:54:12 EST 2021 [ 0.000000] Command line: panic=1 console=ttyS0 edd=off udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=/dev/sdb selinux=0 guestfs_verbose=1 guestfs_network=1 TERM=xterm-256color [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] BIOS-provided physical RAM map: ... [ 32.9] Finishing off > @Minxi or @Richard, do we know what the error message is? What is actually failing?
In the original bug report there was no output after the usual
SeaBIOS messages, so either we didn't get into the kernel at all,
or we did get into the kernel but earlyprintk didn't work (most
likely the former). Unfortunately it's impossible to tell what
really happened without a local reproducer and a ton of debugging.
(In reply to Richard W.M. Jones from comment #10) > > @Minxi or @Richard, do we know what the error message is? What is actually failing? > > In the original bug report there was no output after the usual > SeaBIOS messages, so either we didn't get into the kernel at all, > or we did get into the kernel but earlyprintk didn't work (most > likely the former). Unfortunately it's impossible to tell what > really happened without a local reproducer and a ton of debugging. Richard, can you offer an advice if/how we should move forward? If I understood the report correctly, this failure only affects 8.2 (ie. it works with more recent RT kernels). We'll have Virt customers on 8.2.z for a possibly a long time. Also, if this is a kernel regression of some kind that could potentially affect other RT kernel customers, then it makes it more important to debug it. The Virt-RT engineers could have a look, but we'd need a reproducer (Prasad, could you have a look?). I can only offer suggestions I'm afraid. If we cannot reproduce it locally then we cannot debug it. It should be possible to reproduce it by: - Install RHEL 8.2 on baremetal - Install RT kernel - Install libguestfs - Run: libguestfs-test-tool If the bug is reproduced, you'll see output like this: Booting from ROM... \x1b[2Jlibguestfs: child_cleanup: 0x562783851870: child process died where the "Booting from ROM" message and \x1b[2J control codes come from seabios, there is no kernel output, then about 600 seconds later libguestfs times out waiting for the kernel and prints the debug message. (Yongkui tried that above and couldn't reproduce it, so maybe it's hardware dependent) I still can not reproduce this issue. Probably this problem has something with specific hardware. If needed I can provide my test environment to dev to see this issue. or any other help I can provide. (In reply to Luiz Capitulino from comment #13) > The Virt-RT engineers could have a look, but we'd need a reproducer (Prasad, > could you have a look?). * Yes, will check. # cat /etc/redhat-release
Red Hat Enterprise Linux release 8.2 (Ootpa)
#
# rpm -q libguestfs libvirt-daemon qemu-kvm seabios-bin kernel-rt
libguestfs-1.40.2-24.module+el8.2.1+7154+47ffd890.x86_64
libvirt-daemon-6.0.0-25.module+el8.2.1+7154+47ffd890.x86_64
qemu-kvm-4.2.0-29.module+el8.2.1+7297+a825794d.x86_64
seabios-bin-1.13.0-2.module+el8.2.1+7284+aa32a2c4.noarch
kernel-rt-4.18.0-193.71.1.rt13.121.el8_2.x86_64
#
# uname -r
4.18.0-193.el8.x86_64
#
#
# LIBGUESTFS_MEMSIZE=2048 LIBGUESTFS_BACKEND=direct virt-customize -a /var/lib/libvirt/images/r84vm.qcow2 --root-password password:"redhat"
[ 0.0] Examining the guest ...
[ 11.9] Setting a random seed
[ 12.0] Setting passwords
[ 14.9] Finishing off
#
#
# libguestfs-test-tool
...
* When reporting bugs, include the COMPLETE, UNEDITED
* output below in your bug report.
*
************************************************************
...
===== TEST FINISHED OK =====
# uname -r
4.18.0-193.71.1.rt13.121.el8_2.x86_64
#
# LIBGUESTFS_MEMSIZE=2048 LIBGUESTFS_BACKEND=direct virt-customize -a /var/lib/libvirt/images/f35vm.qcow2 --root-password password:"redhat"
[ 0.0] Examining the guest ...
[ 7.9] Setting a random seed
[ 7.9] Setting passwords
[ 10.4] Finishing off
#
#
# LIBGUESTFS_MEMSIZE=2048 libguestfs-test-tool
* When reporting bugs, include the COMPLETE, UNEDITED
* output below in your bug report.
*
************************************************************
...
===== TEST FINISHED OK =====
This issue of libguestfs throwing an error about child process running the guest VM dying is not reproducing.
The issue may have something to do with a guest image I guess, not sure. It does not seem to show up now.
@yoguo: @mhou: ...wdyt? Hello Prasad
Sorry for delay responed your message. I can reproduce this issue on my environment. I will share my envionment through e-mail thread. Here is my test step:
# rpm -q libguestfs libvirt-daemon qemu-kvm seabios-bin kernel-rt
libguestfs-1.40.2-24.module+el8.2.1+7154+47ffd890.x86_64
libvirt-daemon-6.0.0-25.module+el8.2.1+7154+47ffd890.x86_64
qemu-kvm-4.2.0-29.module+el8.2.1+7297+a825794d.x86_64
seabios-bin-1.13.0-2.module+el8.2.1+7284+aa32a2c4.noarch
kernel-rt-4.18.0-193.70.1.rt13.120.el8_2.x86_64
# LIBGUESTFS_MEMSIZE=2048 LIBGUESTFS_BACKEND=direct virt-customize -v -x -a /root/rhel7.9.qcow2 --root-password password:"redhat"
[ 0.0] Examining the guest ...
libguestfs: trace: set_verbose true
libguestfs: trace: set_verbose = 0
libguestfs: trace: set_network true
libguestfs: trace: set_network = 0
libguestfs: trace: add_drive "/root/rhel7.9.qcow2" "readonly:false" "protocol:file" "discard:besteffort"
libguestfs: trace: add_drive = 0
libguestfs: trace: launch
libguestfs: trace: max_disks
libguestfs: trace: max_disks = 255
libguestfs: trace: get_tmpdir
libguestfs: trace: get_tmpdir = "/tmp"
libguestfs: trace: version
libguestfs: trace: version = <struct guestfs_version = major: 1, minor: 40, release: 2, extra: rhel=8,release=24.module+el8.2.1+7154+47ffd890,libvirt, >
libguestfs: trace: get_backend
libguestfs: trace: get_backend = "direct"
libguestfs: launch: program=virt-customize
libguestfs: launch: version=1.40.2rhel=8,release=24.module+el8.2.1+7154+47ffd890,libvirt
libguestfs: launch: backend registered: unix
libguestfs: launch: backend registered: uml
libguestfs: launch: backend registered: libvirt
libguestfs: launch: backend registered: direct
libguestfs: launch: backend=direct
libguestfs: launch: tmpdir=/tmp/libguestfsXXkMH4
libguestfs: launch: umask=0022
libguestfs: launch: euid=0
libguestfs: trace: get_cachedir
libguestfs: trace: get_cachedir = "/var/tmp"
libguestfs: begin building supermin appliance
libguestfs: run supermin
libguestfs: command: run: /usr/bin/supermin
libguestfs: command: run: \ --build
libguestfs: command: run: \ --verbose
libguestfs: command: run: \ --if-newer
libguestfs: command: run: \ --lock /var/tmp/.guestfs-0/lock
libguestfs: command: run: \ --copy-kernel
libguestfs: command: run: \ -f ext2
libguestfs: command: run: \ --host-cpu x86_64
libguestfs: command: run: \ /usr/lib64/guestfs/supermin.d
libguestfs: command: run: \ -o /var/tmp/.guestfs-0/appliance.d
supermin: version: 5.1.19
supermin: rpm: detected RPM version 4.14
supermin: package handler: fedora/rpm
supermin: acquiring lock on /var/tmp/.guestfs-0/lock
supermin: if-newer: output does not need rebuilding
libguestfs: finished building supermin appliance
libguestfs: begin testing qemu features
libguestfs: trace: get_cachedir
libguestfs: trace: get_cachedir = "/var/tmp"
libguestfs: checking for previously cached test results of /usr/libexec/qemu-kvm, in /var/tmp/.guestfs-0
libguestfs: loading previously cached test results
libguestfs: qemu version: 4.2
libguestfs: qemu mandatory locking: yes
libguestfs: qemu KVM: enabled
libguestfs: trace: get_backend_setting "force_tcg"
libguestfs: trace: get_backend_setting = NULL (error)
libguestfs: trace: get_sockdir
libguestfs: trace: get_sockdir = "/tmp"
libguestfs: finished testing qemu features
libguestfs: trace: get_backend_setting "gdb"
libguestfs: trace: get_backend_setting = NULL (error)
/usr/libexec/qemu-kvm \
-global virtio-blk-pci.scsi=off \
-no-user-config \
-enable-fips \
-nodefaults \
-display none \
-machine accel=kvm:tcg \
-cpu host \
-m 2048 \
-no-reboot \
-rtc driftfix=slew \
-no-hpet \
-global kvm-pit.lost_tick_policy=discard \
-kernel /var/tmp/.guestfs-0/appliance.d/kernel \
-initrd /var/tmp/.guestfs-0/appliance.d/initrd \
-object rng-random,filename=/dev/urandom,id=rng0 \
-device virtio-rng-pci,rng=rng0 \
-device virtio-scsi-pci,id=scsi \
-drive file=/root/rhel7.9.qcow2,cache=writeback,discard=unmap,id=hd0,if=none \
-device scsi-hd,drive=hd0 \
-drive file=/var/tmp/.guestfs-0/appliance.d/root,snapshot=on,id=appliance,cache=unsafe,if=none,format=raw \
-device scsi-hd,drive=appliance \
-device virtio-serial-pci \
-serial stdio \
-device sga \
-chardev socket,path=/tmp/libguestfs9bDbSw/guestfsd.sock,id=channel0 \
-device virtserialport,chardev=channel0,name=org.libguestfs.channel.0 \
-netdev user,id=usernet,net=169.254.0.0/16 \
-device virtio-net-pci,netdev=usernet \
-append "panic=1 console=ttyS0 edd=off udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=/dev/sdb selinux=0 guestfs_verbose=1 guestfs_network=1 TERM=xterm"
libguestfs: responding to serial console Device Status Report
\x1b[1;256r\x1b[256;256H\x1b[6n
Google, Inc.
Serial Graphics Adapter 11/20/19
SGABIOS $Id$ (mockbuild@) Wed Nov 20 15:03:34 UTC 2019
Term: 80x24
4 0
SeaBIOS (version 1.13.0-2.module+el8.2.1+7284+aa32a2c4)
iPXE (http://ipxe.org) 00:05.0 C100 PCI2.10 PnP PMM+7FF907A0+7FED07A0 C100
Booting from ROM...
\x1b[2Jlibguestfs: child_cleanup: 0x56348dff4870: child process died
libguestfs: sending SIGTERM to process 25488
libguestfs: qemu maxrss 259236K
libguestfs: trace: launch = -1 (error)
virt-customize: error: libguestfs error: guestfs_launch failed, see earlier
error messages
libguestfs: trace: close
libguestfs: closing guestfs handle 0x56348dff4870 (state 0)
libguestfs: command: run: rm
libguestfs: command: run: \ -rf /tmp/libguestfsXXkMH4
libguestfs: command: run: rm
libguestfs: command: run: \ -rf /tmp/libguestfs9bDbSw
Hello Minxi,
* Thank you for sharing the output. Going through the logs this issue does not seem related to the realtime kernel. I'm able to reproduce it with a non-realtime kernel too.
# rpm -q kernel
kernel-4.18.0-193.el8.x86_64
* It is related to the console unix socket, which is getting deleted for some reason. It causes the guest process to terminate, which leads to the error
libguestfs: trace: launch = -1 (error)
virt-customize: error: libguestfs error: guestfs_launch failed
* libvirtd(8) logs show
Domain id=2 name='guestfs-gf0paodqqinxr0p7' uuid=7505f366-6029-406f-9d53-d398596a4ea2 is tainted: custom-argv
Domain id=2 name='guestfs-gf0paodqqinxr0p7' uuid=7505f366-6029-406f-9d53-d398596a4ea2 is tainted: host-cpu
internal error: End of file from qemu monitor
info : libvirt version: 6.0.0, package: 25.6.module+el8.2.1+12457+868e9540 (Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>, 2021-08-31-09:13:22, )
info : hostname: dell-per750-03.rhts.eng.pek2.redhat.com
warning : virSecurityDACTransactionRun:292 : Ignoring failed restore attempt on /tmp/libguestfs0lZYeW/overlay2.qcow2
warning : virSecurityDACTransactionRun:292 : Ignoring failed restore attempt on /var/lib/libvirt/images/f34vm.qcow2
warning : virSecurityDACTransactionRun:292 : Ignoring failed restore attempt on /tmp/libguestfs0lZYeW/overlay1.qcow2
internal error: child reported (status=125): unable to stat: /tmp/libguestfs0dYTfY/console.sock: No such file or directory
Unable to run security manager transaction
* I set-up the [Advanced-virt] repository on another machine and run the test, but all seems to work fine there.
/me wondering what's going on...?
@rjones: ...wdyt? I don't think the console socket is being deleted. What happens if we don't detect the kernel and userspace inside the appliance coming up within (by default) 600 seconds, then libguestfs kills qemu and deletes all the temporary files. The libvirtd messages are consistent with this, and it's normal behaviour (on this error path). The problem is that the kernel is not starting, or maybe it is starting but hanging/crashing very early before it is able to print any messages. These sorts of bugs are very hard to diagnose in my experience. They usually involve me recompiling the kernel and inserting more printk's very early on to try to find out exactly where it is hanging. If I was going to guess I'd say that the RT kernel isn't compatible with TCG + -cpu max. Do we have a test system I can log into (eg in Beaker) where this problem is reproducible? I'll need to compile and install some stuff, so a throwaway system there would be ideal. Hello Rjones, > I don't think the console socket is being deleted. > ...then libguestfs kills qemu and deletes all the temporary files. > * Yes, running strace(1) against libguestfs-test-tool(1) confirms that socket files get deleted after qemu terminates. === ... accept4(7, NULL, NULL, SOCK_CLOEXEC|SOCK_NONBLOCK) = 8 <= console_socket close(7) ... write(2, "libguestfs: error: error reading"..., 95libguestfs: error: error reading console messages from the appliance: Connection reset by peer) = 95 ... unlink("/tmp/libguestfs1lN6WV/guestfsd.sock") = 0 unlink("/tmp/libguestfs1lN6WV/console.sock") = 0 ... ==== * I compiled libguestfs-1.40.2 from -> https://download.libguestfs.org/1.40-stable/ And going through the trace, it seems to follow following path guestfs_launch -> guestfs_impl_launch -> launch_libvirt -> guestfs_int_recv_from_daemon -> recv_from_daemon -> read_data -> handle_log_message -> perrorf (g, _("error reading console messages from the appliance")): Connection reset by peer By this stage, Qemu process is already gone. * Guest boot-up seems to hang after printing Serial Graphics Adapter 11/20/19 SGABIOS $Id$ (mockbuild@) Wed Nov 20 15:03:34 UTC 2019 Booting from ROM... ... Likely guest kernel boot has not initiated yet. * I wonder why is it only happening on the this one machine and not others. I tried to replicate the same 'Advanced-virt' qemu-kvm/libvirt/libguestfs versions on another machines, issue does not show up there. Thank you. @mhou: would it be possible to share the machine with Rjones? Thank you. I'm afraid this has just missed the non-blocker deadline for 8.6, so it's more likely to go into 8.7. Upstream fix: https://github.com/libguestfs/supermin/commit/c44d685e662e001f5fe70f0a98d0964cb561e1ec (In reply to Prasad J Pandit from comment #25) > * I wonder why is it only happening on the this one machine and not others. > I tried to replicate the same 'Advanced-virt' qemu-kvm/libvirt/libguestfs > versions > on another machines, issue does not show up there. It's because the kernel-rt-debug kernel is installed on this machine. Libguestfs chooses this kernel to boot from, and it doesn't work. A simple reproducer is below. My fix was to make libguestfs/supermin pick a non-debug kernel instead. ----- reproducer ----- exec /usr/libexec/qemu-kvm \ -no-user-config \ -enable-fips \ -nodefaults \ -display none \ -machine accel=kvm:tcg \ -cpu host \ -m 2048 \ -no-reboot \ -rtc driftfix=slew \ -no-hpet \ -global kvm-pit.lost_tick_policy=discard \ -kernel /boot/vmlinuz-4.18.0-193.70.1.rt13.120.el8_2.x86_64+debug \ -initrd /var/tmp/.guestfs-0/appliance.d/initrd \ -object rng-random,filename=/dev/urandom,id=rng0 \ -device virtio-rng-pci,rng=rng0 \ -device virtio-scsi-pci,id=scsi \ -drive file=/var/tmp/.guestfs-0/appliance.d/root,snapshot=on,id=appliance,cache=unsafe,if=none,format=raw \ -device scsi-hd,drive=appliance \ -device virtio-serial-pci \ -serial stdio \ -device sga \ -append "console=ttyS0 printk.time=1 root=/dev/sda selinux=0 guestfs_verbose=1 TERM=xterm-256color ignore_loglevel" ------- (In reply to mhou from comment #31) > Hello Richard > > Do you still need this machine to troubleshoot more about the bug? If yes, I > will keep reserved for you, otherwise, I need to take this machine to do > more tests. I've finished with it now, thanks. Nice debugging work, Prasad and Rich! Quick question: can you speculate if kernel-debug RT could have a booting issue that's unrelated to Virt/TCG and would happen on bare-metal as well? If yes, I might get the RT kernel team attention to this issue. That machine is running 4.18.0-193.el8.x86_64 (non-debug) on the host, and I didn't want to reboot it to see if the debug kernel would have booted. The only thing I tested was the debug kernel as a guest under KVM (reproducer in comment 32). I don't know if the RT kernels at all, or the debug kernels in particular, are meant to work under KVM. I'd like to say that it's difficult to reproduce this issue on other machines even though the debug RT kernel packages are installed. I tried but failed to reproduce it(using debug RT kernel, see comment 9). Anyway, the fix (supermin ignores the debug kernel) is ok. (In reply to YongkuiGuo from comment #35) > I'd like to say that it's difficult to reproduce this issue on other > machines even though the debug RT kernel packages are installed. I tried but > failed to reproduce it(using debug RT kernel, see comment 9). I think to reproduce it, the kernel-rt-debug kernel must be a higher version than any other kernel/kernel-rt installed on the system. supermin will choose the higher version number first, and only if that is a +debug kernel will it fail. If kernel-rt-debug is the highest version and it doesn't fail, then send me the output of libguestfs-test-tool ls -l /lib/modules/*/vmlinuz (In reply to Richard W.M. Jones from comment #36) > I think to reproduce it, the kernel-rt-debug kernel must be a higher > version than any other kernel/kernel-rt installed on the system. > supermin will choose the higher version number first, and only if > that is a +debug kernel will it fail. > > If kernel-rt-debug is the highest version and it doesn't fail, then > send me the output of > > libguestfs-test-tool > ls -l /lib/modules/*/vmlinuz Still fails to reproduce on rhel8.6 host # uname -r 4.18.0-367.el8.x86_64 # rpm -q kernel kernel-rt kernel-4.18.0-367.el8.x86_64 kernel-rt-4.18.0-367.rt7.152.el8.x86_64 # rpm -q kernel-rt-debug kernel-rt-debug-4.18.0-369.rt7.154.el8.x86_64 # ls -l /lib/modules/*/vmlinuz -rwxr-xr-x. 1 root root 10460352 Feb 10 15:17 /lib/modules/4.18.0-367.el8.x86_64/vmlinuz -rwxr-xr-x. 1 root root 10337472 Feb 14 10:29 /lib/modules/4.18.0-367.rt7.152.el8.x86_64/vmlinuz -rwxr-xr-x. 1 root root 11365568 Feb 23 15:53 /lib/modules/4.18.0-369.rt7.154.el8.x86_64+debug/vmlinuz # libguestfs-test-tool ************************************************************ * IMPORTANT NOTICE * * When reporting bugs, include the COMPLETE, UNEDITED * output below in your bug report. * ************************************************************ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin XDG_RUNTIME_DIR=/run/user/0 SELinux: Enforcing guestfs_get_append: (null) guestfs_get_autosync: 1 guestfs_get_backend: libvirt guestfs_get_backend_settings: [] guestfs_get_cachedir: /var/tmp guestfs_get_hv: /usr/libexec/qemu-kvm guestfs_get_memsize: 1280 guestfs_get_network: 0 guestfs_get_path: /usr/lib64/guestfs guestfs_get_pgroup: 0 guestfs_get_program: libguestfs-test-tool guestfs_get_recovery_proc: 1 guestfs_get_smp: 1 guestfs_get_sockdir: /tmp guestfs_get_tmpdir: /tmp guestfs_get_trace: 0 guestfs_get_verbose: 1 host_cpu: x86_64 Launching appliance, timeout set to 600 seconds. libguestfs: launch: program=libguestfs-test-tool libguestfs: launch: version=1.44.0rhel=8,release=5.module+el8.6.0+13732+b2b9b31d,libvirt libguestfs: launch: backend registered: unix libguestfs: launch: backend registered: uml libguestfs: launch: backend registered: libvirt libguestfs: launch: backend registered: direct libguestfs: launch: backend=libvirt libguestfs: launch: tmpdir=/tmp/libguestfsmQjQMw libguestfs: launch: umask=0022 libguestfs: launch: euid=0 libguestfs: libvirt version = 8000000 (8.0.0) libguestfs: guest random name = guestfs-51iu167zji5srfc6 libguestfs: connect to libvirt libguestfs: opening libvirt handle: URI = qemu:///system, auth = default+wrapper, flags = 0 libguestfs: successfully opened libvirt handle: conn = 0x55ad08784030 libguestfs: qemu version (reported by libvirt) = 6002000 (6.2.0) libguestfs: get libvirt capabilities libguestfs: parsing capabilities XML libguestfs: parsing domcapabilities XML libguestfs: build appliance libguestfs: begin building supermin appliance libguestfs: run supermin libguestfs: command: run: /usr/bin/supermin libguestfs: command: run: \ --build libguestfs: command: run: \ --verbose libguestfs: command: run: \ --if-newer libguestfs: command: run: \ --lock /var/tmp/.guestfs-0/lock libguestfs: command: run: \ --copy-kernel libguestfs: command: run: \ -f ext2 libguestfs: command: run: \ --host-cpu x86_64 libguestfs: command: run: \ /usr/lib64/guestfs/supermin.d libguestfs: command: run: \ -o /var/tmp/.guestfs-0/appliance.d supermin: version: 5.2.1 supermin: rpm: detected RPM version 4.14 supermin: rpm: detected RPM architecture x86_64 supermin: package handler: fedora/rpm supermin: acquiring lock on /var/tmp/.guestfs-0/lock supermin: build: /usr/lib64/guestfs/supermin.d supermin: reading the supermin appliance supermin: build: visiting /usr/lib64/guestfs/supermin.d/base.tar.gz type gzip base image (tar) supermin: build: visiting /usr/lib64/guestfs/supermin.d/daemon.tar.gz type gzip base image (tar) supermin: build: visiting /usr/lib64/guestfs/supermin.d/excludefiles type uncompressed excludefiles supermin: build: visiting /usr/lib64/guestfs/supermin.d/hostfiles type uncompressed hostfiles supermin: build: visiting /usr/lib64/guestfs/supermin.d/init.tar.gz type gzip base image (tar) supermin: build: visiting /usr/lib64/guestfs/supermin.d/packages type uncompressed packages supermin: build: visiting /usr/lib64/guestfs/supermin.d/udev-rules.tar.gz type gzip base image (tar) supermin: mapping package names to installed packages supermin: resolving full list of package dependencies supermin: build: 175 packages, including dependencies supermin: build: 33809 files supermin: build: 6679 files, after matching excludefiles supermin: build: 6693 files, after adding hostfiles supermin: build: 6680 files, after removing unreadable files supermin: build: 6715 files, after munging supermin: kernel: looking for kernel using environment variables ... supermin: kernel: looking for kernels in /lib/modules/*/vmlinuz ... supermin: kernel: picked vmlinuz /lib/modules/4.18.0-369.rt7.154.el8.x86_64+debug/vmlinuz supermin: kernel: kernel_version 4.18.0-369.rt7.154.el8.x86_64+debug supermin: kernel: modpath /lib/modules/4.18.0-369.rt7.154.el8.x86_64+debug supermin: ext2: creating empty ext2 filesystem '/var/tmp/.guestfs-0/appliance.d.jkjf6b5u/root' supermin: ext2: populating from base image supermin: ext2: copying files from host filesystem supermin: ext2: copying kernel modules supermin: ext2: creating minimal initrd '/var/tmp/.guestfs-0/appliance.d.jkjf6b5u/initrd' supermin: ext2: wrote 36 modules to minimal initrd supermin: renaming /var/tmp/.guestfs-0/appliance.d.jkjf6b5u to /var/tmp/.guestfs-0/appliance.d libguestfs: finished building supermin appliance libguestfs: command: run: qemu-img libguestfs: command: run: \ create libguestfs: command: run: \ -f qcow2 libguestfs: command: run: \ -o backing_file=/var/tmp/.guestfs-0/appliance.d/root,backing_fmt=raw libguestfs: command: run: \ /tmp/libguestfsmQjQMw/overlay2.qcow2 Formatting '/tmp/libguestfsmQjQMw/overlay2.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=4294967296 backing_file=/var/tmp/.guestfs-0/appliance.d/root backing_fmt=raw lazy_refcounts=off refcount_bits=16 libguestfs: create libvirt XML libguestfs: command: run: file libguestfs: command: run: \ -- /var/tmp/.guestfs-0/appliance.d/root libguestfs: libvirt XML:\n<?xml version="1.0"?>\n<domain type="kvm" xmlns:qemu="http://libvirt.org/schemas/domain/qemu/1.0">\n <name>guestfs-51iu167zji5srfc6</name>\n <memory unit="MiB">1280</memory>\n <currentMemory unit="MiB">1280</currentMemory>\n <cpu mode="host-passthrough">\n <model fallback="allow"/>\n </cpu>\n <vcpu>1</vcpu>\n <clock offset="utc">\n <timer name="rtc" tickpolicy="catchup"/>\n <timer name="pit" tickpolicy="delay"/>\n <timer name="hpet" present="no"/>\n </clock>\n <os>\n <type>hvm</type>\n <kernel>/var/tmp/.guestfs-0/appliance.d/kernel</kernel>\n <initrd>/var/tmp/.guestfs-0/appliance.d/initrd</initrd>\n <cmdline>panic=1 console=ttyS0 edd=off udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=UUID=743e66cb-0a0b-4232-ab17-f37fcc5ffc1c selinux=0 guestfs_verbose=1 TERM=xterm-256color</cmdline>\n <bios useserial="yes"/>\n </os>\n <on_reboot>destroy</on_reboot>\n <devices>\n <rng model="virtio">\n <backend model="random">/dev/urandom</backend>\n </rng>\n <controller type="scsi" index="0" model="virtio-scsi"/>\n <disk device="disk" type="file">\n <source file="/tmp/libguestfsmQjQMw/scratch1.img"/>\n <target dev="sda" bus="scsi"/>\n <driver name="qemu" type="raw" cache="unsafe"/>\n <address type="drive" controller="0" bus="0" target="0" unit="0"/>\n </disk>\n <disk type="file" device="disk">\n <source file="/tmp/libguestfsmQjQMw/overlay2.qcow2"/>\n <target dev="sdb" bus="scsi"/>\n <driver name="qemu" type="qcow2" cache="unsafe"/>\n <address type="drive" controller="0" bus="0" target="1" unit="0"/>\n </disk>\n <serial type="unix">\n <source mode="connect" path="/tmp/libguestfsBMgK5u/console.sock"/>\n <target port="0"/>\n </serial>\n <channel type="unix">\n <source mode="connect" path="/tmp/libguestfsBMgK5u/guestfsd.sock"/>\n <target type="virtio" name="org.libguestfs.channel.0"/>\n </channel>\n <controller type="usb" model="none"/>\n <memballoon model="none"/>\n </devices>\n <qemu:commandline>\n <qemu:env name="TMPDIR" value="/var/tmp"/>\n </qemu:commandline>\n</domain>\n libguestfs: command: run: ls libguestfs: command: run: \ -a libguestfs: command: run: \ -l libguestfs: command: run: \ -R libguestfs: command: run: \ -Z /var/tmp/.guestfs-0 libguestfs: /var/tmp/.guestfs-0: libguestfs: total 0 libguestfs: drwxr-xr-x. 3 root root unconfined_u:object_r:user_tmp_t:s0 37 Mar 4 04:03 . libguestfs: drwxrwxrwt. 6 root root system_u:object_r:tmp_t:s0 200 Mar 4 04:03 .. libguestfs: drwxr-xr-x. 2 root root unconfined_u:object_r:user_tmp_t:s0 46 Mar 4 04:03 appliance.d libguestfs: -rw-r--r--. 1 root root unconfined_u:object_r:user_tmp_t:s0 0 Mar 4 04:03 lock libguestfs: libguestfs: /var/tmp/.guestfs-0/appliance.d: libguestfs: total 274620 libguestfs: drwxr-xr-x. 2 root root unconfined_u:object_r:user_tmp_t:s0 46 Mar 4 04:03 . libguestfs: drwxr-xr-x. 3 root root unconfined_u:object_r:user_tmp_t:s0 37 Mar 4 04:03 .. libguestfs: -rw-r--r--. 1 root root unconfined_u:object_r:user_tmp_t:s0 5726208 Mar 4 04:03 initrd libguestfs: -rwxr-xr-x. 1 root root unconfined_u:object_r:user_tmp_t:s0 11365568 Mar 4 04:03 kernel libguestfs: -rw-r--r--. 1 root root unconfined_u:object_r:user_tmp_t:s0 4294967296 Mar 4 04:03 root libguestfs: command: run: ls libguestfs: command: run: \ -a libguestfs: command: run: \ -l libguestfs: command: run: \ -Z /tmp/libguestfsBMgK5u libguestfs: total 4 libguestfs: drwxr-xr-x. 2 root root unconfined_u:object_r:user_tmp_t:s0 47 Mar 4 04:03 . libguestfs: drwxrwxrwt. 11 root root system_u:object_r:tmp_t:s0 4096 Mar 4 04:03 .. libguestfs: srw-rw----. 1 root qemu unconfined_u:object_r:user_tmp_t:s0 0 Mar 4 04:03 console.sock libguestfs: srw-rw----. 1 root qemu unconfined_u:object_r:user_tmp_t:s0 0 Mar 4 04:03 guestfsd.sock libguestfs: launch libvirt guest \x1bc\x1b[?7l\x1b[2J\x1b[0mSeaBIOS (version 1.15.0-1.module+el8.6.0+13725+61ae1949) Machine UUID 985ebafc-f1ec-469b-a530-fcee2db45753 Booting from ROM..\x1bc\x1b[?7l\x1b[2J[ 0.000000] Linux version 4.18.0-369.rt7.154.el8.x86_64+debug (mockbuild.eng.bos.redhat.com) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-10) (GCC)) #1 SMP PREEMPT_RT Wed Feb 23 15:25:34 EST 2022 [ 0.000000] Command line: panic=1 console=ttyS0 edd=off udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=UUID=743e66cb-0a0b-4232-ab17-f37fcc5ffc1c selinux=0 guestfs_verbose=1 TERM=xterm-256color [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000004fffcfff] usable [ 0.000000] BIOS-e820: [mem 0x000000004fffd000-0x000000004fffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: Red Hat KVM, BIOS 1.15.0-1.module+el8.6.0+13725+61ae1949 04/01/2014 [ 0.000000] Hypervisor detected: KVM ... ===== TEST FINISHED OK ===== Sorry for uploading the same log of libguestfs-test-tool twice. Interestingly we picked the debug kernel and booted it:
> supermin: kernel: picked vmlinuz /lib/modules/4.18.0-369.rt7.154.el8.x86_64+debug/vmlinuz
> supermin: kernel: kernel_version 4.18.0-369.rt7.154.el8.x86_64+debug
> supermin: kernel: modpath /lib/modules/4.18.0-369.rt7.154.el8.x86_64+debug
It seems as if the other debug kernel which failed (4.18.0-193.71.1.rt13.121.el8_2)
probably has a bug that stops it from booting on qemu (or qemu has a bug
that breaks emulation).
Anyway, choosing non-debug kernels is better because the debug ones run
slower and use more RAM, assuming they're anything like the Fedora debug
kernels. So the change to supermin is still valid.
(In reply to Richard W.M. Jones from comment #42) > Anyway, choosing non-debug kernels is better because the debug ones run > slower and use more RAM, assuming they're anything like the Fedora debug > kernels. So the change to supermin is still valid. Yes. Ignoring the debug RT kernel is the most direct way to avoid this kind of issue. BTW, could you help change the 'Product' from RHEL AV to RHEL? Change component to RHEL since RHEL AV & RHEL have converged in >= 8.6 I don't think this is a blocker bug, so probably we will not be able to fix it in RHEL 8.6 and it'll have to wait til 8.7. I had to remove the TestOnly keyword because: remote: *** Resolves: remote: *** Unapproved: remote: *** rhbz#2051332 (qe_test_coverage?, qa_ack+, internal_target_release=8.7.0, devel_ack+, mirror+, release+, stale-, TestOnly, Triaged, ZStream) remote: *** This is unapproved because at least one of the BZs contains the TestOnly keyword. TestOnly keyword must be removed in order to pass approval. On rhel8.7 host:
1.
# rpm -q kernel kernel-rt kernel-rt-debug
kernel-4.18.0-369.el8.x86_64
kernel-rt-4.18.0-369.rt7.154.el8.x86_64
kernel-rt-debug-4.18.0-369.rt7.154.el8.x86_64
2.Tested with previous supermin package: supermin-5.2.1-1.module+el8.6.0+12861+13975d62.x86_64
# libguestfs-test-tool
************************************************************
* IMPORTANT NOTICE
*
* When reporting bugs, include the COMPLETE, UNEDITED
* output below in your bug report.
*
************************************************************
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin
XDG_RUNTIME_DIR=/run/user/0
SELinux: Enforcing
guestfs_get_append: (null)
guestfs_get_autosync: 1
guestfs_get_backend: libvirt
guestfs_get_backend_settings: []
guestfs_get_cachedir: /var/tmp
guestfs_get_hv: /usr/libexec/qemu-kvm
guestfs_get_memsize: 1280
guestfs_get_network: 0
guestfs_get_path: /usr/lib64/guestfs
guestfs_get_pgroup: 0
guestfs_get_program: libguestfs-test-tool
guestfs_get_recovery_proc: 1
guestfs_get_smp: 1
guestfs_get_sockdir: /tmp
guestfs_get_tmpdir: /tmp
guestfs_get_trace: 0
guestfs_get_verbose: 1
host_cpu: x86_64
Launching appliance, timeout set to 600 seconds.
libguestfs: launch: program=libguestfs-test-tool
libguestfs: launch: version=1.44.0rhel=8,release=5.module+el8.6.0+13732+b2b9b31d,libvirt
libguestfs: launch: backend registered: unix
libguestfs: launch: backend registered: uml
libguestfs: launch: backend registered: libvirt
libguestfs: launch: backend registered: direct
libguestfs: launch: backend=libvirt
libguestfs: launch: tmpdir=/tmp/libguestfsHvvTdy
libguestfs: launch: umask=0022
libguestfs: launch: euid=0
libguestfs: libvirt version = 8000000 (8.0.0)
libguestfs: guest random name = guestfs-yrkb1h2bigcrsmtg
libguestfs: connect to libvirt
libguestfs: opening libvirt handle: URI = qemu:///system, auth = default+wrapper, flags = 0
libguestfs: successfully opened libvirt handle: conn = 0x55ae8ce7e030
libguestfs: qemu version (reported by libvirt) = 6002000 (6.2.0)
libguestfs: get libvirt capabilities
libguestfs: parsing capabilities XML
libguestfs: parsing domcapabilities XML
libguestfs: build appliance
libguestfs: begin building supermin appliance
libguestfs: run supermin
libguestfs: command: run: /usr/bin/supermin
libguestfs: command: run: \ --build
libguestfs: command: run: \ --verbose
libguestfs: command: run: \ --if-newer
libguestfs: command: run: \ --lock /var/tmp/.guestfs-0/lock
libguestfs: command: run: \ --copy-kernel
libguestfs: command: run: \ -f ext2
libguestfs: command: run: \ --host-cpu x86_64
libguestfs: command: run: \ /usr/lib64/guestfs/supermin.d
libguestfs: command: run: \ -o /var/tmp/.guestfs-0/appliance.d
supermin: version: 5.2.1
supermin: rpm: detected RPM version 4.14
supermin: rpm: detected RPM architecture x86_64
supermin: package handler: fedora/rpm
supermin: acquiring lock on /var/tmp/.guestfs-0/lock
supermin: build: /usr/lib64/guestfs/supermin.d
supermin: reading the supermin appliance
supermin: build: visiting /usr/lib64/guestfs/supermin.d/base.tar.gz type gzip base image (tar)
supermin: build: visiting /usr/lib64/guestfs/supermin.d/daemon.tar.gz type gzip base image (tar)
supermin: build: visiting /usr/lib64/guestfs/supermin.d/excludefiles type uncompressed excludefiles
supermin: build: visiting /usr/lib64/guestfs/supermin.d/hostfiles type uncompressed hostfiles
supermin: build: visiting /usr/lib64/guestfs/supermin.d/init.tar.gz type gzip base image (tar)
supermin: build: visiting /usr/lib64/guestfs/supermin.d/packages type uncompressed packages
supermin: build: visiting /usr/lib64/guestfs/supermin.d/udev-rules.tar.gz type gzip base image (tar)
supermin: mapping package names to installed packages
supermin: resolving full list of package dependencies
supermin: build: 175 packages, including dependencies
supermin: build: 33813 files
supermin: build: 6683 files, after matching excludefiles
supermin: build: 6697 files, after adding hostfiles
supermin: build: 6685 files, after removing unreadable files
supermin: build: 6720 files, after munging
supermin: kernel: looking for kernel using environment variables ...
supermin: kernel: looking for kernels in /lib/modules/*/vmlinuz ...
supermin: kernel: picked vmlinuz /lib/modules/4.18.0-369.rt7.154.el8.x86_64+debug/vmlinuz
supermin: kernel: kernel_version 4.18.0-369.rt7.154.el8.x86_64+debug
supermin: kernel: modpath /lib/modules/4.18.0-369.rt7.154.el8.x86_64+debug
supermin: ext2: creating empty ext2 filesystem '/var/tmp/.guestfs-0/appliance.d.1wijz8eh/root'
supermin: ext2: populating from base image
supermin: ext2: copying files from host filesystem
supermin: ext2: copying kernel modules
supermin: ext2: creating minimal initrd '/var/tmp/.guestfs-0/appliance.d.1wijz8eh/initrd'
supermin: ext2: wrote 36 modules to minimal initrd
supermin: renaming /var/tmp/.guestfs-0/appliance.d.1wijz8eh to /var/tmp/.guestfs-0/appliance.d
libguestfs: finished building supermin appliance
libguestfs: command: run: qemu-img
libguestfs: command: run: \ create
libguestfs: command: run: \ -f qcow2
libguestfs: command: run: \ -o backing_file=/var/tmp/.guestfs-0/appliance.d/root,backing_fmt=raw
libguestfs: command: run: \ /tmp/libguestfsHvvTdy/overlay2.qcow2
Formatting '/tmp/libguestfsHvvTdy/overlay2.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=4294967296 backing_file=/var/tmp/.guestfs-0/appliance.d/root backing_fmt=raw lazy_refcounts=off refcount_bits=16
libguestfs: create libvirt XML
libguestfs: command: run: file
libguestfs: command: run: \ -- /var/tmp/.guestfs-0/appliance.d/root
libguestfs: libvirt XML:\n<?xml version="1.0"?>\n<domain type="kvm" xmlns:qemu="http://libvirt.org/schemas/domain/qemu/1.0">\n <name>guestfs-yrkb1h2bigcrsmtg</name>\n <memory unit="MiB">1280</memory>\n <currentMemory unit="MiB">1280</currentMemory>\n <cpu mode="host-passthrough">\n <model fallback="allow"/>\n </cpu>\n <vcpu>1</vcpu>\n <clock offset="utc">\n <timer name="rtc" tickpolicy="catchup"/>\n <timer name="pit" tickpolicy="delay"/>\n <timer name="hpet" present="no"/>\n </clock>\n <os>\n <type>hvm</type>\n <kernel>/var/tmp/.guestfs-0/appliance.d/kernel</kernel>\n <initrd>/var/tmp/.guestfs-0/appliance.d/initrd</initrd>\n <cmdline>panic=1 console=ttyS0 edd=off udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=UUID=8495c444-92af-4290-ac6c-e416be6622cd selinux=0 guestfs_verbose=1 TERM=xterm-256color</cmdline>\n <bios useserial="yes"/>\n </os>\n <on_reboot>destroy</on_reboot>\n <devices>\n <rng model="virtio">\n <backend model="random">/dev/urandom</backend>\n </rng>\n <controller type="scsi" index="0" model="virtio-scsi"/>\n <disk device="disk" type="file">\n <source file="/tmp/libguestfsHvvTdy/scratch1.img"/>\n <target dev="sda" bus="scsi"/>\n <driver name="qemu" type="raw" cache="unsafe"/>\n <address type="drive" controller="0" bus="0" target="0" unit="0"/>\n </disk>\n <disk type="file" device="disk">\n <source file="/tmp/libguestfsHvvTdy/overlay2.qcow2"/>\n <target dev="sdb" bus="scsi"/>\n <driver name="qemu" type="qcow2" cache="unsafe"/>\n <address type="drive" controller="0" bus="0" target="1" unit="0"/>\n </disk>\n <serial type="unix">\n <source mode="connect" path="/tmp/libguestfsGQ1qqz/console.sock"/>\n <target port="0"/>\n </serial>\n <channel type="unix">\n <source mode="connect" path="/tmp/libguestfsGQ1qqz/guestfsd.sock"/>\n <target type="virtio" name="org.libguestfs.channel.0"/>\n </channel>\n <controller type="usb" model="none"/>\n <memballoon model="none"/>\n </devices>\n <qemu:commandline>\n <qemu:env name="TMPDIR" value="/var/tmp"/>\n </qemu:commandline>\n</domain>\n
libguestfs: command: run: ls
libguestfs: command: run: \ -a
libguestfs: command: run: \ -l
libguestfs: command: run: \ -R
libguestfs: command: run: \ -Z /var/tmp/.guestfs-0
libguestfs: /var/tmp/.guestfs-0:
libguestfs: total 0
libguestfs: drwxr-xr-x. 3 root root unconfined_u:object_r:user_tmp_t:s0 37 Mar 8 04:09 .
libguestfs: drwxrwxrwt. 5 root root system_u:object_r:tmp_t:s0 121 Mar 8 04:09 ..
libguestfs: drwxr-xr-x. 2 root root unconfined_u:object_r:user_tmp_t:s0 46 Mar 8 04:09 appliance.d
libguestfs: -rw-r--r--. 1 root root unconfined_u:object_r:user_tmp_t:s0 0 Mar 8 04:09 lock
libguestfs:
libguestfs: /var/tmp/.guestfs-0/appliance.d:
libguestfs: total 274636
libguestfs: drwxr-xr-x. 2 root root unconfined_u:object_r:user_tmp_t:s0 46 Mar 8 04:09 .
libguestfs: drwxr-xr-x. 3 root root unconfined_u:object_r:user_tmp_t:s0 37 Mar 8 04:09 ..
libguestfs: -rw-r--r--. 1 root root unconfined_u:object_r:user_tmp_t:s0 5726208 Mar 8 04:09 initrd
libguestfs: -rwxr-xr-x. 1 root root unconfined_u:object_r:user_tmp_t:s0 11365568 Mar 8 04:09 kernel
libguestfs: -rw-r--r--. 1 root root unconfined_u:object_r:user_tmp_t:s0 4294967296 Mar 8 04:09 root
libguestfs: command: run: ls
libguestfs: command: run: \ -a
libguestfs: command: run: \ -l
libguestfs: command: run: \ -Z /tmp/libguestfsGQ1qqz
libguestfs: total 4
libguestfs: drwxr-xr-x. 2 root root unconfined_u:object_r:user_tmp_t:s0 47 Mar 8 04:09 .
libguestfs: drwxrwxrwt. 11 root root system_u:object_r:tmp_t:s0 4096 Mar 8 04:09 ..
libguestfs: srw-rw----. 1 root qemu unconfined_u:object_r:user_tmp_t:s0 0 Mar 8 04:09 console.sock
libguestfs: srw-rw----. 1 root qemu unconfined_u:object_r:user_tmp_t:s0 0 Mar 8 04:09 guestfsd.sock
libguestfs: launch libvirt guest
\x1bc\x1b[?7l\x1b[2J\x1b[0mSeaBIOS (version 1.15.0-1.module+el8.6.0+13725+61ae1949)
Machine UUID 8581bc39-2091-40d3-8495-56468a2c4720
Booting from ROM..\x1bc\x1b[?7l\x1b[2J[ 0.000000] Linux version 4.18.0-369.rt7.154.el8.x86_64+debug (mockbuild.eng.bos.redhat.com) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-10) (GCC)) #1 SMP PREEMPT_RT Wed Feb 23 15:25:34 EST 2022
[ 0.000000] Command line: panic=1 console=ttyS0 edd=off udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=UUID=8495c444-92af-4290-ac6c-e416be6622cd selinux=0 guestfs_verbose=1 TERM=xterm-256color
...
===== TEST FINISHED OK =====
The debug RT kernel was picked by supermin.
3.Tested with the fixed supermin package: supermin-5.2.1-2.module+el8.7.0+14396+56bc2570.x86_64
# libguestfs-test-tool
************************************************************
* IMPORTANT NOTICE
*
* When reporting bugs, include the COMPLETE, UNEDITED
* output below in your bug report.
*
************************************************************
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin
XDG_RUNTIME_DIR=/run/user/0
SELinux: Enforcing
guestfs_get_append: (null)
guestfs_get_autosync: 1
guestfs_get_backend: libvirt
guestfs_get_backend_settings: []
guestfs_get_cachedir: /var/tmp
guestfs_get_hv: /usr/libexec/qemu-kvm
guestfs_get_memsize: 1280
guestfs_get_network: 0
guestfs_get_path: /usr/lib64/guestfs
guestfs_get_pgroup: 0
guestfs_get_program: libguestfs-test-tool
guestfs_get_recovery_proc: 1
guestfs_get_smp: 1
guestfs_get_sockdir: /tmp
guestfs_get_tmpdir: /tmp
guestfs_get_trace: 0
guestfs_get_verbose: 1
host_cpu: x86_64
Launching appliance, timeout set to 600 seconds.
libguestfs: launch: program=libguestfs-test-tool
libguestfs: launch: version=1.44.0rhel=8,release=5.module+el8.6.0+13732+b2b9b31d,libvirt
libguestfs: launch: backend registered: unix
libguestfs: launch: backend registered: uml
libguestfs: launch: backend registered: libvirt
libguestfs: launch: backend registered: direct
libguestfs: launch: backend=libvirt
libguestfs: launch: tmpdir=/tmp/libguestfs2EG86U
libguestfs: launch: umask=0022
libguestfs: launch: euid=0
libguestfs: libvirt version = 8000000 (8.0.0)
libguestfs: guest random name = guestfs-aezowj4a4x8sv4kc
libguestfs: connect to libvirt
libguestfs: opening libvirt handle: URI = qemu:///system, auth = default+wrapper, flags = 0
libguestfs: successfully opened libvirt handle: conn = 0x5575c3282030
libguestfs: qemu version (reported by libvirt) = 6002000 (6.2.0)
libguestfs: get libvirt capabilities
libguestfs: parsing capabilities XML
libguestfs: parsing domcapabilities XML
libguestfs: build appliance
libguestfs: begin building supermin appliance
libguestfs: run supermin
libguestfs: command: run: /usr/bin/supermin
libguestfs: command: run: \ --build
libguestfs: command: run: \ --verbose
libguestfs: command: run: \ --if-newer
libguestfs: command: run: \ --lock /var/tmp/.guestfs-0/lock
libguestfs: command: run: \ --copy-kernel
libguestfs: command: run: \ -f ext2
libguestfs: command: run: \ --host-cpu x86_64
libguestfs: command: run: \ /usr/lib64/guestfs/supermin.d
libguestfs: command: run: \ -o /var/tmp/.guestfs-0/appliance.d
supermin: version: 5.2.1
supermin: rpm: detected RPM version 4.14
supermin: rpm: detected RPM architecture x86_64
supermin: package handler: fedora/rpm
supermin: acquiring lock on /var/tmp/.guestfs-0/lock
supermin: build: /usr/lib64/guestfs/supermin.d
supermin: reading the supermin appliance
supermin: build: visiting /usr/lib64/guestfs/supermin.d/base.tar.gz type gzip base image (tar)
supermin: build: visiting /usr/lib64/guestfs/supermin.d/daemon.tar.gz type gzip base image (tar)
supermin: build: visiting /usr/lib64/guestfs/supermin.d/excludefiles type uncompressed excludefiles
supermin: build: visiting /usr/lib64/guestfs/supermin.d/hostfiles type uncompressed hostfiles
supermin: build: visiting /usr/lib64/guestfs/supermin.d/init.tar.gz type gzip base image (tar)
supermin: build: visiting /usr/lib64/guestfs/supermin.d/packages type uncompressed packages
supermin: build: visiting /usr/lib64/guestfs/supermin.d/udev-rules.tar.gz type gzip base image (tar)
supermin: mapping package names to installed packages
supermin: resolving full list of package dependencies
supermin: build: 175 packages, including dependencies
supermin: build: 33813 files
supermin: build: 6683 files, after matching excludefiles
supermin: build: 6697 files, after adding hostfiles
supermin: build: 6685 files, after removing unreadable files
supermin: build: 6720 files, after munging
supermin: kernel: looking for kernel using environment variables ...
supermin: kernel: looking for kernels in /lib/modules/*/vmlinuz ...
supermin: kernel: picked vmlinuz /lib/modules/4.18.0-369.rt7.154.el8.x86_64/vmlinuz
supermin: kernel: kernel_version 4.18.0-369.rt7.154.el8.x86_64
supermin: kernel: modpath /lib/modules/4.18.0-369.rt7.154.el8.x86_64
supermin: ext2: creating empty ext2 filesystem '/var/tmp/.guestfs-0/appliance.d.6fmn3krb/root'
supermin: ext2: populating from base image
supermin: ext2: copying files from host filesystem
supermin: ext2: copying kernel modules
supermin: ext2: creating minimal initrd '/var/tmp/.guestfs-0/appliance.d.6fmn3krb/initrd'
supermin: ext2: wrote 36 modules to minimal initrd
supermin: renaming /var/tmp/.guestfs-0/appliance.d.6fmn3krb to /var/tmp/.guestfs-0/appliance.d
libguestfs: finished building supermin appliance
libguestfs: command: run: qemu-img
libguestfs: command: run: \ create
libguestfs: command: run: \ -f qcow2
libguestfs: command: run: \ -o backing_file=/var/tmp/.guestfs-0/appliance.d/root,backing_fmt=raw
libguestfs: command: run: \ /tmp/libguestfs2EG86U/overlay2.qcow2
Formatting '/tmp/libguestfs2EG86U/overlay2.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=4294967296 backing_file=/var/tmp/.guestfs-0/appliance.d/root backing_fmt=raw lazy_refcounts=off refcount_bits=16
libguestfs: create libvirt XML
libguestfs: command: run: file
libguestfs: command: run: \ -- /var/tmp/.guestfs-0/appliance.d/root
libguestfs: libvirt XML:\n<?xml version="1.0"?>\n<domain type="kvm" xmlns:qemu="http://libvirt.org/schemas/domain/qemu/1.0">\n <name>guestfs-aezowj4a4x8sv4kc</name>\n <memory unit="MiB">1280</memory>\n <currentMemory unit="MiB">1280</currentMemory>\n <cpu mode="host-passthrough">\n <model fallback="allow"/>\n </cpu>\n <vcpu>1</vcpu>\n <clock offset="utc">\n <timer name="rtc" tickpolicy="catchup"/>\n <timer name="pit" tickpolicy="delay"/>\n <timer name="hpet" present="no"/>\n </clock>\n <os>\n <type>hvm</type>\n <kernel>/var/tmp/.guestfs-0/appliance.d/kernel</kernel>\n <initrd>/var/tmp/.guestfs-0/appliance.d/initrd</initrd>\n <cmdline>panic=1 console=ttyS0 edd=off udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=UUID=3c1e4f94-b6ba-40f9-8f70-e7b0b5e65f4d selinux=0 guestfs_verbose=1 TERM=xterm-256color</cmdline>\n <bios useserial="yes"/>\n </os>\n <on_reboot>destroy</on_reboot>\n <devices>\n <rng model="virtio">\n <backend model="random">/dev/urandom</backend>\n </rng>\n <controller type="scsi" index="0" model="virtio-scsi"/>\n <disk device="disk" type="file">\n <source file="/tmp/libguestfs2EG86U/scratch1.img"/>\n <target dev="sda" bus="scsi"/>\n <driver name="qemu" type="raw" cache="unsafe"/>\n <address type="drive" controller="0" bus="0" target="0" unit="0"/>\n </disk>\n <disk type="file" device="disk">\n <source file="/tmp/libguestfs2EG86U/overlay2.qcow2"/>\n <target dev="sdb" bus="scsi"/>\n <driver name="qemu" type="qcow2" cache="unsafe"/>\n <address type="drive" controller="0" bus="0" target="1" unit="0"/>\n </disk>\n <serial type="unix">\n <source mode="connect" path="/tmp/libguestfsBKI0cV/console.sock"/>\n <target port="0"/>\n </serial>\n <channel type="unix">\n <source mode="connect" path="/tmp/libguestfsBKI0cV/guestfsd.sock"/>\n <target type="virtio" name="org.libguestfs.channel.0"/>\n </channel>\n <controller type="usb" model="none"/>\n <memballoon model="none"/>\n </devices>\n <qemu:commandline>\n <qemu:env name="TMPDIR" value="/var/tmp"/>\n </qemu:commandline>\n</domain>\n
libguestfs: command: run: ls
libguestfs: command: run: \ -a
libguestfs: command: run: \ -l
libguestfs: command: run: \ -R
libguestfs: command: run: \ -Z /var/tmp/.guestfs-0
libguestfs: /var/tmp/.guestfs-0:
libguestfs: total 0
libguestfs: drwxr-xr-x. 3 root root unconfined_u:object_r:user_tmp_t:s0 37 Mar 8 04:03 .
libguestfs: drwxrwxrwt. 5 root root system_u:object_r:tmp_t:s0 121 Mar 8 04:03 ..
libguestfs: drwxr-xr-x. 2 root root unconfined_u:object_r:user_tmp_t:s0 46 Mar 8 04:03 appliance.d
libguestfs: -rw-r--r--. 1 root root unconfined_u:object_r:user_tmp_t:s0 0 Mar 8 04:02 lock
libguestfs:
libguestfs: /var/tmp/.guestfs-0/appliance.d:
libguestfs: total 270108
libguestfs: drwxr-xr-x. 2 root root unconfined_u:object_r:user_tmp_t:s0 46 Mar 8 04:03 .
libguestfs: drwxr-xr-x. 3 root root unconfined_u:object_r:user_tmp_t:s0 37 Mar 8 04:03 ..
libguestfs: -rw-r--r--. 1 root root unconfined_u:object_r:user_tmp_t:s0 5346816 Mar 8 04:03 initrd
libguestfs: -rwxr-xr-x. 1 root root unconfined_u:object_r:user_tmp_t:s0 10333376 Mar 8 04:03 kernel
libguestfs: -rw-r--r--. 1 root root unconfined_u:object_r:user_tmp_t:s0 4294967296 Mar 8 04:03 root
libguestfs: command: run: ls
libguestfs: command: run: \ -a
libguestfs: command: run: \ -l
libguestfs: command: run: \ -Z /tmp/libguestfsBKI0cV
libguestfs: total 4
libguestfs: drwxr-xr-x. 2 root root unconfined_u:object_r:user_tmp_t:s0 47 Mar 8 04:03 .
libguestfs: drwxrwxrwt. 11 root root system_u:object_r:tmp_t:s0 4096 Mar 8 04:03 ..
libguestfs: srw-rw----. 1 root qemu unconfined_u:object_r:user_tmp_t:s0 0 Mar 8 04:03 console.sock
libguestfs: srw-rw----. 1 root qemu unconfined_u:object_r:user_tmp_t:s0 0 Mar 8 04:03 guestfsd.sock
libguestfs: launch libvirt guest
\x1bc\x1b[?7l\x1b[2J\x1b[0mSeaBIOS (version 1.15.0-1.module+el8.6.0+13725+61ae1949)
Machine UUID d987f608-c2f0-44be-bcde-d4ada4d71010
Booting from ROM..\x1bc\x1b[?7l\x1b[2J[ 0.000000] Linux version 4.18.0-369.rt7.154.el8.x86_64 (mockbuild.eng.bos.redhat.com) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-10) (GCC)) #1 SMP PREEMPT_RT Wed Feb 23 15:53:30 EST 2022
[ 0.000000] Command line: panic=1 console=ttyS0 edd=off udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=UUID=3c1e4f94-b6ba-40f9-8f70-e7b0b5e65f4d selinux=0 guestfs_verbose=1 TERM=xterm-256color
...
===== TEST FINISHED OK =====
Supermin ignored the debug RT kernel and use RT kernel instead. The behavior is as expected.
Verified this bug since the test case for this bug has been automated and passed in the latest nightly compose test. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Low: virt:rhel and virt-devel:rhel security, bug fix, and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:7472 |