Bug 1999419
Summary: | libguestfs breaks with qemu 6.1 with error "Backing file specified without backing format" | ||
---|---|---|---|
Product: | Red Hat Enterprise Linux 9 | Reporter: | Richard W.M. Jones <rjones> |
Component: | libguestfs | Assignee: | Richard W.M. Jones <rjones> |
Status: | CLOSED ERRATA | QA Contact: | YongkuiGuo <yoguo> |
Severity: | high | Docs Contact: | |
Priority: | high | ||
Version: | 9.0 | CC: | eblake, kkiwi, lersek, mxie, rjones, tyan, tzheng, virt-maint, vwu, xiaodwan, yoguo |
Target Milestone: | rc | Keywords: | Triaged |
Target Release: | --- | ||
Hardware: | Unspecified | ||
OS: | Unspecified | ||
Whiteboard: | |||
Fixed In Version: | libguestfs-1.45.6-14.el9 | Doc Type: | If docs needed, set a value |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2022-05-17 12:28:37 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: | |||
Bug Depends On: | 1998820 | ||
Bug Blocks: |
Description
Richard W.M. Jones
2021-08-31 07:45:07 UTC
QE: To reproduce this you simply need to run "libguestfs-test-tool". In fact it's likely that gating tests will fail since they run this command. I can reproduce this issue on fedora 34 after upgrading the qemu-* related packages to 6.1.0. # libguestfs-test-tool PATH=/root/.local/bin:/root/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin XDG_RUNTIME_DIR=/run/user/0 SELinux: Enforcing guestfs_get_append: (null) guestfs_get_autosync: 1 guestfs_get_backend: libvirt guestfs_get_backend_settings: [] guestfs_get_cachedir: /var/tmp guestfs_get_hv: /usr/bin/qemu-kvm guestfs_get_memsize: 1280 guestfs_get_network: 0 guestfs_get_path: /usr/lib64/guestfs guestfs_get_pgroup: 0 guestfs_get_program: libguestfs-test-tool guestfs_get_recovery_proc: 1 guestfs_get_smp: 1 guestfs_get_sockdir: /tmp guestfs_get_tmpdir: /tmp guestfs_get_trace: 0 guestfs_get_verbose: 1 host_cpu: x86_64 Launching appliance, timeout set to 600 seconds. libguestfs: launch: program=libguestfs-test-tool libguestfs: launch: version=1.45.6fedora=34,release=4.fc34,libvirt libguestfs: launch: backend registered: unix libguestfs: launch: backend registered: uml libguestfs: launch: backend registered: libvirt libguestfs: launch: backend registered: direct libguestfs: launch: backend=libvirt libguestfs: launch: tmpdir=/tmp/libguestfsnnQqqL libguestfs: launch: umask=0022 libguestfs: launch: euid=0 libguestfs: libvirt version = 7000000 (7.0.0) libguestfs: guest random name = guestfs-6o2kjjbcza35qeap libguestfs: connect to libvirt libguestfs: opening libvirt handle: URI = qemu:///system, auth = default+wrapper, flags = 0 libguestfs: successfully opened libvirt handle: conn = 0x55fc440b0030 libguestfs: qemu version (reported by libvirt) = 6001000 (6.1.0) libguestfs: get libvirt capabilities libguestfs: parsing capabilities XML libguestfs: parsing domcapabilities XML libguestfs: build appliance libguestfs: begin building supermin appliance libguestfs: run supermin libguestfs: command: run: /usr/bin/supermin libguestfs: command: run: \ --build libguestfs: command: run: \ --verbose libguestfs: command: run: \ --if-newer libguestfs: command: run: \ --lock /var/tmp/.guestfs-0/lock libguestfs: command: run: \ --copy-kernel libguestfs: command: run: \ -f ext2 libguestfs: command: run: \ --host-cpu x86_64 libguestfs: command: run: \ /usr/lib64/guestfs/supermin.d libguestfs: command: run: \ -o /var/tmp/.guestfs-0/appliance.d supermin: version: 5.2.1 supermin: rpm: detected RPM version 4.16 supermin: rpm: detected RPM architecture x86_64 supermin: package handler: fedora/rpm supermin: acquiring lock on /var/tmp/.guestfs-0/lock supermin: build: /usr/lib64/guestfs/supermin.d supermin: reading the supermin appliance supermin: build: visiting /usr/lib64/guestfs/supermin.d/base.tar.gz type gzip base image (tar) supermin: build: visiting /usr/lib64/guestfs/supermin.d/daemon.tar.gz type gzip base image (tar) supermin: build: visiting /usr/lib64/guestfs/supermin.d/excludefiles type uncompressed excludefiles supermin: build: visiting /usr/lib64/guestfs/supermin.d/hostfiles type uncompressed hostfiles supermin: build: visiting /usr/lib64/guestfs/supermin.d/init.tar.gz type gzip base image (tar) supermin: build: visiting /usr/lib64/guestfs/supermin.d/packages type uncompressed packages supermin: build: visiting /usr/lib64/guestfs/supermin.d/udev-rules.tar.gz type gzip base image (tar) supermin: mapping package names to installed packages supermin: resolving full list of package dependencies supermin: build: 190 packages, including dependencies supermin: build: 36385 files supermin: build: 8237 files, after matching excludefiles supermin: build: 8248 files, after adding hostfiles supermin: build: 8234 files, after removing unreadable files supermin: build: 8256 files, after munging supermin: kernel: looking for kernel using environment variables ... supermin: kernel: looking for kernels in /lib/modules/*/vmlinuz ... supermin: kernel: picked vmlinuz /lib/modules/5.11.12-300.fc34.x86_64/vmlinuz supermin: kernel: kernel_version 5.11.12-300.fc34.x86_64 supermin: kernel: modpath /lib/modules/5.11.12-300.fc34.x86_64 supermin: ext2: creating empty ext2 filesystem '/var/tmp/.guestfs-0/appliance.d.sjxeu1v4/root' supermin: ext2: populating from base image supermin: ext2: copying files from host filesystem supermin: ext2: copying kernel modules supermin: ext2: creating minimal initrd '/var/tmp/.guestfs-0/appliance.d.sjxeu1v4/initrd' supermin: ext2: wrote 33 modules to minimal initrd supermin: renaming /var/tmp/.guestfs-0/appliance.d.sjxeu1v4 to /var/tmp/.guestfs-0/appliance.d libguestfs: finished building supermin appliance libguestfs: command: run: qemu-img libguestfs: command: run: \ create libguestfs: command: run: \ -f qcow2 libguestfs: command: run: \ -o backing_file=/var/tmp/.guestfs-0/appliance.d/root libguestfs: command: run: \ /tmp/libguestfsnnQqqL/overlay2.qcow2 qemu-img: /tmp/libguestfsnnQqqL/overlay2.qcow2: Backing file specified without backing format Detected format of raw.libguestfs: error: qemu-img: /tmp/libguestfsnnQqqL/overlay2.qcow2: qemu-img exited with error status 1, see debug messages above libguestfs: closing guestfs handle 0x55fc440aa660 (state 0) libguestfs: command: run: rm libguestfs: command: run: \ -rf /tmp/libguestfsnnQqqL I am ready to verify this bug, but I found that I can 'not' reproduce this issue with previous libguestfs package on RHEL9. Test with packages: libguestfs-1.45.6-13.el9 qemu-img-6.1.0-1.el9.x86_64 Steps: 1. On RHEL9 bare metal with RHEL-9.0.0-20210914.1 compose $ sudo qemu-img create -f qcow2 -o backing_file=rhel-guest-image-9.0-1316.x86_64.qcow2 tmp.qcow2 qemu-img: tmp.qcow2: Backing file specified without backing format Detected format of qcow2. It is expected from qemu 6.1.0. 2. $ sudo qemu-img create -f qcow2 -o backing_file=rhel-guest-image-9.0-1316.x86_64.qcow2,backing_fmt=qcow2 tmp.qcow2 Formatting 'tmp.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=10737418240 backing_file=rhel-guest-image-9.0-1316.x86_64.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 It works when the format of the backing file is specified. 3. $ libguestfs-test-tool ... libguestfs: launch: program=libguestfs-test-tool libguestfs: launch: version=1.45.6rhel=9,release=13.el9,libvirt libguestfs: launch: backend registered: direct libguestfs: launch: backend registered: libvirt libguestfs: launch: backend registered: uml libguestfs: launch: backend registered: unix libguestfs: launch: backend=libvirt libguestfs: launch: tmpdir=/tmp/libguestfsZlCpSC libguestfs: launch: umask=0002 libguestfs: launch: euid=1002 libguestfs: libvirt version = 7006000 (7.6.0) libguestfs: guest random name = guestfs-zbx0wr9ks6slyjt7 libguestfs: connect to libvirt libguestfs: opening libvirt handle: URI = qemu:///session, auth = default+wrapper, flags = 0 libguestfs: successfully opened libvirt handle: conn = 0x55b3f9819030 libguestfs: qemu version (reported by libvirt) = 6001000 (6.1.0) libguestfs: get libvirt capabilities libguestfs: parsing capabilities XML libguestfs: parsing domcapabilities XML libguestfs: build appliance libguestfs: begin building supermin appliance libguestfs: run supermin libguestfs: command: run: /usr/bin/supermin libguestfs: command: run: \ --build libguestfs: command: run: \ --verbose libguestfs: command: run: \ --if-newer libguestfs: command: run: \ --lock /var/tmp/.guestfs-1002/lock libguestfs: command: run: \ --copy-kernel libguestfs: command: run: \ -f ext2 libguestfs: command: run: \ --host-cpu x86_64 libguestfs: command: run: \ /usr/lib64/guestfs/supermin.d libguestfs: command: run: \ -o /var/tmp/.guestfs-1002/appliance.d supermin: version: 5.2.1 supermin: rpm: detected RPM version 4.16 supermin: rpm: detected RPM architecture x86_64 supermin: package handler: fedora/rpm supermin: acquiring lock on /var/tmp/.guestfs-1002/lock supermin: build: /usr/lib64/guestfs/supermin.d supermin: reading the supermin appliance supermin: build: visiting /usr/lib64/guestfs/supermin.d/base.tar.gz type gzip base image (tar) supermin: build: visiting /usr/lib64/guestfs/supermin.d/daemon.tar.gz type gzip base image (tar) supermin: build: visiting /usr/lib64/guestfs/supermin.d/excludefiles type uncompressed excludefiles supermin: build: visiting /usr/lib64/guestfs/supermin.d/hostfiles type uncompressed hostfiles supermin: build: visiting /usr/lib64/guestfs/supermin.d/init.tar.gz type gzip base image (tar) supermin: build: visiting /usr/lib64/guestfs/supermin.d/packages type uncompressed packages supermin: build: visiting /usr/lib64/guestfs/supermin.d/udev-rules.tar.gz type gzip base image (tar) supermin: mapping package names to installed packages supermin: resolving full list of package dependencies supermin: build: 176 packages, including dependencies supermin: build: 34612 files supermin: build: 8003 files, after matching excludefiles supermin: build: 8016 files, after adding hostfiles supermin: build: 8003 files, after removing unreadable files supermin: build: 8029 files, after munging supermin: kernel: looking for kernel using environment variables ... supermin: kernel: looking for kernels in /lib/modules/*/vmlinuz ... supermin: kernel: picked vmlinuz /lib/modules/5.14.0-1.el9.x86_64/vmlinuz supermin: kernel: kernel_version 5.14.0-1.el9.x86_64 supermin: kernel: modpath /lib/modules/5.14.0-1.el9.x86_64 supermin: ext2: creating empty ext2 filesystem '/var/tmp/.guestfs-1002/appliance.d.twe2e0wr/root' supermin: ext2: populating from base image supermin: ext2: copying files from host filesystem supermin: warning: /usr/libexec/utempter/utempter: Permission denied (ignored) Some distro files are not public readable, so supermin cannot copy them into the appliance. This is a problem with your Linux distro. Please ask your distro to stop doing pointless security by obscurity. You can ignore these warnings. You *do not* need to use sudo. supermin: warning: /usr/sbin/unix_update: Permission denied (ignored) supermin: warning: /var/lib/systemd/random-seed: Permission denied (ignored) supermin: ext2: copying kernel modules supermin: warning: /lib/modules/5.14.0-1.el9.x86_64/System.map: Permission denied (ignored) supermin: ext2: creating minimal initrd '/var/tmp/.guestfs-1002/appliance.d.twe2e0wr/initrd' supermin: ext2: wrote 37 modules to minimal initrd supermin: renaming /var/tmp/.guestfs-1002/appliance.d.twe2e0wr to /var/tmp/.guestfs-1002/appliance.d libguestfs: finished building supermin appliance libguestfs: command: run: qemu-img libguestfs: command: run: \ create libguestfs: command: run: \ -f qcow2 libguestfs: command: run: \ -o backing_file=/var/tmp/.guestfs-1002/appliance.d/root,backing_fmt=raw libguestfs: command: run: \ /tmp/libguestfsZlCpSC/overlay2.qcow2 Formatting '/tmp/libguestfsZlCpSC/overlay2.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=4294967296 backing_file=/var/tmp/.guestfs-1002/appliance.d/root backing_fmt=raw lazy_refcounts=off refcount_bits=16 ... ===== TEST FINISHED OK ===== See the above output, the 'backing_fmt=raw' is added in qemu-img command. rjones, could you help look into it? Thanks. The reason is because Fedora (only) uses: ./configure --enable-appliance-format-auto This causes APPLIANCE_FORMAT_AUTO to be defined. This causes the following path to be taken in the code: https://github.com/libguestfs/libguestfs/blob/f47e0bb6725434778384cf79ba3b08610f8c3796/lib/launch-libvirt.c#L500 The reason for setting this in Fedora was because of bug 1967166 which is (another) workaround for containers being broken. We don't have that change in RHEL which means this probably wasn't broken in RHEL, but it's good to have the patch anyway. (In reply to Richard W.M. Jones from comment #6) > The reason is because Fedora (only) uses: > > ./configure --enable-appliance-format-auto > > This causes APPLIANCE_FORMAT_AUTO to be defined. > > This causes the following path to be taken in the code: > > https://github.com/libguestfs/libguestfs/blob/ > f47e0bb6725434778384cf79ba3b08610f8c3796/lib/launch-libvirt.c#L500 > > The reason for setting this in Fedora was because of bug 1967166 > which is (another) workaround for containers being broken. > > We don't have that change in RHEL which means this probably > wasn't broken in RHEL, but it's good to have the patch anyway. Got it. Thanks a lot. Verified with packages: libguestfs-1.45.6-14.el9 qemu-kvm-6.1.0-1.el9 Steps: 1. On RHEL9 bare metal with RHEL-9.0.0-20210914.1 compose $ libguestfs-test-tool ... Launching appliance, timeout set to 600 seconds. libguestfs: launch: program=libguestfs-test-tool libguestfs: launch: version=1.45.6rhel=9,release=14.el9,libvirt libguestfs: launch: backend registered: direct libguestfs: launch: backend registered: libvirt libguestfs: launch: backend registered: uml libguestfs: launch: backend registered: unix libguestfs: launch: backend=libvirt libguestfs: launch: tmpdir=/tmp/libguestfshWmNDI libguestfs: launch: umask=0002 libguestfs: launch: euid=1002 libguestfs: libvirt version = 7006000 (7.6.0) libguestfs: guest random name = guestfs-wkuugt5932y5hfea libguestfs: connect to libvirt libguestfs: opening libvirt handle: URI = qemu:///session, auth = default+wrapper, flags = 0 libguestfs: successfully opened libvirt handle: conn = 0x557e02b40030 libguestfs: qemu version (reported by libvirt) = 6001000 (6.1.0) libguestfs: get libvirt capabilities libguestfs: parsing capabilities XML libguestfs: parsing domcapabilities XML libguestfs: build appliance libguestfs: begin building supermin appliance libguestfs: run supermin libguestfs: command: run: /usr/bin/supermin libguestfs: command: run: \ --build libguestfs: command: run: \ --verbose libguestfs: command: run: \ --if-newer libguestfs: command: run: \ --lock /var/tmp/.guestfs-1002/lock libguestfs: command: run: \ --copy-kernel libguestfs: command: run: \ -f ext2 libguestfs: command: run: \ --host-cpu x86_64 libguestfs: command: run: \ /usr/lib64/guestfs/supermin.d libguestfs: command: run: \ -o /var/tmp/.guestfs-1002/appliance.d supermin: version: 5.2.1 supermin: rpm: detected RPM version 4.16 supermin: rpm: detected RPM architecture x86_64 supermin: package handler: fedora/rpm supermin: acquiring lock on /var/tmp/.guestfs-1002/lock supermin: build: /usr/lib64/guestfs/supermin.d supermin: reading the supermin appliance supermin: build: visiting /usr/lib64/guestfs/supermin.d/base.tar.gz type gzip base image (tar) supermin: build: visiting /usr/lib64/guestfs/supermin.d/daemon.tar.gz type gzip base image (tar) supermin: build: visiting /usr/lib64/guestfs/supermin.d/excludefiles type uncompressed excludefiles supermin: build: visiting /usr/lib64/guestfs/supermin.d/hostfiles type uncompressed hostfiles supermin: build: visiting /usr/lib64/guestfs/supermin.d/init.tar.gz type gzip base image (tar) supermin: build: visiting /usr/lib64/guestfs/supermin.d/packages type uncompressed packages supermin: build: visiting /usr/lib64/guestfs/supermin.d/udev-rules.tar.gz type gzip base image (tar) supermin: mapping package names to installed packages supermin: resolving full list of package dependencies supermin: build: 176 packages, including dependencies supermin: build: 34612 files supermin: build: 8003 files, after matching excludefiles supermin: build: 8016 files, after adding hostfiles supermin: build: 8003 files, after removing unreadable files supermin: build: 8029 files, after munging supermin: kernel: looking for kernel using environment variables ... supermin: kernel: looking for kernels in /lib/modules/*/vmlinuz ... supermin: kernel: picked vmlinuz /lib/modules/5.14.0-1.el9.x86_64/vmlinuz supermin: kernel: kernel_version 5.14.0-1.el9.x86_64 supermin: kernel: modpath /lib/modules/5.14.0-1.el9.x86_64 supermin: ext2: creating empty ext2 filesystem '/var/tmp/.guestfs-1002/appliance.d.3742fvkg/root' supermin: ext2: populating from base image supermin: ext2: copying files from host filesystem supermin: warning: /usr/libexec/utempter/utempter: Permission denied (ignored) Some distro files are not public readable, so supermin cannot copy them into the appliance. This is a problem with your Linux distro. Please ask your distro to stop doing pointless security by obscurity. You can ignore these warnings. You *do not* need to use sudo. supermin: warning: /usr/sbin/unix_update: Permission denied (ignored) supermin: warning: /var/lib/systemd/random-seed: Permission denied (ignored) supermin: ext2: copying kernel modules supermin: warning: /lib/modules/5.14.0-1.el9.x86_64/System.map: Permission denied (ignored) supermin: ext2: creating minimal initrd '/var/tmp/.guestfs-1002/appliance.d.3742fvkg/initrd' supermin: ext2: wrote 37 modules to minimal initrd supermin: renaming /var/tmp/.guestfs-1002/appliance.d.3742fvkg to /var/tmp/.guestfs-1002/appliance.d libguestfs: finished building supermin appliance libguestfs: command: run: qemu-img libguestfs: command: run: \ create libguestfs: command: run: \ -f qcow2 libguestfs: command: run: \ -o backing_file=/var/tmp/.guestfs-1002/appliance.d/root,backing_fmt=raw libguestfs: command: run: \ /tmp/libguestfshWmNDI/overlay2.qcow2 Formatting '/tmp/libguestfshWmNDI/overlay2.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=4294967296 backing_file=/var/tmp/.guestfs-1002/appliance.d/root backing_fmt=raw lazy_refcounts=off refcount_bits=16 libguestfs: create libvirt XML libguestfs: libvirt XML:\n<?xml version="1.0"?>\n<domain type="kvm" xmlns:qemu="http://libvirt.org/schemas/domain/qemu/1.0">\n <name>guestfs-wkuugt5932y5hfea</name>\n <memory unit="MiB">1280</memory>\n <currentMemory unit="MiB">1280</currentMemory>\n <cpu mode="maximum"/>\n <vcpu>1</vcpu>\n <clock offset="utc">\n <timer name="rtc" tickpolicy="catchup"/>\n <timer name="pit" tickpolicy="delay"/>\n <timer name="hpet" present="no"/>\n </clock>\n <os>\n <type>hvm</type>\n <kernel>/var/tmp/.guestfs-1002/appliance.d/kernel</kernel>\n <initrd>/var/tmp/.guestfs-1002/appliance.d/initrd</initrd>\n <cmdline>panic=1 console=ttyS0 edd=off udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=UUID=0a0c723b-25c3-4a54-93de-9549747b63a0 selinux=0 guestfs_verbose=1 TERM=xterm-256color</cmdline>\n <bios useserial="yes"/>\n </os>\n <on_reboot>destroy</on_reboot>\n <devices>\n <rng model="virtio">\n <backend model="random">/dev/urandom</backend>\n </rng>\n <controller type="scsi" index="0" model="virtio-scsi"/>\n <disk device="disk" type="file">\n <source file="/tmp/libguestfshWmNDI/scratch1.img"/>\n <target dev="sda" bus="scsi"/>\n <driver name="qemu" type="raw" cache="unsafe"/>\n <address type="drive" controller="0" bus="0" target="0" unit="0"/>\n </disk>\n <disk type="file" device="disk">\n <source file="/tmp/libguestfshWmNDI/overlay2.qcow2"/>\n <target dev="sdb" bus="scsi"/>\n <driver name="qemu" type="qcow2" cache="unsafe"/>\n <address type="drive" controller="0" bus="0" target="1" unit="0"/>\n </disk>\n <serial type="unix">\n <source mode="connect" path="/run/user/0/libguestfsW9y8LT/console.sock"/>\n <target port="0"/>\n </serial>\n <channel type="unix">\n <source mode="connect" path="/run/user/0/libguestfsW9y8LT/guestfsd.sock"/>\n <target type="virtio" name="org.libguestfs.channel.0"/>\n </channel>\n <controller type="usb" model="none"/>\n <memballoon model="none"/>\n </devices>\n <qemu:commandline>\n <qemu:env name="TMPDIR" value="/var/tmp"/>\n </qemu:commandline>\n</domain>\n ... ===== TEST FINISHED OK ===== Combined with comment 6, verified this bug. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (new packages: libguestfs), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2022:2317 |