Bug 2121141

Summary: [libvirt] Kernel does not provide mount namespace
Product: Red Hat Enterprise Linux 9 Reporter: mhou <mhou>
Component: libvirtAssignee: Michal Privoznik <mprivozn>
libvirt sub component: General QA Contact: zhentang <zhetang>
Status: CLOSED ERRATA Docs Contact:
Severity: high    
Priority: unspecified CC: jdenemar, jsuchane, junzhao, kzhang, lmen, mprivozn, pvlasin, virt-maint, yalzhang
Version: 9.1Keywords: Triaged
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-8.5.0-6.el9 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 2123371 (view as bug list) Environment:
Last Closed: 2022-11-15 10:04:47 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2091421, 2123371    

Description mhou 2022-08-24 15:00:19 UTC
Description of problem:
virtqemud service error when looping copy files to virtual machine and restarting virtual machine

Version-Release number of selected component (if applicable):
kernel version:5.14.0-149.el9.x86_64
libvirt version: libvirt-8.5.0-5.el9.x86_64
qemu version: qemu-kvm-7.0.0-11.el9.x86_64
libguestfs version: 
libguestfs-appliance-1.48.4-2.el9.x86_64
libguestfs-1.48.4-2.el9.x86_64
libguestfs-xfs-1.48.4-2.el9.x86_64

How reproducible: 100%


Steps to Reproduce:
1. define a guest as below:
# cat g1.xml 
<domain type="kvm">
  <name>guest30032</name>
  <uuid>37425e76-af6a-44a6-aba0-73434afe34c0</uuid>
  <memory unit="KiB">8388608</memory>
  <currentMemory unit="KiB">8388608</currentMemory>
  <vcpu placement="static">9</vcpu>
  <cputune>
    <vcpupin vcpu="0" cpuset="2" />
    <vcpupin vcpu="1" cpuset="34" />
    <vcpupin vcpu="2" cpuset="4" />
    <vcpupin vcpu="3" cpuset="36" />
    <vcpupin vcpu="4" cpuset="6" />
    <vcpupin vcpu="5" cpuset="38" />
    <vcpupin vcpu="6" cpuset="8" />
    <vcpupin vcpu="7" cpuset="40" />
    <vcpupin vcpu="8" cpuset="10" />
    <vcpusched vcpus="0" scheduler="fifo" priority="1" />
    <vcpusched vcpus="1" scheduler="fifo" priority="1" />
    <vcpusched vcpus="2" scheduler="fifo" priority="1" />
    <vcpusched vcpus="3" scheduler="fifo" priority="1" />
    <vcpusched vcpus="4" scheduler="fifo" priority="1" />
    <vcpusched vcpus="5" scheduler="fifo" priority="1" />
    <vcpusched vcpus="6" scheduler="fifo" priority="1" />
    <vcpusched vcpus="7" scheduler="fifo" priority="1" />
    <vcpusched vcpus="8" scheduler="fifo" priority="1" />
    <emulatorpin cpuset="0,32" />
  </cputune>
  <numatune>
    <memory mode="strict" nodeset="0" />
    <memnode cellid="0" mode="strict" nodeset="0" />
  </numatune>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch="x86_64" machine="q35">hvm</type>
    <boot dev="hd" />
  </os>
  <features>
    <acpi />
    <pmu state="off" />
    <vmport state="off" />
    <ioapic driver="qemu" />
  </features>
  <cpu mode="host-passthrough" check="none">
    <feature policy="require" name="tsc-deadline" />
    <numa>
      <cell id="0" cpus="0-8" memory="8388608" unit="KiB" memAccess="shared" />
    </numa>
  </cpu>
  <clock offset="utc">
    <timer name="rtc" tickpolicy="catchup" />
    <timer name="pit" tickpolicy="delay" />
    <timer name="hpet" present="no" />
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <pm>
    <suspend-to-mem enabled="no" />
    <suspend-to-disk enabled="no" />
  </pm>
  <devices>
    <emulator>/usr/libexec/qemu-kvm</emulator>
    <disk type="file" device="disk">
      <driver name="qemu" type="qcow2" />
      <source file="/home/guest30032.qcow2" />
      <target dev="vda" bus="virtio" />
      <address type="pci" domain="0x0000" bus="0x01" slot="0x00" function="0x0" />
    </disk>
    <controller type="usb" index="0" model="none" />
    <controller type="pci" index="0" model="pcie-root" />
    <controller type="pci" index="1" model="pcie-root-port">
      <model name="pcie-root-port" />
      <target chassis="1" port="0x10" />
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x0" />
    </controller>
    <controller type="pci" index="2" model="pcie-root-port">
      <model name="pcie-root-port" />
      <target chassis="2" port="0x11" />
      <address type="pci" domain="0x0000" bus="0x00" slot="0x03" function="0x0" />
    </controller>
    <controller type="pci" index="3" model="pcie-root-port">
      <model name="pcie-root-port" />
      <target chassis="3" port="0x8" />
      <address type="pci" domain="0x0000" bus="0x00" slot="0x04" function="0x0" />
    </controller>
    <controller type="pci" index="4" model="pcie-root-port">
      <model name="pcie-root-port" />
      <target chassis="4" port="0x9" />
      <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0" />
    </controller>
    <controller type="pci" index="5" model="pcie-root-port">
      <model name="pcie-root-port" />
      <target chassis="5" port="0xa" />
      <address type="pci" domain="0x0000" bus="0x00" slot="0x06" function="0x0" />
    </controller>
    <controller type="pci" index="6" model="pcie-root-port">
      <model name="pcie-root-port" />
      <target chassis="6" port="0xb" />
      <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0" />
    </controller>
    <controller type="sata" index="0">
      <address type="pci" domain="0x0000" bus="0x00" slot="0x1f" function="0x2" />
    </controller>
    <interface type="bridge">
      <mac address="00:de:ad:01:01:01" />
      <source bridge="virbr0" />
      <model type="virtio" />
      <address type="pci" domain="0x0000" bus="0x02" slot="0x00" function="0x0" />
    </interface>
    <serial type="pty">
      <target type="isa-serial" port="0">
        <model name="isa-serial" />
      </target>
    </serial>
    <console type="pty">
      <target type="serial" port="0" />
    </console>
    <input type="mouse" bus="ps2" />
    <input type="keyboard" bus="ps2" />
    <graphics type="vnc" port="-1" autoport="yes" listen="0.0.0.0">
      <listen type="address" address="0.0.0.0" />
    </graphics>
    <video>
      <model type="cirrus" vram="16384" heads="1" primary="yes" />
      <address type="pci" domain="0x0000" bus="0x05" slot="0x00" function="0x0" />
    </video>
    <memballoon model="virtio">
      <address type="pci" domain="0x0000" bus="0x06" slot="0x00" function="0x0" />
    </memballoon>
    <panic model="isa" />
    <iommu model="intel">
      <driver intremap="on" caching_mode="on" iotlb="on" />
    </iommu>
  </devices>
  <seclabel type="dynamic" model="selinux" relabel="yes" />
</domain>

2. download a rhel9.1 image and rename to guest30032.qcow2
# ll /home/guest30032.qcow2 
-rw-r--r--. 1 qemu hugetlbfs 3802791936 Aug 24 10:52 /home/guest30032.qcow2

3. run script as below
#! /bin/bash
set -x
ssh_guest30032()
{
	sshpass -p redhat ssh -X -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no root.122.123 "$@"
}
for i in $(seq 1 10); do
	echo "====================start ${i} test============================"
	LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1 LIBGUESTFS_BACKEND=direct virt-copy-in -a /home/guest30032.qcow2 /home/21.11-1/ /tmp
	virsh create g1.xml
	virsh reboot guest30032
	TIMEOUT=120; while ! ssh_guest30032 "dracut -f" 2>/dev/null && ((--TIMEOUT)); do sleep 1; done
	virsh reboot guest30032
	virsh destroy guest30032
	echo "====================finish ${i} test==========================="
done

4. check virtqemud status
# systemctl status virtqemud  -l --no-pager
● virtqemud.service - Virtualization qemu daemon
     Loaded: loaded (/usr/lib/systemd/system/virtqemud.service; enabled; vendor preset: enabled)
     Active: active (running) since Wed 2022-08-24 10:45:23 EDT; 8min ago
TriggeredBy: ● virtqemud-admin.socket
             ● virtqemud.socket
             ● virtqemud-ro.socket
       Docs: man:virtqemud(8)
             https://libvirt.org
   Main PID: 2692 (virtqemud)
      Tasks: 20 (limit: 32768)
     Memory: 10.4M
        CPU: 2.497s
     CGroup: /system.slice/virtqemud.service
             └─2692 /usr/sbin/virtqemud --timeout 120

Aug 24 10:51:58 dell-per740-09.knqe.lab.eng.bos.redhat.com virtqemud[2692]: internal error: child reported (status=125): Kernel does not provide mount namespace: No such file or directory
Aug 24 10:51:58 dell-per740-09.knqe.lab.eng.bos.redhat.com virtqemud[2692]: Kernel does not provide mount namespace: No such file or directory
Aug 24 10:51:58 dell-per740-09.knqe.lab.eng.bos.redhat.com virtqemud[2692]: internal error: child reported (status=125): Kernel does not provide mount namespace: No such file or directory
Aug 24 10:51:58 dell-per740-09.knqe.lab.eng.bos.redhat.com virtqemud[2692]: Kernel does not provide mount namespace: No such file or directory
Aug 24 10:51:58 dell-per740-09.knqe.lab.eng.bos.redhat.com virtqemud[2692]: Unable to remove disk metadata on vm guest30032 from /home/guest30032.qcow2 (disk target vda)
Aug 24 10:52:44 dell-per740-09.knqe.lab.eng.bos.redhat.com virtqemud[2692]: internal error: child reported (status=125): Kernel does not provide mount namespace: No such file or directory
Aug 24 10:52:44 dell-per740-09.knqe.lab.eng.bos.redhat.com virtqemud[2692]: Kernel does not provide mount namespace: No such file or directory
Aug 24 10:52:44 dell-per740-09.knqe.lab.eng.bos.redhat.com virtqemud[2692]: internal error: child reported (status=125): Kernel does not provide mount namespace: No such file or directory
Aug 24 10:52:44 dell-per740-09.knqe.lab.eng.bos.redhat.com virtqemud[2692]: Kernel does not provide mount namespace: No such file or directory
Aug 24 10:52:44 dell-per740-09.knqe.lab.eng.bos.redhat.com virtqemud[2692]: Unable to remove disk metadata on vm guest30032 from /home/guest30032.qcow2 (disk target vda)


Actual results:
1. got "ernel does not provide mount namespace: No such file or directory" message on virtqemud service.

Expected results:
1. no more error message on virtqemud

Additional info:

Add the debug output when run step3 as below:
+ LIBGUESTFS_DEBUG=1
+ LIBGUESTFS_TRACE=1
+ LIBGUESTFS_BACKEND=direct
+ virt-copy-in -a /home/guest30032.qcow2 /home/21.11-1/ /tmp
libguestfs: trace: set_verbose true
libguestfs: trace: set_verbose = 0
libguestfs: trace: set_backend "direct"
libguestfs: trace: set_backend = 0
libguestfs: create: flags = 0, handle = 0x562ca7e87200, program = virt-copy-in
libguestfs: trace: set_pgroup true
libguestfs: trace: set_pgroup = 0
libguestfs: trace: add_drive "/home/guest30032.qcow2"
libguestfs: trace: add_drive = 0
libguestfs: trace: is_config
libguestfs: trace: is_config = 1
libguestfs: trace: launch
libguestfs: trace: max_disks
libguestfs: trace: max_disks = 255
libguestfs: trace: get_tmpdir
libguestfs: trace: get_tmpdir = "/tmp"
libguestfs: trace: version
libguestfs: trace: version = <struct guestfs_version = major: 1, minor: 48, release: 4, extra: rhel=9,release=2.el9,libvirt, >
libguestfs: trace: get_backend
libguestfs: trace: get_backend = "direct"
libguestfs: launch: program=virt-copy-in
libguestfs: launch: version=1.48.4rhel=9,release=2.el9,libvirt
libguestfs: launch: backend registered: direct
libguestfs: launch: backend registered: libvirt
libguestfs: launch: backend=direct
libguestfs: launch: tmpdir=/tmp/libguestfs2y510B
libguestfs: launch: umask=0022
libguestfs: launch: euid=0
libguestfs: trace: get_cachedir
libguestfs: trace: get_cachedir = "/var/tmp"
libguestfs: begin building supermin appliance
libguestfs: run supermin
libguestfs: command: run: /usr/bin/supermin
libguestfs: command: run: \ --build
libguestfs: command: run: \ --verbose
libguestfs: command: run: \ --if-newer
libguestfs: command: run: \ --lock /var/tmp/.guestfs-0/lock
libguestfs: command: run: \ --copy-kernel
libguestfs: command: run: \ -f ext2
libguestfs: command: run: \ --host-cpu x86_64
libguestfs: command: run: \ /usr/lib64/guestfs/supermin.d
libguestfs: command: run: \ -o /var/tmp/.guestfs-0/appliance.d
supermin: version: 5.2.1
supermin: rpm: detected RPM version 4.16
supermin: rpm: detected RPM architecture x86_64
supermin: package handler: fedora/rpm
supermin: acquiring lock on /var/tmp/.guestfs-0/lock
supermin: if-newer: output does not need rebuilding
libguestfs: finished building supermin appliance
libguestfs: begin testing qemu features
libguestfs: trace: get_cachedir
libguestfs: trace: get_cachedir = "/var/tmp"
libguestfs: checking for previously cached test results of /usr/libexec/qemu-kvm, in /var/tmp/.guestfs-0
libguestfs: loading previously cached test results
libguestfs: qemu version: 7.0
libguestfs: qemu mandatory locking: yes
libguestfs: qemu KVM: enabled
libguestfs: trace: get_backend_setting "force_tcg"
libguestfs: trace: get_backend_setting = NULL (error)
libguestfs: trace: get_backend_setting "force_kvm"
libguestfs: trace: get_backend_setting = NULL (error)
libguestfs: trace: get_sockdir
libguestfs: trace: get_sockdir = "/tmp"
libguestfs: finished testing qemu features
libguestfs: trace: get_backend_setting "gdb"
libguestfs: trace: get_backend_setting = NULL (error)
/usr/libexec/qemu-kvm \
    -global virtio-blk-pci.scsi=off \
    -no-user-config \
    -nodefaults \
    -display none \
    -machine accel=kvm:tcg,graphics=off \
    -cpu max,la57=off \
    -m 1280 \
    -no-reboot \
    -rtc driftfix=slew \
    -no-hpet \
    -global kvm-pit.lost_tick_policy=discard \
    -kernel /var/tmp/.guestfs-0/appliance.d/kernel \
    -initrd /var/tmp/.guestfs-0/appliance.d/initrd \
    -object rng-random,filename=/dev/urandom,id=rng0 \
    -device virtio-rng-pci,rng=rng0 \
    -device virtio-scsi-pci,id=scsi \
    -drive file=/home/guest30032.qcow2,cache=writeback,id=hd0,if=none \
    -device scsi-hd,drive=hd0 \
    -drive file=/var/tmp/.guestfs-0/appliance.d/root,snapshot=on,id=appliance,cache=unsafe,if=none \
    -device scsi-hd,drive=appliance \
    -device virtio-serial-pci \
    -serial stdio \
    -chardev socket,path=/tmp/libguestfsQPzvBK/guestfsd.sock,id=channel0 \
    -device virtserialport,chardev=channel0,name=org.libguestfs.channel.0 \
    -append "panic=1 console=ttyS0 edd=off udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=UUID=b2f57284-8b7d-44a9-9140-b74131f8061c selinux=0 guestfs_verbose=1 TERM=xterm"
qemu-kvm: Machine type 'pc-i440fx-rhel7.6.0' is deprecated: machine types for previous major releases are deprecated
\x1bc\x1b[?7l\x1b[2J\x1b[0mSeaBIOS (version 1.16.0-4.el9)
Booting from ROM..\x1bc\x1b[?7l\x1b[2J[    0.000000] Linux version 5.14.0-150.rt21.150.el9.x86_64 (mockbuild.eng.bos.redhat.com) (gcc (GCC) 11.3.1 20220421 (Red Hat 11.3.1-2), GNU ld version 2.35.2-24.el9) #1 SMP PREEMPT_RT Sat Aug 20 22:48:16 EDT 2022
[    0.000000] The list of certified hardware and cloud instances for Red Hat Enterprise Linux 9 can be viewed at the Red Hat Ecosystem Catalog, https://catalog.redhat.com.
[    0.000000] Command line: panic=1 console=ttyS0 edd=off udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=UUID=b2f57284-8b7d-44a9-9140-b74131f8061c selinux=0 guestfs_verbose=1 TERM=xterm
[    0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x008: 'MPX bounds registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x010: 'MPX CSR'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'
[    0.000000] x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256
[    0.000000] x86/fpu: xstate_offset[3]:  832, xstate_sizes[3]:   64
[    0.000000] x86/fpu: xstate_offset[4]:  896, xstate_sizes[4]:   64
[    0.000000] x86/fpu: xstate_offset[5]:  960, xstate_sizes[5]:   64
[    0.000000] x86/fpu: xstate_offset[6]: 1024, xstate_sizes[6]:  512
[    0.000000] x86/fpu: xstate_offset[7]: 1536, xstate_sizes[7]: 1024
[    0.000000] x86/fpu: xstate_offset[9]: 2560, xstate_sizes[9]:    8
[    0.000000] x86/fpu: Enabled xstate features 0x2ff, context size is 2568 bytes, using 'compacted' format.
[    0.000000] signal: max sigframe size: 3632
[    0.000000] BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[    0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000004ffdcfff] usable
[    0.000000] BIOS-e820: [mem 0x000000004ffdd000-0x000000004fffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] SMBIOS 2.8 present.
[    0.000000] DMI: Red Hat^C KVM, BIOS 1.16.0-4.el9 04/01/2014
[    0.000000] Hypervisor detected: KVM
[    0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00
[    0.000002] kvm-clock: using sched offset of 232384194 cycles
[    0.000009] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns
[    0.000019] tsc: Detected 2893.202 MHz processor
[    0.000227] last_pfn = 0x4ffdd max_arch_pfn = 0x400000000
[    0.000271] x86/PAT: Configuration [0-7]: WB  WC  UC- UC  WB  WP  UC- WT  
[    0.007122] found SMP MP-table at [mem 0x000f5b50-0x000f5b5f]
[    0.007228] Using GB pages for direct mapping
[    0.007600] RAMDISK: [mem 0x4f886000-0x4ffcffff]
[    0.007686] ACPI: Early table checksum verification disabled
[    0.007716] ACPI: RSDP 0x00000000000F5940 000014 (v00 BOCHS )
[    0.007727] ACPI: RSDT 0x000000004FFE17D1 000030 (v01 BOCHS  BXPC     00000001 BXPC 00000001)
[    0.007739] ACPI: FACP 0x000000004FFE16BD 000074 (v01 BOCHS  BXPC     00000001 BXPC 00000001)
[    0.007748] ACPI: DSDT 0x000000004FFE0040 00167D (v01 BOCHS  BXPC     00000001 BXPC 00000001)
[    0.007751] ACPI: FACS 0x000000004FFE0000 000040
[    0.007753] ACPI: APIC 0x000000004FFE1731 000078 (v01 BOCHS  BXPC     00000001 BXPC 00000001)
[    0.007756] ACPI: WAET 0x000000004FFE17A9 000028 (v01 BOCHS  BXPC     00000001 BXPC 00000001)
[    0.007758] ACPI: Reserving FACP table memory at [mem 0x4ffe16bd-0x4ffe1730]
[    0.007759] ACPI: Reserving DSDT table memory at [mem 0x4ffe0040-0x4ffe16bc]
[    0.007760] ACPI: Reserving FACS table memory at [mem 0x4ffe0000-0x4ffe003f]
[    0.007760] ACPI: Reserving APIC table memory at [mem 0x4ffe1731-0x4ffe17a8]
[    0.007761] ACPI: Reserving WAET table memory at [mem 0x4ffe17a9-0x4ffe17d0]
[    0.008046] No NUMA configuration found
[    0.008048] Faking a node at [mem 0x0000000000000000-0x000000004ffdcfff]
[    0.008063] NODE_DATA(0) allocated [mem 0x4f85b000-0x4f885fff]
[    0.008521] Zone ranges:
[    0.008524]   DMA      [mem 0x0000000000001000-0x0000000000ffffff]
[    0.008526]   DMA32    [mem 0x0000000001000000-0x000000004ffdcfff]
[    0.008529]   Normal   empty
[    0.008529]   Device   empty
[    0.008530] Movable zone start for each node
[    0.008533] Early memory node ranges
[    0.008534]   node   0: [mem 0x0000000000001000-0x000000000009efff]
[    0.008535]   node   0: [mem 0x0000000000100000-0x000000004ffdcfff]
[    0.008536] Initmem setup node 0 [mem 0x0000000000001000-0x000000004ffdcfff]
[    0.008556] On node 0, zone DMA: 1 pages in unavailable ranges
[    0.008723] On node 0, zone DMA: 97 pages in unavailable ranges
[    0.011414] On node 0, zone DMA32: 35 pages in unavailable ranges
[    0.011695] ACPI: PM-Timer IO Port: 0x608
[    0.011715] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
[    0.011757] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23
[    0.011762] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[    0.011763] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
[    0.011767] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[    0.011771] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
[    0.011772] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
[    0.011777] ACPI: Using ACPI (MADT) for SMP configuration information
[    0.011783] TSC deadline timer available
[    0.011792] smpboot: Allowing 1 CPUs, 0 hotplug CPUs
[    0.011864] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]
[    0.011867] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x0009ffff]
[    0.011868] PM: hibernation: Registered nosave memory: [mem 0x000a0000-0x000effff]
[    0.011869] PM: hibernation: Registered nosave memory: [mem 0x000f0000-0x000fffff]
[    0.011872] [mem 0x50000000-0xfeffbfff] available for PCI devices
[    0.011875] Booting paravirtualized kernel on KVM
[    0.011890] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns
[    0.016461] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1
[    0.017878] percpu: Embedded 53 pages/cpu s180224 r8192 d28672 u2097152
[    0.017924] kvm-guest: PV spinlocks disabled, single CPU
[    0.017950] Built 1 zonelists, mobility grouping on.  Total pages: 322269
[    0.017953] Policy zone: DMA32
[    0.017955] Kernel command line: panic=1 console=ttyS0 edd=off udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=UUID=b2f57284-8b7d-44a9-9140-b74131f8061c selinux=0 guestfs_verbose=1 TERM=xterm
[    0.018276] cgroup: Disabling memory control group subsystem
[    0.018353] Unknown kernel command line parameters "edd=off udevtimeout=6000 guestfs_verbose=1", will be passed to user space.
[    0.019660] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[    0.019708] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)
[    0.021072] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.031067] Memory: 260860K/1310188K available (14345K kernel code, 6217K rwdata, 9084K rodata, 2724K init, 7032K bss, 89512K reserved, 0K cma-reserved)
[    0.031085] random: random: get_random_u64 called from kmem_cache_open+0x20/0x2b0 with crng_init=0
[    0.031286] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[    0.031310] ftrace: allocating 43534 entries in 171 pages
[    0.054045] ftrace: allocated 171 pages with 5 groups
[    0.054293] rcu: Preemptible hierarchical RCU implementation.
[    0.054298] rcu: \tRCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=1.
[    0.054300] rcu: \tRCU priority boosting: priority 1 delay 500 ms.
[    0.054301] rcu: \tRCU_SOFTIRQ processing moved to rcuc kthreads.
[    0.054301] \tNo expedited grace period (rcu_normal_after_boot).
[    0.054303] \tTrampoline variant of Tasks RCU enabled.
[    0.054304] \tRude variant of Tasks RCU enabled.
[    0.054304] \tTracing variant of Tasks RCU enabled.
[    0.054304] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.
[    0.054305] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
[    0.058731] NR_IRQS: 524544, nr_irqs: 256, preallocated irqs: 16
[    0.059141] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)
[    0.059151] random: crng init done (trusting CPU's manufacturer)
[    0.059246] Console: colour *CGA 80x25
[    0.059283] printk: console [ttyS0] enabled
[    0.059297] ACPI: Core revision 20211217
[    0.059346] APIC: Switch to symmetric I/O mode setup
[    0.059521] x2apic enabled
[    0.059688] Switched APIC routing to physical x2apic.
[    0.060310] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x29b42d51279, max_idle_ns: 440795263682 ns
[    0.060318] Calibrating delay loop (skipped) preset value.. 5786.40 BogoMIPS (lpj=2893202)
[    0.060320] pid_max: default: 32768 minimum: 301
[    0.060346] LSM: Security Framework initializing
[    0.060354] Yama: becoming mindful.
[    0.060364] LSM support for eBPF active
[    0.060412] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)
[    0.060439] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)
[    0.061312] x86/cpu: User Mode Instruction Prevention (UMIP) activated
[    0.061312] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0
[    0.061312] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0
[    0.061312] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization
[    0.061312] Spectre V2 : Mitigation: Enhanced IBRS
[    0.061312] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch
[    0.061312] RETBleed: Mitigation: Enhanced IBRS
[    0.061312] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier
[    0.061312] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl
[    0.061312] TAA: Mitigation: TSX disabled
[    0.061312] MMIO Stale Data: Vulnerable: Clear CPU buffers attempted, no microcode
[    0.061312] Freeing SMP alternatives memory: 32K
[    0.061312] smpboot: CPU0: Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz (family: 0x6, model: 0x55, stepping: 0x7)
[    0.061312] cblist_init_generic: Setting adjustable number of callback queues.
[    0.061312] cblist_init_generic: Setting shift to 0 and lim to 1.
[    0.061312] cblist_init_generic: Setting shift to 0 and lim to 1.
[    0.061312] cblist_init_generic: Setting shift to 0 and lim to 1.
[    0.061312] Performance Events: Skylake events, full-width counters, Intel PMU driver.
[    0.061312] ... version:                2
[    0.061312] ... bit width:              48
[    0.061312] ... generic registers:      4
[    0.061312] ... value mask:             0000ffffffffffff
[    0.061312] ... max period:             00007fffffffffff
[    0.061312] ... fixed-purpose events:   3
[    0.061312] ... event mask:             000000070000000f
[    0.061312] rcu: Hierarchical SRCU implementation.
[    0.061749] smp: Bringing up secondary CPUs ...
[    0.061751] smp: Brought up 1 node, 1 CPU
[    0.061753] smpboot: Max logical packages: 1
[    0.061754] smpboot: Total of 1 processors activated (5786.40 BogoMIPS)
[    0.073257] node 0 deferred pages initialised in 11ms
[    0.073858] devtmpfs: initialized
[    0.073927] x86/mm: Memory block size: 128MB
[    0.074428] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns
[    0.074433] futex hash table entries: 256 (order: 2, 16384 bytes, linear)
[    0.074515] pinctrl core: initialized pinctrl subsystem
[    0.074736] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[    0.074868] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations
[    0.074881] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
[    0.074886] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
[    0.074910] audit: initializing netlink subsys (disabled)
[    0.075062] thermal_sys: Registered thermal governor 'fair_share'
[    0.075063] thermal_sys: Registered thermal governor 'step_wise'
[    0.075063] thermal_sys: Registered thermal governor 'user_space'
[    0.075071] cpuidle: using governor menu
[    0.075176] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
[    0.075314] PCI: Using configuration type 1 for base access
[    0.076973] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.
[    0.076992] audit: type=2000 audit(1661352764.959:1): state=initialized audit_enabled=0 res=1
[    0.077111] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
[    0.077112] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[    0.077239] cryptd: max_cpu_qlen set to 1000
[    0.077476] ACPI: Added _OSI(Module Device)
[    0.077476] ACPI: Added _OSI(Processor Device)
[    0.077477] ACPI: Added _OSI(3.0 _SCP Extensions)
[    0.077478] ACPI: Added _OSI(Processor Aggregator Device)
[    0.077478] ACPI: Added _OSI(Linux-Dell-Video)
[    0.077479] ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio)
[    0.077480] ACPI: Added _OSI(Linux-HPI-Hybrid-Graphics)
[    0.078168] ACPI: 1 ACPI AML tables successfully acquired and loaded
[    0.078951] ACPI: Interpreter enabled
[    0.078961] ACPI: PM: (supports S0 S5)
[    0.078963] ACPI: Using IOAPIC for interrupt routing
[    0.078977] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[    0.079063] ACPI: Enabled 2 GPEs in block 00 to 0F
[    0.080943] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[    0.080947] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI EDR HPX-Type3]
[    0.080949] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI]
[    0.080957] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge.
[    0.081195] acpiphp: Slot [2] registered
[    0.081211] acpiphp: Slot [3] registered
[    0.081224] acpiphp: Slot [4] registered
[    0.081238] acpiphp: Slot [5] registered
[    0.081251] acpiphp: Slot [6] registered
[    0.081264] acpiphp: Slot [7] registered
[    0.081276] acpiphp: Slot [8] registered
[    0.081288] acpiphp: Slot [9] registered
[    0.081302] acpiphp: Slot [10] registered
[    0.081323] acpiphp: Slot [11] registered
[    0.081338] acpiphp: Slot [12] registered
[    0.081350] acpiphp: Slot [13] registered
[    0.081362] acpiphp: Slot [14] registered
[    0.081374] acpiphp: Slot [15] registered
[    0.081389] acpiphp: Slot [16] registered
[    0.081403] acpiphp: Slot [17] registered
[    0.081415] acpiphp: Slot [18] registered
[    0.081429] acpiphp: Slot [19] registered
[    0.081441] acpiphp: Slot [20] registered
[    0.081453] acpiphp: Slot [21] registered
supermin: mounting /proc
supermin: ext2 mini initrd starting up: 5.2.1 glibc
supermin: cmdline: panic=1 console=ttyS0 edd=off udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=UUID=b2f57284-8b7d-44a9-9140-b74131f8061c selinux=0 guestfs_verbose=1 TERM=xterm
supermin: uptime: 0.41 0.00
supermin: mounting /sys
supermin: internal insmod crc32-pclmul.ko
[    0.081468] acpiphp: Slot [22] registered
[    0.081480] acpiphp: Slot [23] registered
[    0.081492] acpiphp: Slot [24] registered
[    0.081506] acpiphp: Slot [25] registered
[    0.081520] acpiphp: Slot [26] registered
[    0.081532] acpiphp: Slot [27] registered
[    0.081544] acpiphp: Slot [28] registered
[    0.081561] acpiphp: Slot [29] registered
[    0.081574] acpiphp: Slot [30] registered
[    0.081586] acpiphp: Slot [31] registered
[    0.081593] PCI host bridge to bus 0000:00
[    0.081594] pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7 window]
[    0.081596] pci_bus 0000:00: root bus resource [io  0x0d00-0xffff window]
[    0.081597] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]
[    0.081598] pci_bus 0000:00: root bus resource [mem 0x50000000-0xfebfffff window]
[    0.081599] pci_bus 0000:00: root bus resource [mem 0x100000000-0x17fffffff window]
[    0.081600] pci_bus 0000:00: root bus resource [bus 00-ff]
[    0.081693] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000
[    0.082411] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100
[    0.082802] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180
[    0.084159] pci 0000:00:01.1: reg 0x20: [io  0xc0a0-0xc0af]
[    0.084765] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io  0x01f0-0x01f7]
[    0.084767] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io  0x03f6]
[    0.084768] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io  0x0170-0x0177]
[    0.084768] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io  0x0376]
[    0.084935] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000
[    0.085205] pci 0000:00:01.3: quirk: [io  0x0600-0x063f] claimed by PIIX4 ACPI
[    0.085213] pci 0000:00:01.3: quirk: [io  0x0700-0x070f] claimed by PIIX4 SMB
[    0.085473] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00
supermin: internal insmod crc32c-intel.ko
[    0.086082] pci 0000:00:02.0: reg 0x10: [io  0xc080-0xc09f]
[    0.088166] pci 0000:00:02.0: reg 0x20: [mem 0xfebf0000-0xfebf3fff 64bit pref]
[    0.090141] pci 0000:00:03.0: [1af4:1004] type 00 class 0x010000
supermin: internal insmod crct10dif-pclmul.ko
[    0.090660] pci 0000:00:03.0: reg 0x10: [io  0xc000-0xc03f]
[    0.091273] pci 0000:00:03.0: reg 0x14: [mem 0xfebfe000-0xfebfefff]
[    0.092626] pci 0000:00:03.0: reg 0x20: [mem 0xfebf4000-0xfebf7fff 64bit pref]
[    0.094091] pci 0000:00:04.0: [1af4:1003] type 00 class 0x078000
[    0.094747] pci 0000:00:04.0: reg 0x10: [io  0xc040-0xc07f]
supermin: internal insmod crc32_generic.ko
[    0.095317] pci 0000:00:04.0: reg 0x14: [mem 0xfebff000-0xfebfffff]
[    0.097317] pci 0000:00:04.0: reg 0x20: [mem 0xfebf8000-0xfebfbfff 64bit pref]
supermin: internal insmod libnvdimm.ko
supermin: internal insmod nfit.ko
[    0.099633] ACPI: PCI: Interrupt link LNKA configured for IRQ 10
[    0.099712] ACPI: PCI: Interrupt link LNKB configured for IRQ 10
[    0.099781] ACPI: PCI: Interrupt link LNKC configured for IRQ 11
supermin: internal insmod libata.ko
[    0.099850] ACPI: PCI: Interrupt link LNKD configured for IRQ 11
[    0.099886] ACPI: PCI: Interrupt link LNKS configured for IRQ 9
[    0.100053] iommu: Default domain type: Translated 
[    0.100054] iommu: DMA domain TLB invalidation policy: lazy mode 
[    0.100166] SCSI subsystem initialized
[    0.100197] usbcore: USB support disabled
[    0.100230] pps_core: LinuxPPS API ver. 1 registered
[    0.100231] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti>
[    0.100235] PTP clock support registered
[    0.102872] EDAC MC: Ver: 3.0.0
[    0.103180] NetLabel: Initializing
[    0.103181] NetLabel:  domain hash size = 128
[    0.103181] NetLabel:  protocols = UNLABELED CIPSOv4 CALIPSO
[    0.103208] NetLabel:  unlabeled traffic allowed by default
supermin: internal insmod ata_piix.ko
[    0.103209] PCI: Using ACPI for IRQ routing
[    0.103309] vgaarb: loaded
[    0.103491] clocksource: Switched to clocksource kvm-clock
[    0.121416] VFS: Disk quotas dquot_6.6.0
[    0.121433] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.121506] pnp: PnP ACPI init
[    0.121742] pnp: PnP ACPI: found 5 devices
[    0.127428] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
[    0.127488] NET: Registered PF_INET protocol family
[    0.127664] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)
[    0.127994] tcp_listen_portaddr_hash hash table entries: 1024 (order: 3, 49152 bytes, linear)
[    0.128028] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)
[    0.128127] TCP bind hash table entries: 16384 (order: 7, 655360 bytes, linear)
[    0.128539] TCP: Hash tables configured (established 16384 bind 16384)
[    0.128746] MPTCP token hash table entries: 2048 (order: 4, 114688 bytes, linear)
[    0.128763] UDP hash table entries: 1024 (order: 4, 98304 bytes, linear)
[    0.128828] UDP-Lite hash table entries: 1024 (order: 4, 98304 bytes, linear)
[    0.128930] NET: Registered PF_UNIX/PF_LOCAL protocol family
[    0.128938] NET: Registered PF_XDP protocol family
[    0.128943] pci_bus 0000:00: resource 4 [io  0x0000-0x0cf7 window]
[    0.128945] pci_bus 0000:00: resource 5 [io  0x0d00-0xffff window]
[    0.128946] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]
[    0.128947] pci_bus 0000:00: resource 7 [mem 0x50000000-0xfebfffff window]
[    0.128948] pci_bus 0000:00: resource 8 [mem 0x100000000-0x17fffffff window]
[    0.128983] pci 0000:00:01.0: PIIX3: Enabling Passive Release
[    0.128990] pci 0000:00:00.0: Limiting direct PCI/PCI transfers
[    0.128999] pci 0000:00:01.0: Activating ISA DMA hang workarounds
[    0.129031] PCI: CLS 0 bytes, default 64
[    0.129070] Trying to unpack rootfs image as initramfs...
[    0.133349] ACPI: bus type thunderbolt registered
[    0.133436] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x29b42d51279, max_idle_ns: 440795263682 ns
[    0.133846] Initialise system trusted keyrings
[    0.133854] Key type blacklist registered
[    0.135346] workingset: timestamp_bits=36 max_order=19 bucket_order=0
[    0.137492] zbud: loaded
[    0.138605] integrity: Platform Keyring initialized
[    0.146213] Freeing initrd memory: 7464K
[    0.148736] NET: Registered PF_ALG protocol family
[    0.148740] xor: automatically using best checksumming function   avx       
[    0.148741] Key type asymmetric registered
[    0.148742] Asymmetric key parser 'x509' registered
[    0.148742] Running certificate verification selftests
[    0.148787] alg: self-tests disabled
[    0.149339] Loaded X.509 cert 'Certificate verification self-testing key: f58703bb33ce1b73ee02eccdee5b8817518fe3db'
[    0.149781] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 246)
[    0.149817] io scheduler mq-deadline registered
[    0.149818] io scheduler kyber registered
[    0.149853] io scheduler bfq registered
[    0.150398] atomic64_test: passed for x86-64 platform with CX8 and with SSE
[    0.150576] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
[    0.150665] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
[    0.150696] ACPI: button: Power Button [PWRF]
[    0.179559] ACPI: \_SB_.LNKB: Enabled at IRQ 10
[    0.208671] ACPI: \_SB_.LNKC: Enabled at IRQ 11
[    0.237817] ACPI: \_SB_.LNKD: Enabled at IRQ 11
[    0.238505] Serial: 8250/16550 driver, 1 ports, IRQ sharing enabled
[    0.238617] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
[    0.238771] Non-volatile memory driver v1.3
[    0.239181] rdac: device handler registered
[    0.239240] hp_sw: device handler registered
[    0.239241] emc: device handler registered
[    0.239357] alua: device handler registered
[    0.239457] libphy: Fixed MDIO Bus: probed
[    0.239538] usbserial: usb_serial_init - registering generic driver failed
[    0.239543] usbserial: usb_serial_init - returning with error -19
[    0.239570] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12
[    0.240171] serio: i8042 KBD port at 0x60,0x64 irq 1
[    0.240177] serio: i8042 AUX port at 0x60,0x64 irq 12
[    0.240252] mousedev: PS/2 mouse device common for all mice
[    0.240360] rtc_cmos 00:04: RTC can wake from S4
[    0.240591] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1
[    0.241059] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4
[    0.241234] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3
[    0.241523] rtc_cmos 00:04: registered as rtc0
[    0.241564] rtc_cmos 00:04: setting system clock to 2022-08-24T14:52:44 UTC (1661352764)
[    0.241577] rtc_cmos 00:04: alarms up to one day, y3k, 242 bytes nvram
[    0.241587] intel_pstate: CPU model not supported
[    0.241849] hid: raw HID events driver (C) Jiri Kosina
[    0.241884] drop_monitor: Initializing network drop monitor service
[    0.241948] Initializing XFRM netlink socket
[    0.242063] NET: Registered PF_INET6 protocol family
[    0.242350] Segment Routing with IPv6
[    0.242373] NET: Registered PF_PACKET protocol family
[    0.242390] mpls_gso: MPLS GSO support
[    0.242509] No MBM correction factor available
[    0.242513] IPI shorthand broadcast: enabled
[    0.242531] AVX2 version of gcm_enc/dec engaged.
[    0.242544] AES CTR mode by8 optimization enabled
[    0.242708] sched_clock: Marking stable (240914455, 1402345)->(266827235, -24510435)
[    0.242782] printk: console [ttyS0]: printing thread started
[    0.242796] registered taskstats version 1
[    0.242838] Loading compiled-in X.509 certificates
[    0.249851] Loaded X.509 cert 'Red Hat Enterprise Linux kernel signing key: 0469e2fde2b53882fd34a9644c94ba6ff07ed511'
[    0.250120] Loaded X.509 cert 'Red Hat Enterprise Linux Driver Update Program (key 3): bf57f3e87362bc7229d9f465321773dfd1f77a80'
[    0.250409] Loaded X.509 cert 'Red Hat Enterprise Linux kpatch signing key: 4d38fd864ebe18c5f0b72e3852e2014c3a676fc8'
[    0.250477] zswap: loaded using pool lzo/zbud
[    0.252109] page_owner is disabled
[    0.252125] Key type big_key registered
[    0.260340] Key type encrypted registered
[    0.260347] ima: No TPM chip found, activating TPM-bypass!
[    0.260350] Loading compiled-in module X.509 certificates
[    0.260659] Loaded X.509 cert 'Red Hat Enterprise Linux kernel signing key: 0469e2fde2b53882fd34a9644c94ba6ff07ed511'
[    0.260661] ima: Allocated hash algorithm: sha256
[    0.260674] ima: No architecture policies found
[    0.260684] evm: Initialising EVM extended attributes:
[    0.260684] evm: security.selinux
[    0.260685] evm: security.SMACK64 (disabled)
[    0.260686] evm: security.SMACK64EXEC (disabled)
[    0.260686] evm: security.SMACK64TRANSMUTE (disabled)
[    0.260686] evm: security.SMACK64MMAP (disabled)
[    0.260687] evm: security.apparmor (disabled)
[    0.260687] evm: security.ima
[    0.260688] evm: security.capability
[    0.260688] evm: HMAC attrs: 0x1
[    0.383724] Freeing unused decrypted memory: 2036K
[    0.384862] Freeing unused kernel image (initmem) memory: 2724K
[    0.384864] Write protecting the kernel read-only data: 26624k
[    0.386056] Freeing unused kernel image (text/rodata gap) memory: 2036K
[    0.386670] Freeing unused kernel image (rodata/data gap) memory: 1156K
[    0.409031] x86/mm: Checked W+X mappings: passed, no W+X pages found.
[    0.409037] Run /init as init process
[    0.483797] scsi host0: ata_piix
[    0.487676] scsi host1: ata_piix
[    0.487702] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc0a0 irq 14
[    0.487703] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc0a8 irq 15
supermin: internal insmod virtio_blk.ko
supermin: internal insmod virtio_console.ko
supermin: internal insmod failover.ko
supermin: internal insmod net_failover.ko
supermin: internal insmod virtio_net.ko
supermin: internal insmod nd_btt.ko
supermin: internal insmod nd_pmem.ko
supermin: internal insmod t10-pi.ko
supermin: internal insmod sd_mod.ko
supermin: internal insmod virtio_scsi.ko
[    0.667210] scsi host2: Virtio SCSI HBA
[    0.668093] scsi 2:0:0:0: Direct-Access     QEMU     QEMU HARDDISK    2.5+ PQ: 0 ANSI: 5
[    0.668228] scsi 2:0:1:0: Direct-Access     QEMU     QEMU HARDDISK    2.5+ PQ: 0 ANSI: 5
[    0.680619] sd 2:0:0:0: Power-on or device reset occurred
[    0.680781] sd 2:0:0:0: [sda] 104857600 512-byte logical blocks: (53.7 GB/50.0 GiB)
[    0.680800] sd 2:0:0:0: [sda] Write Protect is off
[    0.680891] sd 2:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    0.680900] sd 2:0:1:0: Power-on or device reset occurred
[    0.681153] sd 2:0:1:0: [sdb] 8388608 512-byte logical blocks: (4.29 GB/4.00 GiB)
[    0.681177] sd 2:0:1:0: [sdb] Write Protect is off
[    0.681218] sd 2:0:1:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
supermin: internal insmod virtio_balloon.ko
supermin: internal insmod virtio_dma_buf.ko
supermin: internal insmod virtio_input.ko
supermin: internal insmod virtio_mem.ko
supermin: internal insmod vdpa.ko
supermin: internal insmod virtio_vdpa.ko[    0.684863]  sda: sda1 sda2 sda3
[    0.687490] sd 2:0:1:0: [sdb] Attached SCSI disk
[    0.688486] sd 2:0:0:0: [sda] Attached SCSI disk

supermin: internal insmod jbd2.ko
supermin: internal insmod mbcache.ko
supermin: internal insmod ext4.ko
supermin: internal insmod fuse.ko
supermin: internal insmod virtiofs.ko[    0.719140] fuse: init (API version 7.36)

supermin: internal insmod crc-itu-t.ko
supermin: internal insmod crc7.ko
supermin: internal insmod crc8.ko
supermin: internal insmod libcrc32c.ko
supermin: internal insmod soundcore.ko
supermin: internal insmod snd.ko
supermin: internal insmod snd-timer.ko
supermin: internal insmod snd-pcm.ko
supermin: internal insmod virtio_snd.ko
supermin: picked 8:16 as root device
supermin: creating /dev/root as block special 8:16
supermin: mounting new root on /root
[    0.736272] EXT4-fs (sdb): mounting ext2 file system using the ext4 subsystem
supermin: deleting initramfs files[    0.738157] EXT4-fs (sdb): mounted filesystem without journal. Quota mode: none.

supermin: chroot
Starting /init script ...
+ [[ panic=1 console=ttyS0 edd=off udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=UUID=b2f57284-8b7d-44a9-9140-b74131f8061c selinux=0 guestfs_verbose=1 TERM=xterm == *guestfs_network=1* ]]
+ [[ panic=1 console=ttyS0 edd=off udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=UUID=b2f57284-8b7d-44a9-9140-b74131f8061c selinux=0 guestfs_verbose=1 TERM=xterm == *guestfs_rescue=1* ]]
+ [[ panic=1 console=ttyS0 edd=off udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=UUID=b2f57284-8b7d-44a9-9140-b74131f8061c selinux=0 guestfs_verbose=1 TERM=xterm == *guestfs_noreboot=1* ]]
+ [[ panic=1 console=ttyS0 edd=off udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=UUID=b2f57284-8b7d-44a9-9140-b74131f8061c selinux=0 guestfs_verbose=1 TERM=xterm == *guestfs_boot_analysis=1* ]]
+ mkdir -p /dev/pts /dev/shm
+ mount -t devpts /dev/pts /dev/pts
+ mount -t tmpfs -o mode=1777 shmfs /dev/shm
+ mkdir -p /sysroot
+ mkdir -p /run
+ mount -t tmpfs -o nosuid,size=20%,mode=0755 tmpfs /run
+ mkdir ^C-p /run/lock
+ ln -s ../run/lock /var/lock
+ [[ panic=1 console=ttyS0 edd=off udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=UUID=b2f57284-8b7d-44a9-9140-b74131f8061c selinux=0 guestfs_verbose=1 TERM=xterm == *selinux=1* ]]
+ test -e /etc/mtab
+ ln -s /proc/mounts /etc/mtab
+ test -r /etc/crypto-policies/back-ends/opensslcnf.config
+ test -f /usr/share/crypto-policies/DEFAULT/opensslcnf.txt
+ ln -s /usr/share/crypto-policies/DEFAULT/opensslcnf.txt /etc/crypto-policies/back-ends/opensslcnf.config
+ mkdir -p /run/tmpfiles.d
+ kmod static-nodes --format=tmpfiles --output=/run/tmpfiles.d/kmod.conf
++ od -x -A n
++ dd if=/dev/urandom bs=16 count=1 status=none
+ machine_id=' 1f0b fb7b 8e44 912e ffb3 8b93 1b24 ff1c'
+ echo 1f0bfb7b8e44912effb38b931b24ff1c
+ systemd-tmpfiles --prefix=/dev --prefix=/run --prefix=/var/run --create --boot
^C\x1b[0;1;31m/usr/lib/tmpfiles.d/static-nodes-permissions.conf:17: Failed to resolve group 'kvm'.\x1b[0m
\x1b[0;1;31m/usr/lib/tmpfiles.d/static-nodes-permissions.conf:18: Failed to resolve group 'kvm'.\x1b[0m
\x1b[0;1;31m/usr/lib/tmpfiles.d/static-nodes-permissions.conf:19: Failed to resolve group 'kvm'.\x1b[0m
\x1b[0;1;31m/usr/lib/tmpfiles.d/systemd.conf:11: Failed to resolve group 'utmp'.\x1b[0m
\x1b[0;1;31m/usr/lib/tmpfiles.d/systemd.conf:22: Failed to resolve group 'systemd-journal'.\x1b[0m
\x1b[0;1;31m/usr/lib/tmpfiles.d/systemd.conf:23: Failed to resolve group 'systemd-journal'.\x1b[0m
\x1b[0;1;38;5;185mFailed to parse ACL "default:group:tss:rwx": Invalid argument. Ignoring\x1b[0m
\x1b[0;1;31m/usr/lib/tmpfiles.d/tpm2-tss-fapi.conf:4: Failed to resolve user 'tss': No such process\x1b[0m
\x1b[0;1;38;5;185mFailed to parse ACL "default:group:tss:rwx": Invalid argument. Ignoring\x1b[0m
+ for f in /lib/systemd/systemd-udevd /usr/lib/systemd/systemd-udevd /sbin/udevd /lib/udev/udevd /usr/lib/udev/udevd
+ '[' -x /lib/systemd/systemd-udevd ']'
+ UDEVD=/lib/systemd/systemd-udevd
+ break
+ '[' -z /lib/systemd/systemd-udevd ']'
+ /lib/systemd/systemd-udevd --daemon
Starting version 250-7.el9
+ udevadm trigger
^C^C+ udevadm settle --timeout=600
^C^C[    0.951489] sd 2:0:0:0: Attached scsi generic sg0 type 0
[    0.951726] sd 2:0:1:0: Attached scsi generic sg1 type 0
^C^C^C+ shopt -s nullglob
+ for f in /sys/block/sd*/device/timeout
+ echo 300
+ for f in /sys/block/sd*/device/timeout
+ echo 300
+ for f in /sys/block/{h,s,ub,v}d*/queue/scheduler
+ echo noop
/init: line 119: echo: write error: Invalid argument
+ for f in /sys/block/{h,s,ub,v}d*/queue/scheduler
+ echo noop
/init: line 119: echo: write error: Invalid argument
+ shopt -u nullglob
+ ip addr add 127.0.0.1/8 brd + dev lo scope host
+ ip link set dev lo up
+ test '' = 1
+ mdadm -As --auto=yes --no-degraded
mdadm: No arrays found in config file or automatically
+ mkdir -p /tmp/lvm
+ touch /tmp/lvm/lvm.conf
+ command -v lvmdevices
/sbin/lvmdevices
+ printf 'devices {\n'
+ printf '\tuse_devicesfile = 0\n'
+ printf '}\n'
+ LVM_SYSTEM_DIR=/tmp/lvm
+ export LVM_SYSTEM_DIR
+ lvmetad
/init: line 158: lvmetad: command not found
+ modprobe dm_mod
[    1.083884] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.
[    1.083906] device-mapper: uevent: version 1.0.3
[    1.086269] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel
+ lvm pvscan --cache --activate ay
^C+ mdadm -As --auto=yes --run
mdadm: No arrays found in config file or automatically
+ ldmtool create all
/init: line 168: ldmtool: command not found
+ test 1 = 1
+ test '' '!=' 1
+ uname -a
Linux (none) 5.14.0-150.rt21.150.el9.x86_64 #1 SMP PREEMPT_RT Sat Aug 20 22:48:16 EDT 2022 x86_64 x86_64 x86_64 GNU/Linux
+ ls -lR /dev
/dev:
total 0
crw-r--r-- 1 root root  10, 235 Aug 24 14:52 autofs
drwxr-xr-x 2 root root      140 Aug 24 14:52 block
drwxr-xr-x 2 root root       80 Aug 24 14:52 bsg
drwxr-xr-x 2 root root     2260 Aug 24 14:52 char
crw------- 1 root root   5,   1 Aug 24 14:52 console
drwxr-xr-x 3 root root       60 Aug 24 14:52 cpu
crw------- 1 root root  10, 125 Aug 24 14:52 cpu_dma_latency
crw------- 1 root root  10, 203 Aug 24 14:52 cuse
drwxr-xr-x 6 root root      120 Aug 24 14:52 disk
drwxr-xr-x 2 root root       60 Aug 24 14:52 dma_heap
lrwxrwxrwx 1 root root       13 Aug 24 14:52 fd -> /proc/self/fd
crw-rw-rw- 1 root root   1,   7 Aug 24 14:52 full
crw-rw-rw- 1 root root  10, 229 Aug 24 14:52 fuse
crw------- 1 root root  10, 228 Aug 24 14:52 hpet
crw------- 1 root root  10, 183 Aug 24 14:52 hwrng
drwxr-xr-x 3 root root      200 Aug 24 14:52 input
crw-r--r-- 1 root root   1,  11 Aug 24 14:52 kmsg
crw-rw---- 1 root disk  10, 237 Aug 24 14:52 loop-control
drwxr-xr-x 2 root root       60 Aug 24 14:52 mapper
crw------- 1 root root  10, 227 Aug 24 14:52 mcelog
crw------- 1 root root   1,   1 Aug 24 14:52 mem
drwxr-xr-x 2 root root       60 Aug 24 14:52 net
crw-rw-rw- 1 root root   1,   3 Aug 24 14:52 null
crw------- 1 root root  10, 144 Aug 24 14:52 nvram
crw------- 1 root root   1,   4 Aug 24 14:52 port
crw------- 1 root root 108,   0 Aug 24 14:52 ppp
crw-rw-rw- 1 root root   5,   2 Aug 24 14:52 ptmx
drwxr-xr-x 2 root root        0 Aug 24 14:52 pts
crw-rw-rw- 1 root root   1,   8 Aug 24 14:52 random
crw------- 1 root root  10, 242 Aug 24 14:52 rfkill
lrwxrwxrwx 1 root root        4 Aug 24 14:52 rtc -> rtc0
crw------- 1 root root 250,   0 Aug 24 14:52 rtc0
brw------- 1 root root   8,   0 Aug 24 14:52 sda
brw------- 1 root root   8,   1 Aug 24 14:52 sda1
brw------- 1 root root   8,   2 Aug 24 14:52 sda2
brw------- 1 root root   8,   3 Aug 24 14:52 sda3
brw------- 1 root root   8,  16 Aug 24 14:52 sdb
crw-rw---- 1 root disk  21,   0 Aug 24 14:52 sg0
crw-rw---- 1 root disk  21,   1 Aug 24 14:52 sg1
drwxrwxrwt 2 root root       40 Aug 24 14:52 shm
crw------- 1 root root  10, 231 Aug 24 14:52 snapshot
drwxr-xr-x 2 root root       80 Aug 24 14:52 snd
crw-rw-rw- 1 root root   5,   0 Aug 24 14:52 tty
crw------- 1 root root   4,   0 Aug 24 14:52 tty0
crw------- 1 root root   4,   1 Aug 24 14:52 tty1
crw------- 1 root root   4,  10 Aug 24 14:52 tty10
crw------- 1 root root   4,  11 Aug 24 14:52 tty11
crw------- 1 root root   4,  12 Aug 24 14:52 tty12
crw------- 1 root root   4,  13 Aug 24 14:52 tty13
crw------- 1 root root   4,  14 Aug 24 14:52 tty14
crw------- 1 root root   4,  15 Aug 24 14:52 tty15
crw------- 1 root root   4,  16 Aug 24 14:52 tty16
crw------- 1 root root   4,  17 Aug 24 14:52 tty17
crw------- 1 root root   4,  18 Aug 24 14:52 tty18
crw------- 1 root root   4,  19 Aug 24 14:52 tty19
crw------- 1 root root   4,   2 Aug 24 14:52 tty2
crw------- 1 root root   4,  20 Aug 24 14:52 tty20
crw------- 1 root root   4,  21 Aug 24 14:52 tty21
crw------- 1 root root   4,  22 Aug 24 14:52 tty22
crw------- 1 root root   4,  23 Aug 24 14:52 tty23
crw------- 1 root root   4,  24 Aug 24 14:52 tty24
crw------- 1 root root   4,  25 Aug 24 14:52 tty25
crw------- 1 root root   4,  26 Aug 24 14:52 tty26
crw------- 1 root root   4,  27 Aug 24 14:52 tty27
crw------- 1 root root   4,  28 Aug 24 14:52 tty28
crw------- 1 root root   4,  29 Aug 24 14:52 tty29
crw------- 1 root root   4,   3 Aug 24 14:52 tty3
crw------- 1 root root   4,  30 Aug 24 14:52 tty30
crw------- 1 root root   4,  31 Aug 24 14:52 tty31
crw------- 1 root root   4,  32 Aug 24 14:52 tty32
crw------- 1 root root   4,  33 Aug 24 14:52 tty33
crw------- 1 root root   4,  34 Aug 24 14:52 tty34
crw------- 1 root root   4,  35 Aug 24 14:52 tty35
crw------- 1 root root   4,  36 Aug 24 14:52 tty36
crw------- 1 root root   4,  37 Aug 24 14:52 tty37
crw------- 1 root root   4,  38 Aug 24 14:52 tty38
crw------- 1 root root   4,  39 Aug 24 14:52 tty39
crw------- 1 root root   4,   4 Aug 24 14:52 tty4
crw------- 1 root root   4,  40 Aug 24 14:52 tty40
crw------- 1 root root   4,  41 Aug 24 14:52 tty41
crw------- 1 root root   4,  42 Aug 24 14:52 tty42
crw------- 1 root root   4,  43 Aug 24 14:52 tty43
crw------- 1 root root   4,  44 Aug 24 14:52 tty44
crw------- 1 root root   4,  45 Aug 24 14:52 tty45
crw------- 1 root root   4,  46 Aug 24 14:52 tty46
crw------- 1 root root   4,  47 Aug 24 14:52 tty47
crw------- 1 root root   4,  48 Aug 24 14:52 tty48
crw------- 1 root root   4,  49 Aug 24 14:52 tty49
crw------- 1 root root   4,   5 Aug 24 14:52 tty5
crw------- 1 root root   4,  50 Aug 24 14:52 tty50
crw------- 1 root root   4,  51 Aug 24 14:52 tty51
crw------- 1 root root   4,  52 Aug 24 14:52 tty52
crw------- 1 root root   4,  53 Aug 24 14:52 tty53
crw------- 1 root root   4,  54 Aug 24 14:52 tty54
crw------- 1 root root   4,  55 Aug 24 14:52 tty55
crw------- 1 root root   4,  56 Aug 24 14:52 tty56
crw------- 1 root root   4,  57 Aug 24 14:52 tty57
crw------- 1 root root   4,  58 Aug 24 14:52 tty58
crw------- 1 root root   4,  59 Aug 24 14:52 tty59
crw------- 1 root root   4,   6 Aug 24 14:52 tty6
crw------- 1 root root   4,  60 Aug 24 14:52 tty60
crw------- 1 root root   4,  61 Aug 24 14:52 tty61
crw------- 1 root root   4,  62 Aug 24 14:52 tty62
crw------- 1 root root   4,  63 Aug 24 14:52 tty63
crw------- 1 root root   4,   7 Aug 24 14:52 tty7
crw------- 1 root root   4,   8 Aug 24 14:52 tty8
crw------- 1 root root   4,   9 Aug 24 14:52 tty9
crw------- 1 root root   4,  64 Aug 24 14:52 ttyS0
crw------- 1 root root  10, 126 Aug 24 14:52 udmabuf
crw------- 1 root root  10, 239 Aug 24 14:52 uhid
crw------- 1 root root  10, 223 Aug 24 14:52 uinput
crw-rw-rw- 1 root root   1,   9 Aug 24 14:52 urandom
crw------- 1 root root 243,   0 Aug 24 14:52 usbmon0
crw------- 1 root root   7,   0 Aug 24 14:52 vcs
crw------- 1 root root   7,   1 Aug 24 14:52 vcs1
crw------- 1 root root   7, 128 Aug 24 14:52 vcsa
crw------- 1 root root   7, 129 Aug 24 14:52 vcsa1
crw------- 1 root root   7,  64 Aug 24 14:52 vcsu
crw------- 1 root root   7,  65 Aug 24 14:52 vcsu1
drwxr-xr-x 2 root root       60 Aug 24 14:52 vfio
crw------- 1 root root  10, 127 Aug 24 14:52 vga_arbiter
crw------- 1 root root  10, 137 Aug 24 14:52 vhci
crw-rw-rw- 1 root root  10, 238 Aug 24 14:52 vhost-net
crw-rw-rw- 1 root root  10, 241 Aug 24 14:52 vhost-vsock
drwxr-xr-x 2 root root       60 Aug 24 14:52 virtio-ports
crw------- 1 root root 239,   1 Aug 24 14:52 vport2p1
crw-rw-rw- 1 root root   1,   5 Aug 24 14:52 zero

/dev/block:
total 0
lrwxrwxrwx 1 root root 6 Aug 24 14:52 8:0 -> ../sda
lrwxrwxrwx 1 root root 7 Aug 24 14:52 8:1 -> ../sda1
lrwxrwxrwx 1 root root 6 Aug 24 14:52 8:16 -> ../sdb
lrwxrwxrwx 1 root root 7 Aug 24 14:52 8:2 -> ../sda2
lrwxrwxrwx 1 root root 7 Aug 24 14:52 8:3 -> ../sda3

/dev/bsg:
total 0
crw------- 1 root root 246, 0 Aug 24 14:52 2:0:0:0
crw------- 1 root root 246, 1 Aug 24 14:52 2:0:1:0

/dev/char:
total 0
lrwxrwxrwx 1 root root 18 Aug 24 14:52 10:125 -> ../cpu_dma_latency
lrwxrwxrwx 1 root root 10 Aug 24 14:52 10:126 -> ../udmabuf
lrwxrwxrwx 1 root root 14 Aug 24 14:52 10:127 -> ../vga_arbiter
lrwxrwxrwx 1 root root  8 Aug 24 14:52 10:144 -> ../nvram
lrwxrwxrwx 1 root root  8 Aug 24 14:52 10:183 -> ../hwrng
lrwxrwxrwx 1 root root  9 Aug 24 14:52 10:227 -> ../mcelog
lrwxrwxrwx 1 root root  7 Aug 24 14:52 10:228 -> ../hpet
lrwxrwxrwx 1 root root  7 Aug 24 14:52 10:229 -> ../fuse
lrwxrwxrwx 1 root root 11 Aug 24 14:52 10:231 -> ../snapshot
lrwxrwxrwx 1 root root  9 Aug 24 14:52 10:235 -> ../autofs
lrwxrwxrwx 1 root root 17 Aug 24 14:52 10:236 -> ../mapper/control
lrwxrwxrwx 1 root root 12 Aug 24 14:52 116:33 -> ../snd/timer
lrwxrwxrwx 1 root root 15 Aug 24 14:52 13:32 -> ../input/mouse0
lrwxrwxrwx 1 root root 15 Aug 24 14:52 13:33 -> ../input/mouse1
lrwxrwxrwx 1 root root 13 Aug 24 14:52 13:63 -> ../input/mice
lrwxrwxrwx 1 root root 15 Aug 24 14:52 13:64 -> ../input/event0
lrwxrwxrwx 1 root root 15 Aug 24 14:52 13:65 -> ../input/event1
lrwxrwxrwx 1 root root 15 Aug 24 14:52 13:66 -> ../input/event2
lrwxrwxrwx 1 root root 15 Aug 24 14:52 13:67 -> ../input/event3
lrwxrwxrwx 1 root root  6 Aug 24 14:52 1:1 -> ../mem
lrwxrwxrwx 1 root root  7 Aug 24 14:52 1:11 -> ../kmsg
lrwxrwxrwx 1 root root  7 Aug 24 14:52 1:3 -> ../null
lrwxrwxrwx 1 root root  7 Aug 24 14:52 1:4 -> ../port
lrwxrwxrwx 1 root root  7 Aug 24 14:52 1:5 -> ../zero
lrwxrwxrwx 1 root root  7 Aug 24 14:52 1:7 -> ../full
lrwxrwxrwx 1 root root  9 Aug 24 14:52 1:8 -> ../random
lrwxrwxrwx 1 root root 10 Aug 24 14:52 1:9 -> ../urandom
lrwxrwxrwx 1 root root 12 Aug 24 14:52 202:0 -> ../cpu/0/msr
lrwxrwxrwx 1 root root 14 Aug 24 14:52 203:0 -> ../cpu/0/cpuid
lrwxrwxrwx 1 root root  6 Aug 24 14:52 21:0 -> ../sg0
lrwxrwxrwx 1 root root  6 Aug 24 14:52 21:1 -> ../sg1
lrwxrwxrwx 1 root root 11 Aug 24 14:52 239:1 -> ../vport2p1
lrwxrwxrwx 1 root root 10 Aug 24 14:52 243:0 -> ../usbmon0
lrwxrwxrwx 1 root root 14 Aug 24 14:52 246:0 -> ../bsg/2:0:0:0
lrwxrwxrwx 1 root root 14 Aug 24 14:52 246:1 -> ../bsg/2:0:1:0
lrwxrwxrwx 1 root root  7 Aug 24 14:52 250:0 -> ../rtc0
lrwxrwxrwx 1 root root 18 Aug 24 14:52 251:0 -> ../dma_heap/system
lrwxrwxrwx 1 root root  7 Aug 24 14:52 4:0 -> ../tty0
lrwxrwxrwx 1 root root  7 Aug 24 14:52 4:1 -> ../tty1
lrwxrwxrwx 1 root root  8 Aug 24 14:52 4:10 -> ../tty10
lrwxrwxrwx 1 root root  8 Aug 24 14:52 4:11 -> ../tty11
lrwxrwxrwx 1 root root  8 Aug 24 14:52 4:12 -> ../tty12
lrwxrwxrwx 1 root root  8 Aug 24 14:52 4:13 -> ../tty13
lrwxrwxrwx 1 root root  8 Aug 24 14:52 4:14 -> ../tty14
lrwxrwxrwx 1 root root  8 Aug 24 14:52 4:15 -> ../tty15
lrwxrwxrwx 1 root root  8 Aug 24 14:52 4:16 -> ../tty16
lrwxrwxrwx 1 root root  8 Aug 24 14:52 4:17 -> ../tty17
lrwxrwxrwx 1 root root  8 Aug 24 14:52 4:18 -> ../tty18
lrwxrwxrwx 1 root root  8 Aug 24 14:52 4:19 -> ../tty19
lrwxrwxrwx 1 root root  7 Aug 24 14:52 4:2 -> ../tty2
lrwxrwxrwx 1 root root  8 Aug 24 14:52 4:20 -> ../tty20
lrwxrwxrwx 1 root root  8 Aug 24 14:52 4:21 -> ../tty21
lrwxrwxrwx 1 root root  8 Aug 24 14:52 4:22 -> ../tty22
lrwxrwxrwx 1 root root  8 Aug 24 14:52 4:23 -> ../tty23
lrwxrwxrwx 1 root root  8 Aug 24 14:52 4:24 -> ../tty24
lrwxrwxrwx 1 root root  8 Aug 24 14:52 4:25 -> ../tty25
lrwxrwxrwx 1 root root  8 Aug 24 14:52 4:26 -> ../tty26
lrwxrwxrwx 1 root root  8 Aug 24 14:52 4:27 -> ../tty27
lrwxrwxrwx 1 root root  8 Aug 24 14:52 4:28 -> ../tty28
lrwxrwxrwx 1 root root  8 Aug 24 14:52 4:29 -> ../tty29
lrwxrwxrwx 1 root root  7 Aug 24 14:52 4:3 -> ../tty3
lrwxrwxrwx 1 root root  8 Aug 24 14:52 4:30 -> ../tty30
lrwxrwxrwx 1 root root  8 Aug 24 14:52 4:31 -> ../tty31
lrwxrwxrwx 1 root root  8 Aug 24 14:52 4:32 -> ../tty32
lrwxrwxrwx 1 root root  8 Aug 24 14:52 4:33 -> ../tty33
lrwxrwxrwx 1 root root  8 Aug 24 14:52 4:34 -> ../tty34
lrwxrwxrwx 1 root root  8 Aug 24 14:52 4:35 -> ../tty35
lrwxrwxrwx 1 root root  8 Aug 24 14:52 4:36 -> ../tty36
lrwxrwxrwx 1 root root  8 Aug 24 14:52 4:37 -> ../tty37
lrwxrwxrwx 1 root root  8 Aug 24 14:52 4:38 -> ../tty38
lrwxrwxrwx 1 root root  8 Aug 24 14:52 4:39 -> ../tty39
lrwxrwxrwx 1 root root  7 Aug 24 14:52 4:4 -> ../tty4
lrwxrwxrwx 1 root root  8 Aug 24 14:52

Comment 1 mhou 2022-08-24 15:01:48 UTC
After I interrupt the reproduce.sh and manually run virt-copy-in command. The output as below:
# LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1 LIBGUESTFS_BACKEND=direct virt-copy-in -a /home/guest30032.qcow2 /home/21.11-1/ /tmp
libguestfs: trace: set_verbose true
libguestfs: trace: set_verbose = 0
libguestfs: trace: set_backend "direct"
libguestfs: trace: set_backend = 0
libguestfs: create: flags = 0, handle = 0x5611cc7c0200, program = virt-copy-in
libguestfs: trace: set_pgroup true
libguestfs: trace: set_pgroup = 0
libguestfs: trace: add_drive "/home/guest30032.qcow2"
libguestfs: trace: add_drive = 0
libguestfs: trace: is_config
libguestfs: trace: is_config = 1
libguestfs: trace: launch
libguestfs: trace: max_disks
libguestfs: trace: max_disks = 255
libguestfs: trace: get_tmpdir
libguestfs: trace: get_tmpdir = "/tmp"
libguestfs: trace: version
libguestfs: trace: version = <struct guestfs_version = major: 1, minor: 48, release: 4, extra: rhel=9,release=2.el9,libvirt, >
libguestfs: trace: get_backend
libguestfs: trace: get_backend = "direct"
libguestfs: launch: program=virt-copy-in
libguestfs: launch: version=1.48.4rhel=9,release=2.el9,libvirt
libguestfs: launch: backend registered: direct
libguestfs: launch: backend registered: libvirt
libguestfs: launch: backend=direct
libguestfs: launch: tmpdir=/tmp/libguestfsxE56Xb
libguestfs: launch: umask=0022
libguestfs: launch: euid=0
libguestfs: trace: get_cachedir
libguestfs: trace: get_cachedir = "/var/tmp"
libguestfs: begin building supermin appliance
libguestfs: run supermin
libguestfs: command: run: /usr/bin/supermin
libguestfs: command: run: \ --build
libguestfs: command: run: \ --verbose
libguestfs: command: run: \ --if-newer
libguestfs: command: run: \ --lock /var/tmp/.guestfs-0/lock
libguestfs: command: run: \ --copy-kernel
libguestfs: command: run: \ -f ext2
libguestfs: command: run: \ --host-cpu x86_64
libguestfs: command: run: \ /usr/lib64/guestfs/supermin.d
libguestfs: command: run: \ -o /var/tmp/.guestfs-0/appliance.d
supermin: version: 5.2.1
supermin: rpm: detected RPM version 4.16
supermin: rpm: detected RPM architecture x86_64
supermin: package handler: fedora/rpm
supermin: acquiring lock on /var/tmp/.guestfs-0/lock
supermin: if-newer: output does not need rebuilding
libguestfs: finished building supermin appliance
libguestfs: begin testing qemu features
libguestfs: trace: get_cachedir
libguestfs: trace: get_cachedir = "/var/tmp"
libguestfs: checking for previously cached test results of /usr/libexec/qemu-kvm, in /var/tmp/.guestfs-0
libguestfs: loading previously cached test results
libguestfs: qemu version: 7.0
libguestfs: qemu mandatory locking: yes
libguestfs: qemu KVM: enabled
libguestfs: trace: get_backend_setting "force_tcg"
libguestfs: trace: get_backend_setting = NULL (error)
libguestfs: trace: get_backend_setting "force_kvm"
libguestfs: trace: get_backend_setting = NULL (error)
libguestfs: trace: get_sockdir
libguestfs: trace: get_sockdir = "/tmp"
libguestfs: finished testing qemu features
libguestfs: trace: get_backend_setting "gdb"
libguestfs: trace: get_backend_setting = NULL (error)
/usr/libexec/qemu-kvm \
    -global virtio-blk-pci.scsi=off \
    -no-user-config \
    -nodefaults \
    -display none \
    -machine accel=kvm:tcg,graphics=off \
    -cpu max,la57=off \
    -m 1280 \
    -no-reboot \
    -rtc driftfix=slew \
    -no-hpet \
    -global kvm-pit.lost_tick_policy=discard \
    -kernel /var/tmp/.guestfs-0/appliance.d/kernel \
    -initrd /var/tmp/.guestfs-0/appliance.d/initrd \
    -object rng-random,filename=/dev/urandom,id=rng0 \
    -device virtio-rng-pci,rng=rng0 \
    -device virtio-scsi-pci,id=scsi \
    -drive file=/home/guest30032.qcow2,cache=writeback,id=hd0,if=none \
    -device scsi-hd,drive=hd0 \
    -drive file=/var/tmp/.guestfs-0/appliance.d/root,snapshot=on,id=appliance,cache=unsafe,if=none \
    -device scsi-hd,drive=appliance \
    -device virtio-serial-pci \
    -serial stdio \
    -chardev socket,path=/tmp/libguestfsIkdFu1/guestfsd.sock,id=channel0 \
    -device virtserialport,chardev=channel0,name=org.libguestfs.channel.0 \
    -append "panic=1 console=ttyS0 edd=off udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=UUID=b2f57284-8b7d-44a9-9140-b74131f8061c selinux=0 guestfs_verbose=1 TERM=xterm"
qemu-kvm: -drive file=/home/guest30032.qcow2,cache=writeback,id=hd0,if=none: Failed to get "write" lock
Is another process using the image [/home/guest30032.qcow2]?
libguestfs: error: appliance closed the connection unexpectedly, see earlier error messages
libguestfs: child_cleanup: 0x5611cc7c0200: child process died
libguestfs: sending SIGTERM to process 4987
libguestfs: error: /usr/libexec/qemu-kvm exited with error status 1, see debug messages above
libguestfs: error: guestfs_launch failed, see earlier error messages
libguestfs: trace: launch = -1 (error)
libguestfs: trace: close
libguestfs: closing guestfs handle 0x5611cc7c0200 (state 0)
libguestfs: command: run: rm
libguestfs: command: run: \ -rf /tmp/libguestfsxE56Xb
libguestfs: command: run: rm
libguestfs: command: run: \ -rf /tmp/libguestfsIkdFu1

Comment 2 Jaroslav Suchanek 2022-08-24 17:15:07 UTC
Michal, please have a look. Thanks.

Comment 3 Michal Privoznik 2022-08-25 07:54:37 UTC
I think I know what the problem is. It's fixed by this upstream commit:

https://gitlab.com/libvirt/libvirt/-/commit/589536e75d2af745c8f27134b466b23e4fbe3e95

In fact, I've made a scratch build with that commit backported. mhou, can you give it a try please?

https://kojihub.stream.rdu2.redhat.com/koji/taskinfo?taskID=1415940

Comment 4 mhou 2022-08-25 08:57:17 UTC
Test on libvirt-8.5.0-6.el9_rc.cbdf36bb5e.x86_64, can't hit this issue again.
# cat test.sh 
#! /bin/bash
set -x
ssh_guest30032()
{
	sshpass -p redhat ssh -X -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no root.122.123 "$@"
}
for i in $(seq 1 10); do
	echo "====================start ${i} test============================"
	LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1 LIBGUESTFS_BACKEND=direct virt-copy-in -a /root/guest30032.qcow2 /root/test/ /tmp
	virsh create g1.xml
	virsh reboot guest30032
	TIMEOUT=120; while ! ssh_guest30032 "dracut -f" 2>/dev/null && ((--TIMEOUT)); do sleep 1; done
	virsh reboot guest30032
	virsh destroy guest30032
	echo "====================finish ${i} test==========================="
done

# systemctl status virtqemud --no-pager -l
● virtqemud.service - Virtualization qemu daemon
     Loaded: loaded (/usr/lib/systemd/system/virtqemud.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2022-08-25 04:48:40 EDT; 8min ago
TriggeredBy: ● virtqemud.socket
             ● virtqemud-admin.socket
             ● virtqemud-ro.socket
       Docs: man:virtqemud(8)
             https://libvirt.org
   Main PID: 126146 (virtqemud)
      Tasks: 19 (limit: 32768)
     Memory: 16.5M
        CPU: 1.265s
     CGroup: /system.slice/virtqemud.service
             └─126146 /usr/sbin/virtqemud --timeout 120

Aug 25 04:50:40 dell-per740-60.rhts.eng.pek2.redhat.com virtqemud[126146]: Domain id=2 name='guest30032' uuid=37425e76-af6a-44a6-aba0-73434afe34c0 is tainted: deprecated-config (machine type 'pc-q35-rhel8.6.0')
Aug 25 04:51:52 dell-per740-60.rhts.eng.pek2.redhat.com virtqemud[126146]: Domain id=3 name='guest30032' uuid=37425e76-af6a-44a6-aba0-73434afe34c0 is tainted: deprecated-config (machine type 'pc-q35-rhel8.6.0')
Aug 25 04:52:27 dell-per740-60.rhts.eng.pek2.redhat.com virtqemud[126146]: Domain id=4 name='guest30032' uuid=37425e76-af6a-44a6-aba0-73434afe34c0 is tainted: deprecated-config (machine type 'pc-q35-rhel8.6.0')
Aug 25 04:53:01 dell-per740-60.rhts.eng.pek2.redhat.com virtqemud[126146]: Domain id=5 name='guest30032' uuid=37425e76-af6a-44a6-aba0-73434afe34c0 is tainted: deprecated-config (machine type 'pc-q35-rhel8.6.0')
Aug 25 04:53:36 dell-per740-60.rhts.eng.pek2.redhat.com virtqemud[126146]: Domain id=6 name='guest30032' uuid=37425e76-af6a-44a6-aba0-73434afe34c0 is tainted: deprecated-config (machine type 'pc-q35-rhel8.6.0')
Aug 25 04:54:08 dell-per740-60.rhts.eng.pek2.redhat.com virtqemud[126146]: Domain id=7 name='guest30032' uuid=37425e76-af6a-44a6-aba0-73434afe34c0 is tainted: deprecated-config (machine type 'pc-q35-rhel8.6.0')
Aug 25 04:54:43 dell-per740-60.rhts.eng.pek2.redhat.com virtqemud[126146]: Domain id=8 name='guest30032' uuid=37425e76-af6a-44a6-aba0-73434afe34c0 is tainted: deprecated-config (machine type 'pc-q35-rhel8.6.0')
Aug 25 04:55:17 dell-per740-60.rhts.eng.pek2.redhat.com virtqemud[126146]: Domain id=9 name='guest30032' uuid=37425e76-af6a-44a6-aba0-73434afe34c0 is tainted: deprecated-config (machine type 'pc-q35-rhel8.6.0')
Aug 25 04:55:50 dell-per740-60.rhts.eng.pek2.redhat.com virtqemud[126146]: Domain id=10 name='guest30032' uuid=37425e76-af6a-44a6-aba0-73434afe34c0 is tainted: deprecated-config (machine type 'pc-q35-rhel8.6.0')
Aug 25 04:56:25 dell-per740-60.rhts.eng.pek2.redhat.com virtqemud[126146]: Domain id=11 name='guest30032' uuid=37425e76-af6a-44a6-aba0-73434afe34c0 is tainted: deprecated-config (machine type 'pc-q35-rhel8.6.0')

Comment 5 yalzhang@redhat.com 2022-08-25 10:01:17 UTC
*** Bug 2109460 has been marked as a duplicate of this bug. ***

Comment 6 Michal Privoznik 2022-08-25 10:48:43 UTC
To POST:

https://gitlab.com/redhat/rhel/src/libvirt/-/merge_requests/44

Comment 11 zhentang 2022-09-06 09:10:50 UTC
tested on 
libvirt-8.5.0-6.el9

not hit this issue again

# vi bugtest.sh

#! /bin/bash
set -x
ssh_rhel9-vnc()
{
        sshpass -p redhat ssh -X -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no root.116.105 "$@"
}
for i in $(seq 1 10); do
        echo "====================start ${i} test============================"
        LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1 LIBGUESTFS_BACKEND=direct virt-copy-in -a /var/lib/libvirt/images/nfs/RHEL-9.1.0-20220830.1-x86_64.qcow2 /root/test/ /tmp
        virsh start rhel9-vnc
        virsh reboot rhel9-vnc
        TIMEOUT=120; while ! ssh_rhel9-vnc "dracut -f" 2>/dev/null && ((--TIMEOUT)); do sleep 1; done
        virsh reboot rhel9-vnc
        virsh destroy rhel9-vnc
        echo "====================finish ${i} test==========================="
done

# systemctl status virtqemud
● virtqemud.service - Virtualization qemu daemon
     Loaded: loaded (/usr/lib/systemd/system/virtqemud.service; enabled; vendor preset: enabled)
     Active: active (running) since Mon 2022-09-05 22:30:05 EDT; 6h ago
TriggeredBy: ● virtqemud-ro.socket
             ● virtqemud-admin.socket
             ● virtqemud.socket
       Docs: man:virtqemud(8)
             https://libvirt.org
   Main PID: 784226 (virtqemud)
      Tasks: 22 (limit: 32768)
     Memory: 20.4M
        CPU: 2.877s
     CGroup: /system.slice/virtqemud.service
             └─784226 /usr/sbin/virtqemud --timeout 120

Sep 05 22:30:05 zhetang-rhel9 systemd[1]: Starting Virtualization qemu daemon...
Sep 05 22:30:05 zhetang-rhel9 systemd[1]: Started Virtualization qemu daemon.

Comment 15 errata-xmlrpc 2022-11-15 10:04:47 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Low: libvirt security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:8003