Description of problem: Fedora CoreOS assembler fails to run prepared supermin appliance during build of Fedora CoreOS on power9 systems(works just fine on Power8 bare metal hosts). Version-Release number of selected component (if applicable): supermin-5.1.20-11.fc31.ppc64le How reproducible: Always on power9 bare metal system Steps to Reproduce: 1. Build Fedora CoreOS on Power9 system(I should be able to provide access to system in which the issue can be reproduced). Unfortunately I don't have yet more reduced reproducer. Actual results: qemu-system-ppc64 -accel kvm -smp 16 -vga none -nodefaults -nographic -m 2048 -no-reboot -cpu host -kernel ./tmp/supermin.build/kernel -initrd ./tmp/supermin.build/initrd -netdev user,id=eth0,hostname=supermin -device virtio-net-pci,netdev=eth0 -device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x3 -object rng-random,filename=/dev/urandom,id=rng0 -device virtio-rng-pci,rng=rng0 -drive if=none,id=drive-scsi0-0-0-0,snapshot=on,file=./tmp/supermin.build/root -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1 -virtfs local,id=workdir,path=./,security_model=none,mount_tag=workdir -serial stdio -append 'root=/dev/sda console=hvc0 selinux=1 enforcing=0 autorelabel=1' -drive if=virtio,id=target,format=qcow2,file=./tmp/build/fedora-coreos-31.20200110.dev.0-qemu.ppc64le.qcow2.tmp,cache=unsafe SLOF ********************************************************************** QEMU Starting Build Date = Nov 12 2019 00:00:00 FW Version = mockbuild@ release 20191022 Press "s" to enter Open Firmware. Populating /vdevice methods Populating /vdevice/vty@71000000 Populating /vdevice/nvram@71000001 Populating /pci@800000020000000 00 0000 (D) : 1af4 1000 virtio [ net ] 00 0800 (D) : 1af4 1005 legacy-device* 00 1000 (D) : 1af4 1009 virtio [ network ] Populating /pci@800000020000000/virtio-9p@2 00 1800 (D) : 1af4 1004 virtio [ scsi ] Populating /pci@800000020000000/scsi@3 SCSI: Looking for devices 100000000000000 DISK : "QEMU QEMU HARDDISK 2.5+" 00 2000 (D) : 1af4 1001 virtio [ block ] No NVRAM common partition, re-initializing... Scanning USB Using default console: /vdevice/vty@71000000 Detected RAM kernel at 400000 (1e54e58 bytes) Welcome to Open Firmware Copyright (c) 2004, 2017 IBM Corporation All rights reserved. This program and the accompanying materials are made available under the terms of the BSD License available at http://www.opensource.org/licenses/bsd-license.php Booting from memory... OF stdout device is: /vdevice/vty@71000000 Preparing to boot Linux version 5.4.8-200.fc31.ppc64le (mockbuild.fedoraproject.org) (gcc version 9.2.1 20190827 (Red Hat 9.2.1-1) (GCC)) #1 SMP Mon Jan 6 16:29:22 UTC 2020 Detected machine type: 0000000000000101 command line: root=/dev/sda console=hvc0 selinux=1 enforcing=0 autorelabel=1 Max number of cores passed to firmware: 2048 (NR_CPUS = 2048) Calling ibm,client-architecture-support...qemu-system-ppc64: warning: kernel_irqchip allowed but unavailable: IRQ_XIVE capability must be present for KVM Falling back to kernel-irqchip=off SLOF ********************************************************************** QEMU Starting Build Date = Nov 12 2019 00:00:00 FW Version = mockbuild@ release 20191022 Press "s" to enter Open Firmware. Populating /vdevice methods Populating /vdevice/vty@71000000 Populating /vdevice/nvram@71000001 Populating /pci@800000020000000 00 0000 (D) : 1af4 1000 virtio [ net ] 00 0800 (D) : 1af4 1005 legacy-device* 00 1000 (D) : 1af4 1009 virtio [ network ] Populating /pci@800000020000000/virtio-9p@2 00 1800 (D) : 1af4 1004 virtio [ scsi ] Populating /pci@800000020000000/scsi@3 SCSI: Looking for devices 100000000000000 DISK : "QEMU QEMU HARDDISK 2.5+" 00 2000 (D) : 1af4 1001 virtio [ block ] Scanning USB Using default console: /vdevice/vty@71000000 Detected RAM kernel at 400000 (1e54e58 bytes) Welcome to Open Firmware Copyright (c) 2004, 2017 IBM Corporation All rights reserved. This program and the accompanying materials are made available under the terms of the BSD License available at http://www.opensource.org/licenses/bsd-license.php Booting from memory... OF stdout device is: /vdevice/vty@71000000 Preparing to boot Linux version 5.4.8-200.fc31.ppc64le (mockbuild.fedoraproject.org) (gcc version 9.2.1 20190827 (Red Hat 9.2.1-1) (GCC)) #1 SMP Mon Jan 6 16:29:22 UTC 2020 Detected machine type: 0000000000000101 command line: root=/dev/sda console=hvc0 selinux=1 enforcing=0 autorelabel=1 Max number of cores passed to firmware: 2048 (NR_CPUS = 2048) Calling ibm,client-architecture-support...Node not supported Node not supported Node not supported not implemented memory layout at init: memory_limit : 0000000000000000 (16 MB aligned) alloc_bottom : 0000000002270000 alloc_top : 0000000010000000 alloc_top_hi : 0000000080000000 rmo_top : 0000000010000000 ram_top : 0000000080000000 instantiating rtas at 0x000000000fff0000... done prom_hold_cpus: skipped copying OF device tree... Building dt strings... Building dt structure... Device tree strings 0x0000000002580000 -> 0x0000000002580b42 Device tree struct 0x0000000002590000 -> 0x00000000025a0000 Quiescing Open Firmware ... Booting Linux via __start() @ 0x0000000000400000 ... [ 0.000000] radix-mmu: Page sizes from device-tree: [ 0.000000] radix-mmu: Page size shift = 12 AP=0x0 [ 0.000000] radix-mmu: Page size shift = 16 AP=0x5 [ 0.000000] radix-mmu: Page size shift = 21 AP=0x1 [ 0.000000] radix-mmu: Page size shift = 30 AP=0x2 [ 0.000000] radix-mmu: Activating Kernel Userspace Execution Prevention [ 0.000000] radix-mmu: Activating Kernel Userspace Access Prevention [ 0.000000] radix-mmu: Mapped 0x0000000000000000-0x0000000040000000 with 1.00 GiB pages (exec) [ 0.000000] radix-mmu: Mapped 0x0000000040000000-0x0000000080000000 with 1.00 GiB pages [ 0.000000] lpar: Using radix MMU under hypervisor [ 0.000000] Linux version 5.4.8-200.fc31.ppc64le (mockbuild.fedoraproject.org) (gcc version 9.2.1 20190827 (Red Hat 9.2.1-1) (GCC)) #1 SMP Mon Jan 6 16:29:22 UTC 2020 [ 0.000000] Found initrd at 0xc000000002270000:0xc0000000024eba00 [ 0.000000] Using pSeries machine description [ 0.000000] printk: bootconsole [udbg0] enabled [ 0.000000] Partition configured for 16 cpus. [ 0.000000] CPU maps initialized for 1 thread per core [ 0.000000] ----------------------------------------------------- [ 0.000000] phys_mem_size = 0x80000000 [ 0.000000] dcache_bsize = 0x80 [ 0.000000] icache_bsize = 0x80 [ 0.000000] cpu_features = 0x0001c06f8f4f91a7 [ 0.000000] possible = 0x0001fbffcf5fb1a7 [ 0.000000] always = 0x00000003800081a1 [ 0.000000] cpu_user_features = 0xdc0065c2 0xaee00000 [ 0.000000] mmu_features = 0xbc006041 [ 0.000000] firmware_features = 0x00000005455a445f [ 0.000000] vmalloc start = 0xc008000000000000 [ 0.000000] IO start = 0xc00a000000000000 [ 0.000000] vmemmap start = 0xc00c000000000000 [ 0.000000] ----------------------------------------------------- [ 0.000000] numa: NODE_DATA [mem 0x7ff5ef80-0x7ff63fff] [ 0.000000] rfi-flush: fallback displacement flush available [ 0.000000] rfi-flush: ori type flush available [ 0.000000] rfi-flush: mttrig type flush available [ 0.000000] link-stack-flush: software flush enabled. [ 0.000000] count-cache-flush: full software flush sequence enabled. [ 0.000000] stf-barrier: eieio barrier available [ 0.000000] PCI host bridge /pci@800000020000000 ranges: [ 0.000000] IO 0x0000200000000000..0x000020000000ffff -> 0x0000000000000000 [ 0.000000] MEM 0x0000200080000000..0x00002000ffffffff -> 0x0000000080000000 [ 0.000000] MEM 0x0000210000000000..0x000021ffffffffff -> 0x0000210000000000 [ 0.000000] PCI: OF: PROBE_ONLY disabled [ 0.000000] PPC64 nvram contains 65536 bytes [ 0.000000] barrier-nospec: using ORI speculation barrier [ 0.000000] Zone ranges: [ 0.000000] Normal [mem 0x0000000000000000-0x000000007fffffff] [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000000000-0x000000007fffffff] [ 0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x000000007fffffff] [ 0.000000] percpu: Embedded 11 pages/cpu s622872 r0 d98024 u720896 [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 32736 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: root=/dev/sda console=hvc0 selinux=1 enforcing=0 autorelabel=1 [ 0.000000] Dentry cache hash table entries: 262144 (order: 5, 2097152 bytes, linear) [ 0.000000] Inode-cache hash table entries: 131072 (order: 4, 1048576 bytes, linear) [ 0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off [ 0.000000] Memory: 1970560K/2097152K available (16000K kernel code, 3776K rwdata, 3968K rodata, 5504K init, 1747K bss, 126592K reserved, 0K cma-reserved) [ 0.000000] random: get_random_u64 called from __kmem_cache_create+0x58/0x9e0 with crng_init=0 [ 0.000000] SLUB: HWalign=128, Order=0-3, MinObjects=0, CPUs=16, Nodes=1 [ 0.000000] ftrace: allocating 32187 entries in 12 pages [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=2048 to nr_cpu_ids=16. [ 0.000000] Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=16 [ 0.000000] NR_IRQS: 512, nr_irqs: 512, preallocated irqs: 16 [ 0.000000] xive: Using IRQ range [0-f] [ 0.000000] xive: Interrupt handling initialized with spapr backend [ 0.000000] xive: Using priority 6 for all interrupts [ 0.000000] xive: Using 64kB queues [ 0.000001] time_init: 56 bit decrementer (max: 7fffffffffffff) [ 0.000487] clocksource: timebase: mask: 0xffffffffffffffff max_cycles: 0x761537d007, max_idle_ns: 440795202126 ns [ 0.001336] clocksource: timebase mult[1f40000] shift[24] registered [ 0.001884] Console: colour dummy device 80x25 [ 0.002251] printk: console [hvc0] enabled [ 0.002251] printk: console [hvc0] enabled [ 0.002609] printk: bootconsole [udbg0] disabled [ 0.002609] printk: bootconsole [udbg0] disabled [ 0.003038] pid_max: default: 32768 minimum: 301 [ 0.003128] LSM: Security Framework initializing [ 0.003195] Yama: becoming mindful. [ 0.003228] SELinux: Initializing. [ 0.003313] *** VALIDATE SELinux *** [ 0.003354] Mount-cache hash table entries: 8192 (order: 0, 65536 bytes, linear) [ 0.003403] Mountpoint-cache hash table entries: 8192 (order: 0, 65536 bytes, linear) [ 0.003470] *** VALIDATE tmpfs *** [ 0.003622] *** VALIDATE proc *** [ 0.003684] *** VALIDATE cgroup1 *** [ 0.003706] *** VALIDATE cgroup2 *** [ 0.003928] EEH: pSeries platform initialized [ 0.003962] POWER9 performance monitor hardware support registered [ 0.004034] rcu: Hierarchical SRCU implementation. [ 0.004478] smp: Bringing up secondary CPUs ... [ 0.019450] smp: Brought up 1 node, 16 CPUs [ 0.019571] numa: Node 0 CPUs: 0-15 [ 0.019596] Using standard scheduler topology [ 0.020214] devtmpfs: initialized [ 0.021718] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns [ 0.021789] futex hash table entries: 4096 (order: 3, 524288 bytes, linear) [ 0.022477] NET: Registered protocol family 16 [ 0.022789] audit: initializing netlink subsys (disabled) [ 0.023078] audit: type=2000 audit(1578676904.020:1): state=initialized audit_enabled=0 res=1 [ 0.023301] cpuidle: using governor menu [ 0.023452] pstore: Registered nvram as persistent store backend Linux ppc64le #1 SMP Mon Jan 6[ 0.028257] PCI: Probing PCI hardware [ 0.028308] PCI host bridge to bus 0000:00 [ 0.028335] pci_bus 0000:00: root bus resource [io 0x10000-0x1ffff] (bus address [0x0000-0xffff]) [ 0.028387] pci_bus 0000:00: root bus resource [mem 0x200080000000-0x2000ffffffff] (bus address [0x80000000-0xffffffff]) [ 0.028448] pci_bus 0000:00: root bus resource [mem 0x210000000000-0x21ffffffffff] [ 0.028494] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.028549] pci 0000:00:00.0: No hypervisor support for SR-IOV on this device, IOV BARs disabled. [ 0.028694] pci 0000:00:01.0: No hypervisor support for SR-IOV on this device, IOV BARs disabled. [ 0.028836] pci 0000:00:02.0: No hypervisor support for SR-IOV on this device, IOV BARs disabled. [ 0.028978] pci 0000:00:03.0: No hypervisor support for SR-IOV on this device, IOV BARs disabled. [ 0.029118] pci 0000:00:04.0: No hypervisor support for SR-IOV on this device, IOV BARs disabled. [ 0.031152] IOMMU table initialized, virtual merging enabled [ 0.031202] pci 0000:00:00.0: Adding to iommu group 0 [ 0.031248] pci 0000:00:00.0: of_irq_parse_pci: failed with rc=134 [ 0.031296] pci 0000:00:01.0: Adding to iommu group 0 [ 0.031338] pci 0000:00:01.0: of_irq_parse_pci: failed with rc=134 [ 0.031388] pci 0000:00:02.0: Adding to iommu group 0 [ 0.031429] pci 0000:00:02.0: of_irq_parse_pci: failed with rc=134 [ 0.031478] pci 0000:00:03.0: Adding to iommu group 0 [ 0.031519] pci 0000:00:03.0: of_irq_parse_pci: failed with rc=134 [ 0.031568] pci 0000:00:04.0: Adding to iommu group 0 [ 0.031609] pci 0000:00:04.0: of_irq_parse_pci: failed with rc=134 [ 0.031731] pci_bus 0000:00: resource 4 [io 0x10000-0x1ffff] [ 0.031773] pci_bus 0000:00: resource 5 [mem 0x200080000000-0x2000ffffffff] [ 0.031814] pci_bus 0000:00: resource 6 [mem 0x210000000000-0x21ffffffffff] [ 0.032412] EEH: No capable adapters found: recovery disabled. [ 0.034921] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.034979] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.202440] alg: No test for 842 (842-generic) [ 0.202847] alg: No test for 842 (842-scomp) [ 0.212460] random: fast init done [ 0.217633] iommu: Default domain type: Translated [ 0.217855] vgaarb: loaded [ 0.217955] SCSI subsystem initialized [ 0.218463] usbcore: registered new interface driver usbfs [ 0.218511] usbcore: registered new interface driver hub [ 0.218881] usbcore: registered new device driver usb [ 0.218954] pps_core: LinuxPPS API ver. 1 registered [ 0.218995] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti> [ 0.219063] PTP clock support registered [ 0.219746] EDAC MC: Ver: 3.0.0 [ 0.220323] NetLabel: Initializing [ 0.220357] NetLabel: domain hash size = 128 [ 0.220403] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.220464] NetLabel: unlabeled traffic allowed by default [ 0.221050] clocksource: Switched to clocksource timebase [ 0.236610] *** VALIDATE bpf *** [ 0.236970] VFS: Disk quotas dquot_6.6.0 [ 0.237024] VFS: Dquot-cache hash table entries: 8192 (order 0, 65536 bytes) [ 0.237082] *** VALIDATE ramfs *** [ 0.237109] *** VALIDATE hugetlbfs *** [ 0.248797] thermal_sys: Registered thermal governor 'fair_share' [ 0.248798] thermal_sys: Registered thermal governor 'step_wise' [ 0.248901] NET: Registered protocol family 2 [ 0.249071] tcp_listen_portaddr_hash hash table entries: 4096 (order: 0, 65536 bytes, linear) [ 0.249128] TCP established hash table entries: 16384 (order: 1, 131072 bytes, linear) [ 0.249193] TCP bind hash table entries: 16384 (order: 2, 262144 bytes, linear) [ 0.249255] TCP: Hash tables configured (established 16384 bind 16384) [ 0.249318] UDP hash table entries: 2048 (order: 0, 65536 bytes, linear) [ 0.249365] UDP-Lite hash table entries: 2048 (order: 0, 65536 bytes, linear) [ 0.249456] NET: Registered protocol family 1 [ 0.249492] NET: Registered protocol family 44 [ 0.249560] PCI: CLS 0 bytes, default 128 [ 0.249617] Trying to unpack rootfs image as initramfs... [ 0.250340] Freeing initrd memory: 2496K [ 0.250454] rtas_flash: no firmware flash support [ 0.250958] Initialise system trusted keyrings [ 0.250999] Key type blacklist registered [ 0.251814] workingset: timestamp_bits=38 max_order=15 bucket_order=0 [ 0.253214] zbud: loaded [ 0.263121] NET: Registered protocol family 38 [ 0.263159] Key type asymmetric registered [ 0.263186] Asymmetric key parser 'x509' registered [ 0.263225] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 245) [ 0.264074] io scheduler mq-deadline registered [ 0.264114] io scheduler kyber registered [ 0.264169] io scheduler bfq registered [ 0.267145] atomic64_test: passed [ 0.267209] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 0.267296] virtio-pci 0000:00:00.0: enabling device (0000 -> 0003) [ 0.267359] virtio-pci 0000:00:00.0: virtio_pci: leaving for legacy driver [ 0.267410] virtio-pci 0000:00:00.0: ibm,query-pe-dma-windows(2026) 0 1000 0 returned -3 [ 0.267531] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) [ 0.267591] virtio-pci 0000:00:01.0: virtio_pci: leaving for legacy driver [ 0.267696] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) [ 0.267758] virtio-pci 0000:00:02.0: virtio_pci: leaving for legacy driver [ 0.267862] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) [ 0.267924] virtio-pci 0000:00:03.0: virtio_pci: leaving for legacy driver [ 0.268032] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) [ 0.268094] virtio-pci 0000:00:04.0: virtio_pci: leaving for legacy driver [ 0.270092] Serial: 8250/16550 driver, 32 ports, IRQ sharing enabled [ 0.275391] Non-volatile memory driver v1.3 [ 0.275601] virtio_rng: probe of virtio1 failed with error -22 [ 0.275986] libphy: Fixed MDIO Bus: probed [ 0.276196] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver [ 0.276247] ehci-pci: EHCI PCI platform driver [ 0.276288] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver [ 0.276335] ohci-pci: OHCI PCI platform driver [ 0.276376] uhci_hcd: USB Universal Host Controller Interface driver [ 0.276449] usbcore: registered new interface driver usbserial_generic [ 0.276496] usbserial: USB Serial support registered for generic [ 0.276696] mousedev: PS/2 mouse device common for all mice [ 0.276920] rtc-generic rtc-generic: registered as rtc0 [ 0.276997] device-mapper: uevent: version 1.0.3 [ 0.277199] device-mapper: ioctl: 4.41.0-ioctl (2019-09-16) initialised: dm-devel [ 0.278250] hidraw: raw HID events driver (C) Jiri Kosina [ 0.278315] usbcore: registered new interface driver usbhid [ 0.278349] usbhid: USB HID core driver [ 0.278600] drop_monitor: Initializing network drop monitor service [ 0.278697] Initializing XFRM netlink socket [ 0.278843] NET: Registered protocol family 10 [ 0.280171] Segment Routing with IPv6 [ 0.280215] mip6: Mobile IPv6 [ 0.280243] NET: Registered protocol family 17 [ 0.280947] drmem: No dynamic reconfiguration memory found [ 0.281331] registered taskstats version 1 [ 0.281412] Loading compiled-in X.509 certificates [ 0.321540] Loaded X.509 cert 'Fedora kernel signing key: 17d6619228eb72d3e5a27c990bdeef850440c07b' [ 0.321665] zswap: loaded using pool lzo/zbud [ 0.322519] Key type ._fscrypt registered [ 0.322561] Key type .fscrypt registered [ 0.322692] pstore: Using crash dump compression: deflate [ 0.325938] Key type big_key registered [ 0.328271] Key type encrypted registered [ 0.328315] ima: No TPM chip found, activating TPM-bypass! [ 0.328365] ima: Allocated hash algorithm: sha256 [ 0.328419] ima: No architecture policies found [ 0.329040] rtc-generic rtc-generic: setting system clock to 2020-01-10T17:21:44 UTC (1578676904) [ 0.331406] Freeing unused kernel memory: 5504K [ 0.331454] This architecture does not have kernel memory protection. [ 0.331509] Run /init as init process supermin: mounting /proc supermin: ext2 mini initrd starting up: 5.1.20 glibc supermin: cmdline: root=/dev/sda console=hvc0 selinux=1 enforcing=0 autorelabel=1 supermin: uptime: 0.32 151.65 supermin: mounting /sys supermin: internal insmod crc32c-vpmsum.ko supermin: internal insmod crct10dif-vpmsum.ko supermin: internal insmod crc32_generic.ko supermin: internal insmod virtio_blk.ko [ 0.446112] virtio_blk: probe of virtio4 failed with error -22 supermin: internal insmod virtio_console.ko supermin: internal insmod crypto_engine.ko supermin: internal insmod virtio_crypto.ko supermin: internal insmod failover.ko supermin: internal insmod net_failover.ko supermin: internal insmod virtio_net.ko [ 0.501110] virtio_net: probe of virtio0 failed with error -22 supermin: internal insmod libnvdimm.ko supermin: internal insmod nd_pmem.ko supermin: internal insmod rpmsg_core.ko supermin: internal insmod virtio_rpmsg_bus.ko supermin: internal insmod virtio_scsi.ko [ 0.561795] virtio_scsi: probe of virtio3 failed with error -22 supermin: internal insmod virtio_balloon.ko supermin: internal insmod virtio_input.ko supermin: internal insmod virtio_mmio.ko supermin: internal insmod fuse.ko [ 0.573292] fuse: init (API version 7.31) [ 0.573362] *** VALIDATE fuse *** [ 0.573401] *** VALIDATE fuse *** supermin: internal insmod virtiofs.ko supermin: internal insmod crc-itu-t.ko supermin: internal insmod crc4.ko supermin: internal insmod crc64.ko supermin: internal insmod crc7.ko supermin: internal insmod crc8.ko supermin: internal insmod libcrc32c.ko supermin: waiting another 1024000000 ns for /sys/block/sda/dev to appear This usually means your kernel doesn't support virtio, or supermin was unable to load some kernel modules (see module loading messages above). supermin: waiting another 2048000000 ns for /sys/block/sda/dev to appear supermin: waiting another 4096000000 ns for /sys/block/sda/dev to appear supermin: waiting another 8192000000 ns for /sys/block/sda/dev to appear supermin: waiting another 16384000000 ns for /sys/block/sda/dev to appear supermin: waiting another 32768000000 ns for /sys/block/sda/dev to appear supermin: waiting another 65536000000 ns for /sys/block/sda/dev to appear supermin: waiting another 131072000000 ns for /sys/block/sda/dev to appear supermin: waiting another 262144000000 ns for /sys/block/sda/dev to appear supermin: creating /dev/root as block special 18:0 supermin: mounting new root on /root mount: /root: No such device or address [ 524.888668] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000100 [ 524.888742] CPU: 0 PID: 1 Comm: init Not tainted 5.4.8-200.fc31.ppc64le #1 [ 524.888800] Call Trace: [ 524.888829] [c00000007a5cfc40] [c000000000f670a4] dump_stack+0xb8/0x104 (unreliable) [ 524.888901] [c00000007a5cfc80] [c00000000016a5dc] panic+0x168/0x40c [ 524.888961] [c00000007a5cfd10] [c000000000173abc] do_exit+0xcac/0xd80 [ 524.889020] [c00000007a5cfdd0] [c000000000173c5c] sys_exit_group+0x5c/0x110 [ 524.889080] [c00000007a5cfe20] [c00000000000b9d0] system_call+0x5c/0x68 Expected results: Applience able to mount the rootfs and coreos assembler finishing. Additional info: lsinitrd ./tmp/supermin.build/initrd Image: ./tmp/supermin.build/initrd: 2.5M ======================================================================== Version: Arguments: dracut modules: ======================================================================== drwxr-xr-x 2 1000 1000 0 Jan 10 18:14 . -rw-r--r-- 1 1000 1000 17486 Jan 10 18:14 crc32c-vpmsum.ko -rw-r--r-- 1 1000 1000 9294 Jan 10 18:14 crc32_generic.ko -rw-r--r-- 1 1000 1000 5446 Jan 10 18:14 crc4.ko -rw-r--r-- 1 1000 1000 7502 Jan 10 18:14 crc64.ko -rw-r--r-- 1 1000 1000 5806 Jan 10 18:14 crc7.ko -rw-r--r-- 1 1000 1000 6246 Jan 10 18:14 crc8.ko -rw-r--r-- 1 1000 1000 6070 Jan 10 18:14 crc-itu-t.ko -rw-r--r-- 1 1000 1000 16142 Jan 10 18:14 crct10dif-vpmsum.ko -rw-r--r-- 1 1000 1000 21702 Jan 10 18:14 crypto_engine.ko -rw-r--r-- 1 1000 1000 16702 Jan 10 18:14 failover.ko -rw-r--r-- 1 1000 1000 280446 Jan 10 18:14 fuse.ko -rwxr-xr-x 1 1000 1000 1280976 Jan 10 18:14 init -rw-r--r-- 1 1000 1000 7766 Jan 10 18:14 libcrc32c.ko -rw-r--r-- 1 1000 1000 390646 Jan 10 18:14 libnvdimm.ko -rw-r--r-- 1 1000 1000 363 Jan 10 18:14 modules -rw-r--r-- 1 1000 1000 9622 Jan 10 18:14 nd_pmem.ko -rw-r--r-- 1 1000 1000 27646 Jan 10 18:14 net_failover.ko -rw-r--r-- 1 1000 1000 26806 Jan 10 18:14 rpmsg_core.ko -rw-r--r-- 1 1000 1000 36134 Jan 10 18:14 virtio_balloon.ko -rw-r--r-- 1 1000 1000 36334 Jan 10 18:14 virtio_blk.ko -rw-r--r-- 1 1000 1000 71574 Jan 10 18:14 virtio_console.ko -rw-r--r-- 1 1000 1000 45662 Jan 10 18:14 virtio_crypto.ko -rw-r--r-- 1 1000 1000 46342 Jan 10 18:14 virtiofs.ko -rw-r--r-- 1 1000 1000 19726 Jan 10 18:14 virtio_input.ko -rw-r--r-- 1 1000 1000 24638 Jan 10 18:14 virtio_mmio.ko -rw-r--r-- 1 1000 1000 109430 Jan 10 18:14 virtio_net.ko -rw-r--r-- 1 1000 1000 35774 Jan 10 18:14 virtio_rpmsg_bus.ko -rw-r--r-- 1 1000 1000 37246 Jan 10 18:14 virtio_scsi.ko ========================================================================
That error is weird: [ 0.561795] virtio_scsi: probe of virtio3 failed with error -22 errno 22 = EINVAL
Similar bug here: https://bugs.launchpad.net/qemu/+bug/1847440 The suggested solution there was to update SLOF (the virtual firmware).
H(In reply to Richard W.M. Jones from comment #2) > Similar bug here: > > https://bugs.launchpad.net/qemu/+bug/1847440 > > The suggested solution there was to update SLOF (the virtual firmware). I have also run build witch used virt-preview repo(https://copr.fedorainfracloud.org/coprs/g/virtmaint-sig/virt-preview/) and I hit same issue. AFAIK it should contain the patched SLOF version(https://copr.fedorainfracloud.org/coprs/g/virtmaint-sig/virt-preview/build/1092953/).
For the record coreos assembler tracking issue https://github.com/coreos/coreos-assembler/issues/1041 .
I have done more testing with different SLOF versions and qemu(with and without virt-preview repo) and it seems that using qemu-slof-20191209 SLOF version with the qemu from F31 works. I'm still looking on what is minimal version of SLOF that will work in this case.
I have done more testing and it seems to be resolved with the most recent SLOF. I have tested it both on p9 and p8 hosts with success. So switching this to SLOF. Could we get rebase to qemu-slof-20191217 in F31?
For the record I have not been able to run coreos assembler(qemu in container) with virt-preview repo enabled on f31(latest kernel) host, with the up mentioned issue/error, both with virt-preview version of SLOF or "mine" latest(qemu-slof-20191217). On power9 bare metal host.
(In reply to Jakub Čajka from comment #7) > For the record I have not been able to run coreos assembler(qemu in > container) with virt-preview repo enabled on f31(latest kernel) host, with > the up mentioned issue/error, both with virt-preview version of SLOF or > "mine" latest(qemu-slof-20191217). On power9 bare metal host. I'm a bit confused, is this an additional problem, or does this invalidate your suggestion to backport rawhide SLOF to f31? I'm happy to do the f31 build if that improves things for you
(In reply to Cole Robinson from comment #8) > (In reply to Jakub Čajka from comment #7) > > For the record I have not been able to run coreos assembler(qemu in > > container) with virt-preview repo enabled on f31(latest kernel) host, with > > the up mentioned issue/error, both with virt-preview version of SLOF or > > "mine" latest(qemu-slof-20191217). On power9 bare metal host. > > I'm a bit confused, is this an additional problem, or does this invalidate > your suggestion to backport rawhide SLOF to f31? > That has been note that I have not been able to run build of coreos/run supermin with the virt-preview repo enabled and using packages from it(regardless on version of SLOF used). Sorry for confusion. > I'm happy to do the f31 build if that improves things for you Thank you. I would very much appreciate that, if that is possible as it resolves issue that I have hit.
I submitted https://bodhi.fedoraproject.org/updates/FEDORA-2020-cf8a0689a8
SLOF-0.1.git20191022-1.fc31 has been pushed to the Fedora 31 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-cf8a0689a8
SLOF-0.1.git20191022-1.fc31 has been pushed to the Fedora 31 stable repository. If problems still persist, please make note of it in this bug report.