Bug 1624539

Summary: qemu-2.11.2-3.fc28 on ppc64le exits with error creating device tree
Product: [Fedora] Fedora Reporter: Kevin Fenzi <kevin>
Component: qemuAssignee: Fedora Virtualization Maintainers <virt-maint>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 28CC: amit, arnd, awilliam, berrange, cfergeau, crobinso, dgibson, dwmw2, itamar, jwboyer, pbonzini, pbrobinson, rjones, virt-maint
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: qemu-2.11.2-4.fc28 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-09-27 17:27:54 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Kevin Fenzi 2018-09-01 03:29:40 UTC 
This is on a fedora 28 koji builder. Trying to build images results in a failure. 

Looking at /var/log/libvirt/qemu/guestfs*log shows: 

2018-09-01 03:20:45.952+0000: starting up libvirt version: 4.1.0, package: 5.fc28 (Fedora Project, 2018-08-23-19:27:33, buildvm-ppc64le-03.ppc.fedoraproject.org), qemu version: 2.11.2(qemu-2.11.2-3.fc28), hostname: buildvm-ppc64le-01.ppc.fedoraproject.org
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin QEMU_AUDIO_DRV=none TMPDIR=/var/tmp /usr/bin/qemu-system-ppc64 -name guest=guestfs-duwbweu1zc6h3kj4,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-2-guestfs-duwbweu1zc6h/master-key.aes -machine pseries-2.11,accel=kvm,usb=off,dump-guest-core=off -cpu host -m 768 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid 059b85a4-83fa-4f54-aa4f-6d13f9866dd4 -display none -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-2-guestfs-duwbweu1zc6h/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -no-reboot -boot strict=on -kernel /var/tmp/.guestfs-0/appliance.d/kernel -initrd /var/tmp/.guestfs-0/appliance.d/initrd -append 'panic=1 console=hvc0 console=ttyS0 edd=off udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=/dev/sdb selinux=0 quiet TERM=linux' -device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x1 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x2 -drive file=/tmp/libguestfsRufj5T/overlay1.qcow2,format=qcow2,if=none,id=drive-scsi0-0-0-0,cache=unsafe -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1 -drive file=/tmp/libguestfsRufj5T/overlay2.qcow2,format=qcow2,if=none,id=drive-scsi0-0-1-0,cache=unsafe -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=1,lun=0,drive=drive-scsi0-0-1-0,id=scsi0-0-1-0 -chardev socket,id=charserial0,path=/tmp/libguestfsayezUy/console.sock -device spapr-vty,chardev=charserial0,id=serial0,reg=0x30000000 -chardev socket,id=charchannel0,path=/tmp/libguestfsayezUy/guestfsd.sock -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.libguestfs.channel.0 -object rng-random,id=objrng0,filename=/dev/urandom -device virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.0,addr=0x3 -msg timestamp=on
2018-09-01 03:20:45.953+0000: Domain id=2 is tainted: custom-argv
2018-09-01 03:20:45.953+0000: Domain id=2 is tainted: host-cpu
2018-09-01T03:20:46.049548Z qemu-system-ppc64: warning: Failed to set KVM's VSMT mode to 8 (errno -22)
2018-09-01T03:20:49.443963Z qemu-system-ppc64: error creating device tree: (fdt_begin_node(fdt_skel, "")): FDT_ERR_BADSTATE

Task info: https://koji.fedoraproject.org/koji/taskinfo?taskID=29407944
Watching tasks (this may be safely interrupted)...
29407944 image (f28, Fedora-Container-Base-20180831, fedora-docker-base.ks): open (buildvm-ppc64le-01.ppc.fedoraproject.org)
  29407945 createImage (f28, Fedora-Container-Base-20180831-20180831.kevin.0, fedora-docker-base.ks, ppc64le): free
  29407945 createImage (f28, Fedora-Container-Base-20180831-20180831.kevin.0, fedora-docker-base.ks, ppc64le): free -> open (buildvm-ppc64le-01.ppc.fedoraproject.org)
  29407945 createImage (f28, Fedora-Container-Base-20180831-20180831.kevin.0, fedora-docker-base.ks, ppc64le): open (buildvm-ppc64le-01.ppc.fedoraproject.org) -> FAILED: ApplianceError: Image status is FAILED: guestfs_launch failed.
This usually means the libguestfs appliance failed to start or crashed.
Do:
  export LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1
and run the command again.  For further information, read:
  http://libguestfs.org/guestfs-faq.1.html#debugging-libguestfs
You can also run 'libguestfs-test-tool' and post the *complete* output
into a bug report or message to the libguestfs mailing list.
  0 free  1 open  0 done  1 failed
29407944 image (f28, Fedora-Container-Base-20180831, fedora-docker-base.ks): open (buildvm-ppc64le-01.ppc.fedoraproject.org) -> FAILED: ApplianceError: Image status is FAILED: guestfs_launch failed.
This usually means the libguestfs appliance failed to start or crashed.
Do:
  export LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1
and run the command again.  For further information, read:
  http://libguestfs.org/guestfs-faq.1.html#debugging-libguestfs
You can also run 'libguestfs-test-tool' and post the *complete* output
into a bug report or message to the libguestfs mailing list.
  0 free  0 open  0 done  2 failed

29407944 image (f28, Fedora-Container-Base-20180831, fedora-docker-base.ks) failed
Comment 1 Kevin Fenzi 2018-09-01 17:47:24 UTC 
The problem package seems to be libfdt-1.4.7-1.fc28. 
Downgrading to libfdt-1.4.6-5 and everything works as expected. 

Moving to the source package there (dtc).
Comment 2 David Gibson 2018-09-03 01:06:32 UTC 
Actually, the bug _is_ in qemu.  qemu used to fail to initialize the reserve map when constructing the initial device tree.  This was always incorrect, but the newer dtc version catches the situation and throws an error, rather than just emitting a dodgy fdt blob.

It's fixed upstream in 127f03e4 "spapr: Initialize reserved areas list in FDT in H_CAS handler".
Comment 3 Cole Robinson 2018-09-04 13:30:57 UTC 
I'll do a build today
Comment 4 Fedora Update System 2018-09-04 19:11:18 UTC 
qemu-2.11.2-4.fc28 has been submitted as an update to Fedora 28. https://bodhi.fedoraproject.org/updates/FEDORA-2018-f06af0ec34
Comment 5 Fedora Update System 2018-09-07 00:06:44 UTC 
qemu-2.11.2-4.fc28 has been pushed to the Fedora 28 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-f06af0ec34
Comment 6 Fedora Update System 2018-09-07 00:07:16 UTC 
qemu-2.11.2-4.fc28 has been pushed to the Fedora 28 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-f06af0ec34
Comment 7 Adam Williamson 2018-09-10 21:45:11 UTC 
openQA ppc64 worker box ran into this recently - all tests were failing because of this bug. Updated qemu fixes it.
Comment 8 Fedora Update System 2018-09-27 17:27:54 UTC 
qemu-2.11.2-4.fc28 has been pushed to the Fedora 28 stable repository. If problems still persist, please make note of it in this bug report.