Bug 1376547

Summary: qemu-system-s390x: -device isa-serial,chardev=charserial0,id=serial0: 'isa-serial' is not a valid device model name
Product: [Fedora] Fedora Reporter: Dan Horák <dan>
Component: libguestfsAssignee: Richard W.M. Jones <rjones>
Status: CLOSED UPSTREAM QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rawhideCC: acarter, agedosier, berrange, clalancette, dan, dzheng, fweimer, itamar, laine, libvirt-maint, mzoeller, pbrobinson, ptoscano, rjones, salmy, thuth, veillard
Target Milestone: ---Keywords: Tracking
Target Release: ---   
Hardware: s390x   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-11-27 14:02:27 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
Bug Depends On:    
Bug Blocks: 467765, 910269    
Attachments:
Description Flags
libguestfs-test-tool output none

Description Dan Horák 2016-09-15 16:37:26 UTC
The test-suite for libguestfs fails on s390x with an "invalid device" from qemu. It could be coming also from libvirt, because I could find the "isa-serial" string in libguestfs.

from build.log of http://s390.koji.fedoraproject.org/koji/taskinfo?taskID=2349154
...
+ cat /builddir/.cache/libvirt/qemu/log/guestfs-aek3g9amc7ipnr19.log
2016-09-15 15:59:08.995+0000: starting up libvirt version: 2.2.0, package: 1.fc26 (Fedora Project, 2016-09-07-15:58:11, fedora2.s390.bos.redhat.com), qemu version: 2.7.0(qemu-2.7.0-1.fc26), hostname: fedora4.s390.bos.redhat.com
LC_ALL=C LD_LIBRARY_PATH=/builddir/build/BUILD/libguestfs-1.35.5/ruby/ext/guestfs:/builddir/build/BUILD/libguestfs-1.35.5/src/.libs:/builddir/build/BUILD/libguestfs-1.35.5/java/.libs:/builddir/build/BUILD/libguestfs-1.35.5/gobject/.libs PATH=/builddir/build/BUILD/libguestfs-1.35.5/v2v:/builddir/build/BUILD/libguestfs-1.35.5/tools:/builddir/build/BUILD/libguestfs-1.35.5/test-tool:/builddir/build/BUILD/libguestfs-1.35.5/sysprep:/builddir/build/BUILD/libguestfs-1.35.5/sparsify:/builddir/build/BUILD/libguestfs-1.35.5/resize:/builddir/build/BUILD/libguestfs-1.35.5/rescue:/builddir/build/BUILD/libguestfs-1.35.5/p2v:/builddir/build/BUILD/libguestfs-1.35.5/make-fs:/builddir/build/BUILD/libguestfs-1.35.5/inspector:/builddir/build/BUILD/libguestfs-1.35.5/get-kernel:/builddir/build/BUILD/libguestfs-1.35.5/fuse:/builddir/build/BUILD/libguestfs-1.35.5/format:/builddir/build/BUILD/libguestfs-1.35.5/fish:/builddir/build/BUILD/libguestfs-1.35.5/erlang:/builddir/build/BUILD/libguestfs-1.35.5/edit:/builddir/build/BUILD/libguestfs-1.35.5/diff:/builddir/build/BUILD/libguestfs-1.35.5/dib:/builddir/build/BUILD/libguestfs-1.35.5/df:/builddir/build/BUILD/libguestfs-1.35.5/customize:/builddir/build/BUILD/libguestfs-1.35.5/cat:/builddir/build/BUILD/libguestfs-1.35.5/builder:/builddir/build/BUILD/libguestfs-1.35.5/align:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/sbin HOME=/builddir USER=mockbuild LOGNAME=mockbuild QEMU_AUDIO_DRV=none TMPDIR=/builddir/build/BUILD/libguestfs-1.35.5/tmp /usr/bin/qemu-system-s390x -name guest=guestfs-aek3g9amc7ipnr19,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/builddir/.config/libvirt/qemu/lib/domain-1-guestfs-aek3g9amc7ip/master-key.aes -machine s390-ccw-virtio-2.7,accel=tcg,usb=off -m 500 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid a195dca8-f07b-4b6f-b9c6-203ddfcf20a1 -display none -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/builddir/.config/libvirt/qemu/lib/domain-1-guestfs-aek3g9amc7ip/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -no-reboot -boot strict=on -kernel /builddir/build/BUILD/libguestfs-1.35.5/tmp/.guestfs-995/appliance.d/kernel -initrd /builddir/build/BUILD/libguestfs-1.35.5/tmp/.guestfs-995/appliance.d/initrd -append 'panic=1 console=ttyS0 udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=/dev/sdb selinux=0 guestfs_verbose=1 TERM=dumb' -device virtio-scsi-ccw,id=scsi0,devno=fe.0.0000 -device virtio-serial-ccw,id=virtio-serial0,devno=fe.0.0001 -drive file=/builddir/build/BUILD/libguestfs-1.35.5/tmp/libguestfs6FLv8x/scratch.1,format=raw,if=none,id=drive-scsi0-0-0-0,cache=unsafe -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1 -drive file=/builddir/build/BUILD/libguestfs-1.35.5/tmp/libguestfs6FLv8x/overlay2,format=qcow2,if=none,id=drive-scsi0-0-1-0,cache=unsafe -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=1,lun=0,drive=drive-scsi0-0-1-0,id=scsi0-0-1-0 -chardev socket,id=charserial0,path=/tmp/libguestfsPCeiJF/console.sock -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,path=/tmp/libguestfsPCeiJF/guestfsd.sock -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.libguestfs.channel.0 -device virtio-balloon-ccw,id=balloon0,devno=fe.0.0002 -object rng-random,id=objrng0,filename=/dev/urandom -device virtio-rng-ccw,rng=objrng0,id=rng0,devno=fe.0.0003 -msg timestamp=on
Domain id=1 is tainted: custom-argv
2016-09-15T15:59:09.209857Z qemu-system-s390x: -device isa-serial,chardev=charserial0,id=serial0: 'isa-serial' is not a valid device model name
2016-09-15 15:59:09.215+0000: shutting down


Version-Release number of selected component (if applicable):
libguestfs-1.35.5-1.fc26

Comment 1 Richard W.M. Jones 2016-09-15 16:59:19 UTC
The good news is that libguestfs builds fine on S/390.  However
it fails this sanity check so it won't work at all.

This is basically a libvirt bug - it is passing a non-existent
device to the guest.

Here are some things you can try in a local build of libguestfs.
You will first need to build libguestfs itself.  You should follow
the instructions in the link closely.  The tests below assume you have
already built libguestfs and are operating in the build directory.
NB do not attempt to use 'make install'.

  http://libguestfs.org/guestfs-building.1.html

    - - -

The tests:

(1) make quickcheck LIBGUESTFS_BACKEND=libvirt

(should replicate the above bug)

(2) make quickcheck LIBGUESTFS_BACKEND=direct

This directs libguestfs to run qemu itself ("direct"-ly).  This may
or may not work.  Since it is using a completely different backend
it could hit a different set of bugs.

https://github.com/libguestfs/libguestfs/blob/master/src/launch-direct.c
https://github.com/libguestfs/libguestfs/blob/master/src/launch-libvirt.c

If test (2) works, then:

(3a) make check -k LIBGUESTFS_BACKEND=direct
     find -name test-suite.log | xargs grep ^FAIL:

Runs the full test suite, and then prints out the failures.  Please file
bugs about anything you find.

If you are interested in automated testing then most tests allow you
to set environment variables to skip tests, but you usually have to
examine the test code to find out what the skip variable is.

(3b) make check -k LIBGUESTFS_BACKEND=direct LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1

This runs the full test suite with additional debugging enabled
which is a good way to track down specific problems.

    - - -

If you have plenty of patience, then:

(4) make check-slow LIBGUESTFS_BACKEND=direct

Runs an extra set of long-running and/or resource-intensive tests.

Comment 2 Dan Horák 2016-10-06 12:06:13 UTC
(1) yes, the above bug is replicated

[sharkcz@devel11 libguestfs-1.35.4]$ make quickcheck LIBGUESTFS_BACKEND=libvirt
./run test-tool/libguestfs-test-tool 
     ************************************************************
     *                    IMPORTANT NOTICE
     *
     * When reporting bugs, include the COMPLETE, UNEDITED
     * output below in your bug report.
     *
     ************************************************************
LIBGUESTFS_PATH=/home/sharkcz/libguestfs/libguestfs-1.35.4/appliance
LIBGUESTFS_CACHEDIR=/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp
LD_LIBRARY_PATH=/home/sharkcz/libguestfs/libguestfs-1.35.4/ruby/ext/guestfs:/home/sharkcz/libguestfs/libguestfs-1.35.4/src/.libs:/home/sharkcz/libguestfs/libguestfs-1.35.4/java/.libs:/home/sharkcz/libguestfs/libguestfs-1.35.4/gobject/.libs
LIBGUESTFS_TMPDIR=/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp
LIBGUESTFS_BACKEND=libvirt
PATH=/home/sharkcz/libguestfs/libguestfs-1.35.4/v2v:/home/sharkcz/libguestfs/libguestfs-1.35.4/tools:/home/sharkcz/libguestfs/libguestfs-1.35.4/test-tool:/home/sharkcz/libguestfs/libguestfs-1.35.4/sysprep:/home/sharkcz/libguestfs/libguestfs-1.35.4/sparsify:/home/sharkcz/libguestfs/libguestfs-1.35.4/resize:/home/sharkcz/libguestfs/libguestfs-1.35.4/rescue:/home/sharkcz/libguestfs/libguestfs-1.35.4/p2v:/home/sharkcz/libguestfs/libguestfs-1.35.4/make-fs:/home/sharkcz/libguestfs/libguestfs-1.35.4/inspector:/home/sharkcz/libguestfs/libguestfs-1.35.4/get-kernel:/home/sharkcz/libguestfs/libguestfs-1.35.4/fuse:/home/sharkcz/libguestfs/libguestfs-1.35.4/format:/home/sharkcz/libguestfs/libguestfs-1.35.4/fish:/home/sharkcz/libguestfs/libguestfs-1.35.4/erlang:/home/sharkcz/libguestfs/libguestfs-1.35.4/edit:/home/sharkcz/libguestfs/libguestfs-1.35.4/diff:/home/sharkcz/libguestfs/libguestfs-1.35.4/dib:/home/sharkcz/libguestfs/libguestfs-1.35.4/df:/home/sharkcz/libguestfs/libguestfs-1.35.4/customize:/home/sharkcz/libguestfs/libguestfs-1.35.4/cat:/home/sharkcz/libguestfs/libguestfs-1.35.4/builder:/home/sharkcz/libguestfs/libguestfs-1.35.4/align:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/sharkcz/.local/bin:/home/sharkcz/bin
XDG_RUNTIME_DIR=/run/user/1001
SELinux: Permissive
guestfs_get_append: (null)
guestfs_get_autosync: 1
guestfs_get_backend: libvirt
guestfs_get_backend_settings: []
guestfs_get_cachedir: /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp
guestfs_get_direct: 0
guestfs_get_hv: /usr/bin/qemu-system-s390x
guestfs_get_memsize: 500
guestfs_get_network: 0
guestfs_get_path: /home/sharkcz/libguestfs/libguestfs-1.35.4/appliance
guestfs_get_pgroup: 0
guestfs_get_program: libguestfs-test-tool
guestfs_get_recovery_proc: 1
guestfs_get_smp: 1
guestfs_get_sockdir: /run/user/1001
guestfs_get_tmpdir: /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp
guestfs_get_trace: 0
guestfs_get_verbose: 1
host_cpu: s390x
Launching appliance, timeout set to 600 seconds.
libguestfs: launch: program=libguestfs-test-tool
libguestfs: launch: version=1.35.4fedora=26,release=1.fc26,libvirt
libguestfs: launch: backend registered: unix
libguestfs: launch: backend registered: uml
libguestfs: launch: backend registered: libvirt
libguestfs: launch: backend registered: direct
libguestfs: launch: backend=libvirt
libguestfs: launch: tmpdir=/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/libguestfs9pQILc
libguestfs: launch: umask=0002
libguestfs: launch: euid=1001
libguestfs: libvirt version = 2002000 (2.2.0)
libguestfs: guest random name = guestfs-7dx1pnhjex9g3q1b
libguestfs: connect to libvirt
libguestfs: opening libvirt handle: URI = qemu:///session, auth = default+wrapper, flags = 0
libguestfs: successfully opened libvirt handle: conn = 0x2aa28c6a360
libguestfs: qemu version (reported by libvirt) = 2007000 (2.7.0)
libguestfs: get libvirt capabilities
libguestfs: parsing capabilities XML
libguestfs: build appliance
libguestfs: begin building supermin appliance
libguestfs: run supermin
libguestfs: command: run: /usr/bin/supermin
libguestfs: command: run: \ --build
libguestfs: command: run: \ --verbose
libguestfs: command: run: \ --if-newer
libguestfs: command: run: \ --lock /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/lock
libguestfs: command: run: \ --copy-kernel
libguestfs: command: run: \ -f ext2
libguestfs: command: run: \ --host-cpu s390x
libguestfs: command: run: \ /home/sharkcz/libguestfs/libguestfs-1.35.4/appliance/supermin.d
libguestfs: command: run: \ -o /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d
supermin: version: 5.1.16
supermin: rpm: detected RPM version 4.13
supermin: package handler: fedora/rpm
supermin: acquiring lock on /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/lock
supermin: if-newer: output does not need rebuilding
libguestfs: finished building supermin appliance
libguestfs: command: run: qemu-img
libguestfs: command: run: \ create
libguestfs: command: run: \ -f qcow2
libguestfs: command: run: \ -o backing_file=/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/root,backing_fmt=raw
libguestfs: command: run: \ /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/libguestfs9pQILc/overlay2
Formatting '/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/libguestfs9pQILc/overlay2', fmt=qcow2 size=4294967296 backing_file=/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/root backing_fmt=raw encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16
libguestfs: create libvirt XML
libguestfs: libvirt XML:\n<?xml version="1.0"?>\n<domain type="kvm" xmlns:qemu="http://libvirt.org/schemas/domain/qemu/1.0">\n  <name>guestfs-7dx1pnhjex9g3q1b</name>\n  <memory unit="MiB">500</memory>\n  <currentMemory unit="MiB">500</currentMemory>\n  <cpu mode="host-passthrough">\n    <model fallback="allow"/>\n  </cpu>\n  <vcpu>1</vcpu>\n  <clock offset="utc">\n    <timer name="rtc" tickpolicy="catchup"/>\n    <timer name="pit" tickpolicy="delay"/>\n  </clock>\n  <os>\n    <type>hvm</type>\n    <kernel>/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/kernel</kernel>\n    <initrd>/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/initrd</initrd>\n    <cmdline>panic=1 console=ttyS0 udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=/dev/sdb selinux=0 guestfs_verbose=1 TERM=xterm-256color</cmdline>\n  </os>\n  <on_reboot>destroy</on_reboot>\n  <devices>\n    <rng model="virtio">\n      <backend model="random">/dev/urandom</backend>\n    </rng>\n    <controller type="scsi" index="0" model="virtio-scsi"/>\n    <disk device="disk" type="file">\n      <source file="/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/libguestfs9pQILc/scratch.1"/>\n      <target dev="sda" bus="scsi"/>\n      <driver name="qemu" type="raw" cache="unsafe"/>\n      <address type="drive" controller="0" bus="0" target="0" unit="0"/>\n    </disk>\n    <disk type="file" device="disk">\n      <source file="/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/libguestfs9pQILc/overlay2"/>\n      <target dev="sdb" bus="scsi"/>\n      <driver name="qemu" type="qcow2" cache="unsafe"/>\n      <address type="drive" controller="0" bus="0" target="1" unit="0"/>\n      <shareable/>\n    </disk>\n    <serial type="unix">\n      <source mode="connect" path="/run/user/1001/libguestfsaBJryO/console.sock"/>\n      <target port="0"/>\n    </serial>\n    <channel type="unix">\n      <source mode="connect" path="/run/user/1001/libguestfsaBJryO/guestfsd.sock"/>\n      <target type="virtio" name="org.libguestfs.channel.0"/>\n    </channel>\n  </devices>\n  <qemu:commandline>\n    <qemu:env name="TMPDIR" value="/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp"/>\n  </qemu:commandline>\n</domain>\n
libguestfs: command: run: ls
libguestfs: command: run: \ -a
libguestfs: command: run: \ -l
libguestfs: command: run: \ -R
libguestfs: command: run: \ -Z /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001
libguestfs: /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001:
libguestfs: total 32
libguestfs: drwxr-xr-x. 3 sharkcz sharkcz unconfined_u:object_r:user_tmp_t:s0    91 Oct  6 12:54 .
libguestfs: drwxr-xr-x. 6 sharkcz sharkcz system_u:object_r:tmp_t:s0            117 Oct  6 12:54 ..
libguestfs: drwxr-xr-x. 2 sharkcz sharkcz unconfined_u:object_r:user_tmp_t:s0    46 Oct  6 12:53 appliance.d
libguestfs: -rw-r--r--. 1 sharkcz sharkcz unconfined_u:object_r:user_tmp_t:s0     0 Sep 15 14:39 lock
libguestfs: -rw-rw-r--. 1 sharkcz sharkcz unconfined_u:object_r:user_tmp_t:s0  3335 Sep 15 19:20 qemu.devices
libguestfs: -rw-rw-r--. 1 sharkcz sharkcz unconfined_u:object_r:user_tmp_t:s0 21933 Sep 15 19:20 qemu.help
libguestfs: -rw-rw-r--. 1 sharkcz sharkcz unconfined_u:object_r:user_tmp_t:s0    48 Sep 15 19:20 qemu.stat
libguestfs: 
libguestfs: /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d:
libguestfs: total 541616
libguestfs: drwxr-xr-x. 2 sharkcz sharkcz unconfined_u:object_r:user_tmp_t:s0         46 Oct  6 12:53 .
libguestfs: drwxr-xr-x. 3 sharkcz sharkcz unconfined_u:object_r:user_tmp_t:s0         91 Oct  6 12:54 ..
libguestfs: -rw-r--r--. 1 sharkcz sharkcz system_u:object_r:virt_content_t:s0     724992 Oct  6 12:54 initrd
libguestfs: -rwxr-xr-x. 1 sharkcz sharkcz system_u:object_r:virt_content_t:s0   16467200 Oct  6 12:54 kernel
libguestfs: -rw-r--r--. 1 sharkcz sharkcz system_u:object_r:virt_content_t:s0 4294967296 Oct  6 12:54 root
libguestfs: command: run: ls
libguestfs: command: run: \ -a
libguestfs: command: run: \ -l
libguestfs: command: run: \ -Z /run/user/1001/libguestfsaBJryO
libguestfs: total 0
libguestfs: drwx------. 2 sharkcz sharkcz unconfined_u:object_r:user_tmp_t:s0  80 Oct  6 12:54 .
libguestfs: drwx------. 5 sharkcz sharkcz system_u:object_r:user_tmp_t:s0     120 Oct  6 12:54 ..
libguestfs: srwxrwxr-x. 1 sharkcz sharkcz unconfined_u:object_r:user_tmp_t:s0   0 Oct  6 12:54 console.sock
libguestfs: srwxrwxr-x. 1 sharkcz sharkcz unconfined_u:object_r:user_tmp_t:s0   0 Oct  6 12:54 guestfsd.sock
libguestfs: launch libvirt guest
libguestfs: error: could not create appliance through libvirt.

Try running qemu directly without libvirt using this environment variable:
export LIBGUESTFS_BACKEND=direct

Original error from libvirt: internal error: qemu unexpectedly closed the monitor: 2016-10-06T10:54:30.177733Z qemu-system-s390x: -device isa-serial,chardev=charserial0,id=serial0: 'isa-serial' is not a valid device model name [code=1 int1=-1]
libguestfs: closing guestfs handle 0x2aa28c666b0 (state 0)
libguestfs: command: run: rm
libguestfs: command: run: \ -rf /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/libguestfs9pQILc
libguestfs: command: run: rm
libguestfs: command: run: \ -rf /run/user/1001/libguestfsaBJryO
make: *** [Makefile:2407: quickcheck] Error 1

Comment 3 Dan Horák 2016-10-06 12:07:04 UTC
(2) fails after a timeout

[sharkcz@devel11 libguestfs-1.35.4]$ make quickcheck LIBGUESTFS_BACKEND=direct
./run test-tool/libguestfs-test-tool 
     ************************************************************
     *                    IMPORTANT NOTICE
     *
     * When reporting bugs, include the COMPLETE, UNEDITED
     * output below in your bug report.
     *
     ************************************************************
LIBGUESTFS_PATH=/home/sharkcz/libguestfs/libguestfs-1.35.4/appliance
LIBGUESTFS_CACHEDIR=/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp
LD_LIBRARY_PATH=/home/sharkcz/libguestfs/libguestfs-1.35.4/ruby/ext/guestfs:/home/sharkcz/libguestfs/libguestfs-1.35.4/src/.libs:/home/sharkcz/libguestfs/libguestfs-1.35.4/java/.libs:/home/sharkcz/libguestfs/libguestfs-1.35.4/gobject/.libs
LIBGUESTFS_TMPDIR=/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp
LIBGUESTFS_BACKEND=direct
PATH=/home/sharkcz/libguestfs/libguestfs-1.35.4/v2v:/home/sharkcz/libguestfs/libguestfs-1.35.4/tools:/home/sharkcz/libguestfs/libguestfs-1.35.4/test-tool:/home/sharkcz/libguestfs/libguestfs-1.35.4/sysprep:/home/sharkcz/libguestfs/libguestfs-1.35.4/sparsify:/home/sharkcz/libguestfs/libguestfs-1.35.4/resize:/home/sharkcz/libguestfs/libguestfs-1.35.4/rescue:/home/sharkcz/libguestfs/libguestfs-1.35.4/p2v:/home/sharkcz/libguestfs/libguestfs-1.35.4/make-fs:/home/sharkcz/libguestfs/libguestfs-1.35.4/inspector:/home/sharkcz/libguestfs/libguestfs-1.35.4/get-kernel:/home/sharkcz/libguestfs/libguestfs-1.35.4/fuse:/home/sharkcz/libguestfs/libguestfs-1.35.4/format:/home/sharkcz/libguestfs/libguestfs-1.35.4/fish:/home/sharkcz/libguestfs/libguestfs-1.35.4/erlang:/home/sharkcz/libguestfs/libguestfs-1.35.4/edit:/home/sharkcz/libguestfs/libguestfs-1.35.4/diff:/home/sharkcz/libguestfs/libguestfs-1.35.4/dib:/home/sharkcz/libguestfs/libguestfs-1.35.4/df:/home/sharkcz/libguestfs/libguestfs-1.35.4/customize:/home/sharkcz/libguestfs/libguestfs-1.35.4/cat:/home/sharkcz/libguestfs/libguestfs-1.35.4/builder:/home/sharkcz/libguestfs/libguestfs-1.35.4/align:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/sharkcz/.local/bin:/home/sharkcz/bin
XDG_RUNTIME_DIR=/run/user/1001
SELinux: Permissive
guestfs_get_append: (null)
guestfs_get_autosync: 1
guestfs_get_backend: direct
guestfs_get_backend_settings: []
guestfs_get_cachedir: /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp
guestfs_get_direct: 0
guestfs_get_hv: /usr/bin/qemu-system-s390x
guestfs_get_memsize: 500
guestfs_get_network: 0
guestfs_get_path: /home/sharkcz/libguestfs/libguestfs-1.35.4/appliance
guestfs_get_pgroup: 0
guestfs_get_program: libguestfs-test-tool
guestfs_get_recovery_proc: 1
guestfs_get_smp: 1
guestfs_get_sockdir: /run/user/1001
guestfs_get_tmpdir: /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp
guestfs_get_trace: 0
guestfs_get_verbose: 1
host_cpu: s390x
Launching appliance, timeout set to 600 seconds.
libguestfs: launch: program=libguestfs-test-tool
libguestfs: launch: version=1.35.4fedora=26,release=1.fc26,libvirt
libguestfs: launch: backend registered: unix
libguestfs: launch: backend registered: uml
libguestfs: launch: backend registered: libvirt
libguestfs: launch: backend registered: direct
libguestfs: launch: backend=direct
libguestfs: launch: tmpdir=/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/libguestfs8KFxd9
libguestfs: launch: umask=0002
libguestfs: launch: euid=1001
libguestfs: begin building supermin appliance
libguestfs: run supermin
libguestfs: command: run: /usr/bin/supermin
libguestfs: command: run: \ --build
libguestfs: command: run: \ --verbose
libguestfs: command: run: \ --if-newer
libguestfs: command: run: \ --lock /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/lock
libguestfs: command: run: \ --copy-kernel
libguestfs: command: run: \ -f ext2
libguestfs: command: run: \ --host-cpu s390x
libguestfs: command: run: \ /home/sharkcz/libguestfs/libguestfs-1.35.4/appliance/supermin.d
libguestfs: command: run: \ -o /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d
supermin: version: 5.1.16
supermin: rpm: detected RPM version 4.13
supermin: package handler: fedora/rpm
supermin: acquiring lock on /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/lock
supermin: if-newer: output does not need rebuilding
libguestfs: finished building supermin appliance
libguestfs: begin testing qemu features
libguestfs: checking for previously cached test results of /usr/bin/qemu-system-s390x, in /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001
libguestfs: loading previously cached test results
libguestfs: qemu version 2.7
libguestfs: finished testing qemu features
[00597ms] /usr/bin/qemu-system-s390x \
    -global virtio-blk-pci.scsi=off \
    -nodefconfig \
    -enable-fips \
    -nodefaults \
    -display none \
    -machine accel=kvm:tcg \
    -cpu host \
    -m 500 \
    -no-reboot \
    -rtc driftfix=slew \
    -global kvm-pit.lost_tick_policy=discard \
    -kernel /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/kernel \
    -initrd /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/initrd \
    -object rng-random,filename=/dev/urandom,id=rng0 \
    -device virtio-rng-pci,rng=rng0 \
    -device virtio-scsi-pci,id=scsi \
    -drive file=/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/libguestfs8KFxd9/scratch.1,cache=unsafe,format=raw,id=hd0,if=none \
    -device scsi-hd,drive=hd0 \
    -drive file=/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/root,snapshot=on,id=appliance,cache=unsafe,if=none,format=raw \
    -device scsi-hd,drive=appliance \
    -device virtio-serial-pci \
    -serial stdio \
    -chardev socket,path=/run/user/1001/libguestfsdKd6kI/guestfsd.sock,id=channel0 \
    -device virtserialport,chardev=channel0,name=org.libguestfs.channel.0 \
    -append 'panic=1 console=ttyS0 udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=/dev/sdb selinux=0 guestfs_verbose=1 TERM=xterm-256color'
qemu-system-s390x: Warning: global kvm-pit.lost_tick_policy has invalid class name
make: *** [Makefile:2407: quickcheck] Alarm clock

Comment 4 Dan Horák 2016-10-06 12:10:14 UTC
Because KVM is supposed to work OK on s390x, I've tried to "fix" the qemu option until it "works". Full history is below.

[sharkcz@devel11 libguestfs-1.35.4]$ file /run/user/1001/libguestfsdKd6kI/guestfsd.sock
/run/user/1001/libguestfsdKd6kI/guestfsd.sock: socket
[sharkcz@devel11 libguestfs-1.35.4]$ /usr/bin/qemu-system-s390x \
>     -global virtio-blk-pci.scsi=off \
>     -nodefconfig \
>     -enable-fips \
>     -nodefaults \
>     -display none \
>     -machine accel=kvm:tcg \
>     -cpu host \
>     -m 500 \
>     -no-reboot \
>     -rtc driftfix=slew \
>     -global kvm-pit.lost_tick_policy=discard \
>     -kernel /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/kernel \
>     -initrd /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/initrd \
>     -object rng-random,filename=/dev/urandom,id=rng0 \
>     -device virtio-rng-pci,rng=rng0 \
>     -device virtio-scsi-pci,id=scsi \
>     -drive file=/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/libguestfs8KFxd9/scratch.1,cache=unsafe,format=raw,id=hd0,if=none \
>     -device scsi-hd,drive=hd0 \
>     -drive file=/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/root,snapshot=on,id=appliance,cache=unsafe,if=none,format=raw \
>     -device scsi-hd,drive=appliance \
>     -device virtio-serial-pci \
>     -serial stdio \
>     -chardev socket,path=/run/user/1001/libguestfsdKd6kI/guestfsd.sock,id=channel0 \
>     -device virtserialport,chardev=channel0,name=org.libguestfs.channel.0 \
>     -append 'panic=1 console=ttyS0 udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=/dev/sdb selinux=0 guestfs_verbose=1 TERM=xterm-256color'
qemu-system-s390x: -chardev socket,path=/run/user/1001/libguestfsdKd6kI/guestfsd.sock,id=channel0: Failed to connect socket: Connection refused
[sharkcz@devel11 libguestfs-1.35.4]$ /usr/bin/qemu-system-s390x     -global virtio-blk-pci.scsi=off     -nodefconfig     -enable-fips     -nodefaults     -display none     -machine accel=kvm:tcg     -cpu host     -m 500     -no-reboot     -rtc driftfix=slew     -global kvm-pit.lost_tick_policy=discard     -kernel /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/kernel     -initrd /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/initrd     -object rng-random,filename=/dev/urandom,id=rng0     -device virtio-rng-pci,rng=rng0     -device virtio-scsi-pci,id=scsi     -drive file=/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/libguestfs8KFxd9/scratch.1,cache=unsafe,format=raw,id=hd0,if=none     -device scsi-hd,drive=hd0     -drive file=/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/root,snapshot=on,id=appliance,cache=unsafe,if=none,format=raw     -device scsi-hd,drive=appliance     -device virtio-serial-pci     -serial stdio     -append 'panic=1 console=ttyS0 udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=/dev/sdb selinux=0 guestfs_verbose=1 TERM=xterm-256color'qemu-system-s390x: Warning: global kvm-pit.lost_tick_policy has invalid class name
qemu-system-s390x: terminating on signal 2
[sharkcz@devel11 libguestfs-1.35.4]$ ^C
[sharkcz@devel11 libguestfs-1.35.4]$ file /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/libguestfs8KFxd9/scratch.1
/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/libguestfs8KFxd9/scratch.1: data
[sharkcz@devel11 libguestfs-1.35.4]$ file /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/initrd
/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/initrd: ASCII cpio archive (SVR4 with no CRC)
[sharkcz@devel11 libguestfs-1.35.4]$ file /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/root
/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/root: Linux rev 1.0 ext2 filesystem data, UUID=322398c2-5a64-4934-a286-c13a8267b2d1 (large files)
[sharkcz@devel11 libguestfs-1.35.4]$ /usr/bin/qemu-system-s390x     -global virtio-blk-pci.scsi=off     -nodefconfig   -nodefaults     -display none     -machine accel=kvm:tcg     -cpu host     -m 500     -no-reboot  -kernel /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/kernel     -initrd /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/initrd     -object rng-random,filename=/dev/urandom,id=rng0     -device virtio-rng-pci,rng=rng0     -device virtio-scsi-pci,id=scsi     -drive file=/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/libguestfs8KFxd9/scratch.1,cache=unsafe,format=raw,id=hd0,if=none     -device scsi-hd,drive=hd0     -drive file=/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/root,snapshot=on,id=appliance,cache=unsafe,if=none,format=raw     -device scsi-hd,drive=appliance     -device virtio-serial-pci     -serial stdio     -append 'panic=1 console=ttyS0 udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=/dev/sdb selinux=0 guestfs_verbose=1 TERM=xterm-256color'
qemu-system-s390x: terminating on signal 2
[sharkcz@devel11 libguestfs-1.35.4]$ ^C
[sharkcz@devel11 libguestfs-1.35.4]$ /usr/bin/qemu-system-s390x  -nodefconfig   -nodefaults     -display none     -machine accel=kvm:tcg     -cpu host     -m 500     -no-reboot  -kernel /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/kernel     -initrd /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/initrd  -device virtio-scsi-pci,id=scsi     -drive file=/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/libguestfs8KFxd9/scratch.1,cache=unsafe,format=raw,id=hd0,if=none     -device scsi-hd,drive=hd0     -drive file=/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/root,snapshot=on,id=appliance,cache=unsafe,if=none,format=raw     -device scsi-hd,drive=appliance     -device virtio-serial-pci     -serial stdio     -append 'panic=1 console=ttyS0 udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=/dev/sdb selinux=0 guestfs_verbose=1 TERM=xterm-256color'qemu-system-s390x: terminating on signal 2
[sharkcz@devel11 libguestfs-1.35.4]$ /usr/bin/qemu-system-s390x  -virtioconsole stdio -nodefconfig   -nodefaults     -display none     -machine accel=kvm:tcg     -cpu host     -m 500     -no-reboot  -kernel /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/kernel     -initrd /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/initrd  -device virtio-scsi-pci,id=scsi     -drive file=/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/libguestfs8KFxd9/scratch.1,cache=unsafe,format=raw,id=hd0,if=none     -device scsi-hd,drive=hd0     -drive file=/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/root,snapshot=on,id=appliance,cache=unsafe,if=none,format=raw     -device scsi-hd,drive=appliance     -device virtio-serial-pci     -serial stdio     -append 'panic=1 console=ttyS0 udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=/dev/sdb selinux=0 guestfs_verbose=1 TERM=xterm-256color'
qemu-system-s390x: -virtioconsole stdio: cannot use stdio by multiple character devices
qemu-system-s390x: -virtioconsole stdio: could not connect virtio console to character backend 'stdio'
[sharkcz@devel11 libguestfs-1.35.4]$ /usr/bin/qemu-system-s390x  -virtioconsole stdio -nodefconfig   -nodefaults     -display none     -machine accel=kvm:tcg     -cpu host     -m 500     -no-reboot  -kernel /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/kernel     -initrd /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/initrd  -device virtio-scsi-pci,id=scsi     -drive file=/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/libguestfs8KFxd9/scratch.1,cache=unsafe,format=raw,id=hd0,if=none     -device scsi-hd,drive=hd0     -drive file=/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/root,snapshot=on,id=appliance,cache=unsafe,if=none,format=raw     -device scsi-hd,drive=appliance      -append 'panic=1 console=ttyS0 udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=/dev/sdb selinux=0 guestfs_verbose=1 TERM=xterm-256color'
[    0.512339] Freeing unused kernel memory: 2632K (0000000000d23000 - 0000000000fb5000)
[    0.512722] Write protected read-only-after-init data: 4k
qemu-system-s390x: terminating on signal 2
[sharkcz@devel11 libguestfs-1.35.4]$ /usr/bin/qemu-system-s390x  -virtioconsole stdio -nodefconfig   -nodefaults     -display none     -machine accel=kvm:tcg     -cpu host     -m 500     -no-reboot  -kernel /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/kernel     -initrd /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/initrd  -device virtio-scsi-pci,id=scsi     -drive file=/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/libguestfs8KFxd9/scratch.1,cache=unsafe,format=raw,id=hd0,if=none     -device scsi-hd,drive=hd0     -drive file=/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/root,snapshot=on,id=appliance,cache=unsafe,if=none,format=raw     -device scsi-hd,drive=appliance      -append 'panic=1 udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=/dev/sdb selinux=0 guestfs_verbose=1 TERM=dumb'
[    0.485177] Freeing unused kernel memory: 2632K (0000000000d23000 - 0000000000fb5000)
[    0.486061] Write protected read-only-after-init data: 4k
qemu-system-s390x: terminating on signal 2
[sharkcz@devel11 libguestfs-1.35.4]$ /usr/bin/qemu-system-s390x  -virtioconsole stdio -nodefconfig   -nodefaults     -display none     -machine accel=kvm:tcg     -cpu host     -m 500     -no-reboot  -kernel /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/kernel     -initrd /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/initrd  -device virtio-scsi-pci,id=scsi     -drive file=/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/libguestfs8KFxd9/scratch.1,cache=unsafe,format=raw,id=hd0,if=none     -device scsi-hd,drive=hd0     -drive file=/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/root,snapshot=on,id=appliance,cache=unsafe,if=none,format=raw     -device scsi-hd,drive=appliance      -append 'panic=1 udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory root=/dev/sdb selinux=0 guestfs_verbose=1 TERM=dumb'[    0.554363] Freeing unused kernel memory: 2632K (0000000000d23000 - 0000000000fb5000)
[    0.557909] Write protected read-only-after-init data: 4k
[   33.384684] random: fast init done
[  310.375898] kworker/dying (28) used greatest stack depth: 12584 bytes left
qemu-system-s390x: terminating on signal 2
[sharkcz@devel11 libguestfs-1.35.4]$ /usr/bin/qemu-system-s390x  -virtioconsole stdio -nodefconfig   -nodefaults     -display none   -enable-kvm   -m 500     -no-reboot  -kernel /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/kernel     -initrd /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/initrd  -device virtio-scsi-pci,id=scsi     -drive file=/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/libguestfs8KFxd9/scratch.1,cache=unsafe,format=raw,id=hd0,if=none     -device scsi-hd,drive=hd0     -drive file=/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/root,snapshot=on,id=appliance,cache=unsafe,if=none,format=raw     -device scsi-hd,drive=appliance      -append 'panic=1 udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory root=/dev/sdb selinux=0 guestfs_verbose=1 TERM=dumb'
[    0.536232] Freeing unused kernel memory: 2632K (0000000000d23000 - 0000000000fb5000)
[    0.542086] Write protected read-only-after-init data: 4k
qemu-system-s390x: terminating on signal 2
[sharkcz@devel11 libguestfs-1.35.4]$ /usr/bin/qemu-system-s390x  -virtioconsole stdio -nodefaults  -nographic  -enable-kvm   -m 500 -kernel /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/kernel     -initrd /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/initrd  -device virtio-scsi-pci,id=scsi     -drive file=/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/libguestfs8KFxd9/scratch.1,cache=unsafe,format=raw,id=hd0,if=none     -device scsi-hd,drive=hd0     -drive file=/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/root,snapshot=on,id=appliance,cache=unsafe,if=none,format=raw     -device scsi-hd,drive=appliance      -append 'panic=1 udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory root=/dev/sdb selinux=0 guestfs_verbose=1'
[    0.555494] Freeing unused kernel memory: 2632K (0000000000d23000 - 0000000000fb5000)
[    0.560202] Write protected read-only-after-init data: 4k
[   33.371238] random: fast init done
qemu-system-s390x: terminating on signal 2
[sharkcz@devel11 libguestfs-1.35.4]$ /usr/bin/qemu-system-s390x  -virtioconsole stdio -nodefaults  -nographic  -enable-kvm   -m 500 -kernel /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/kernel     -initrd /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/initrd -drive file=/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/libguestfs8KFxd9/scratch.1,cache=none,format=raw,if=virtio -drive file=/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/root,cache=none,if=virtio,format=raw -append 'panic=1 udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory root=/dev/sdb selinux=0 guestfs_verbose=1'[    0.557551] Freeing unused kernel memory: 2632K (0000000000d23000 - 0000000000fb5000)
[    0.573742] Write protected read-only-after-init data: 4k
[    0.634745] random: fast init done
qemu-system-s390x: terminating on signal 2
[sharkcz@devel11 libguestfs-1.35.4]$ /usr/bin/qemu-system-s390x  -virtioconsole stdio -nodefaults  -nographic  -enable-kvm   -m 500 -kernel /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/kernel     -initrd /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/initrd -drive file=/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/libguestfs8KFxd9/scratch.1,cache=none,format=raw,if=virtio -drive file=/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/root,cache=none,if=virtio,format=raw -append 'panic=1 udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory root=/dev/vdb selinux=0 guestfs_verbose=1'
[    0.551348] Freeing unused kernel memory: 2632K (0000000000d23000 - 0000000000fb5000)
[    0.562505] Write protected read-only-after-init data: 4k
[    0.640555] random: fast init done
[    0.699758] EXT4-fs (vdb): mounting ext2 file system using the ext4 subsystem
[    0.792117] EXT4-fs (vdb): mounted filesystem without journal. Opts: 
[    1.769892] mkdir (98) used greatest stack depth: 11672 bytes left
[    3.410752] systemd-tmpfiles[115]: [/usr/lib/tmpfiles.d/systemd.conf:11] Unknown group 'utmp'.
[    3.419727] systemd-tmpfiles[115]: [/usr/lib/tmpfiles.d/systemd.conf:19] Unknown user 'systemd-network'.
[    3.429335] systemd-tmpfiles[115]: [/usr/lib/tmpfiles.d/systemd.conf:20] Unknown user 'systemd-network'.
[    3.435680] systemd-tmpfiles[115]: [/usr/lib/tmpfiles.d/systemd.conf:21] Unknown user 'systemd-network'.
[    3.445130] systemd-tmpfiles[115]: [/usr/lib/tmpfiles.d/systemd.conf:25] Unknown group 'systemd-journal'.
[    3.455269] systemd-tmpfiles[115]: [/usr/lib/tmpfiles.d/systemd.conf:26] Unknown group 'systemd-journal'.
[    3.499625] systemd-tmpfile (115) used greatest stack depth: 11568 bytes left
[    3.631834] systemd-udevd[116]: starting version 231
[    4.801146] ip (155) used greatest stack depth: 11072 bytes left
[    7.576916] reboot: Restarting system
[    0.370430] Freeing unused kernel memory: 2632K (0000000000d23000 - 0000000000fb5000)
[    0.372007] Write protected read-only-after-init data: 4k
[    0.457072] EXT4-fs (vdb): mounting ext2 file system using the ext4 subsystem
[    0.462428] random: fast init done
[    0.484162] EXT4-fs (vdb): warning: mounting unchecked fs, running e2fsck is recommended
[    0.499701] EXT4-fs (vdb): mounted filesystem without journal. Opts: 
[    0.910863] mkdir (97) used greatest stack depth: 11672 bytes left
[    1.171283] od (112) used greatest stack depth: 10600 bytes left
[    1.790196] systemd-tmpfiles[113]: [/usr/lib/tmpfiles.d/systemd.conf:11] Unknown group 'utmp'.
[    1.794044] systemd-tmpfiles[113]: [/usr/lib/tmpfiles.d/systemd.conf:19] Unknown user 'systemd-network'.
[    1.803987] systemd-tmpfiles[113]: [/usr/lib/tmpfiles.d/systemd.conf:20] Unknown user 'systemd-network'.
[    1.808152] systemd-tmpfiles[113]: [/usr/lib/tmpfiles.d/systemd.conf:21] Unknown user 'systemd-network'.
[    1.811449] systemd-tmpfiles[113]: [/usr/lib/tmpfiles.d/systemd.conf:25] Unknown group 'systemd-journal'.
[    1.812298] systemd-tmpfiles[113]: [/usr/lib/tmpfiles.d/systemd.conf:26] Unknown group 'systemd-journal'.
[    1.844746] systemd-udevd[114]: starting version 231
[    3.596321] reboot: Restarting system
[    0.401642] Freeing unused kernel memory: 2632K (0000000000d23000 - 0000000000fb5000)
[    0.402765] Write protected read-only-after-init data: 4k
[    0.462452] EXT4-fs (vdb): mounting ext2 file system using the ext4 subsystem
[    0.463185] random: fast init done
[    0.473267] EXT4-fs (vdb): warning: mounting unchecked fs, running e2fsck is recommended
[    0.475617] EXT4-fs (vdb): mounted filesystem without journal. Opts: 
[    0.693859] mkdir (98) used greatest stack depth: 11672 bytes left
[    1.000599] od (113) used greatest stack depth: 10440 bytes left
[    1.436593] systemd-tmpfiles[114]: [/usr/lib/tmpfiles.d/systemd.conf:11] Unknown group 'utmp'.
[    1.443736] systemd-tmpfiles[114]: [/usr/lib/tmpfiles.d/systemd.conf:19] Unknown user 'systemd-network'.
[    1.448902] systemd-tmpfiles[114]: [/usr/lib/tmpfiles.d/systemd.conf:20] Unknown user 'systemd-network'.
[    1.454533] systemd-tmpfiles[114]: [/usr/lib/tmpfiles.d/systemd.conf:21] Unknown user 'systemd-network'.
[    1.460382] systemd-tmpfiles[114]: [/usr/lib/tmpfiles.d/systemd.conf:25] Unknown group 'systemd-journal'.
[    1.465547] systemd-tmpfiles[114]: [/usr/lib/tmpfiles.d/systemd.conf:26] Unknown group 'systemd-journal'.
[    1.575820] systemd-udevd[115]: starting version 231
[    3.082534] reboot: Restarting system
[    0.382995] Freeing unused kernel memory: 2632K (0000000000d23000 - 0000000000fb5000)
[    0.384398] Write protected read-only-after-init data: 4k
[    0.470744] random: fast init done
[    0.479370] EXT4-fs (vdb): mounting ext2 file system using the ext4 subsystem
[    0.510234] EXT4-fs (vdb): warning: mounting unchecked fs, running e2fsck is recommended
[    0.520867] EXT4-fs (vdb): mounted filesystem without journal. Opts: 
[    1.021988] mkdir (98) used greatest stack depth: 11672 bytes left
[    1.256693] od (113) used greatest stack depth: 10600 bytes left
[    1.616630] systemd-tmpfiles[114]: [/usr/lib/tmpfiles.d/systemd.conf:11] Unknown group 'utmp'.
[    1.617979] systemd-tmpfiles[114]: [/usr/lib/tmpfiles.d/systemd.conf:19] Unknown user 'systemd-network'.
[    1.618335] systemd-tmpfiles[114]: [/usr/lib/tmpfiles.d/systemd.conf:20] Unknown user 'systemd-network'.
[    1.618671] systemd-tmpfiles[114]: [/usr/lib/tmpfiles.d/systemd.conf:21] Unknown user 'systemd-network'.
[    1.618929] systemd-tmpfiles[114]: [/usr/lib/tmpfiles.d/systemd.conf:25] Unknown group 'systemd-journal'.
[    1.619160] systemd-tmpfiles[114]: [/usr/lib/tmpfiles.d/systemd.conf:26] Unknown group 'systemd-journal'.
[    1.644753] systemd-udevd[115]: starting version 231
[    3.708873] reboot: Restarting system
[    0.422235] Freeing unused kernel memory: 2632K (0000000000d23000 - 0000000000fb5000)
[    0.423524] Write protected read-only-after-init data: 4k
[    0.489863] random: fast init done
[    0.514141] EXT4-fs (vdb): mounting ext2 file system using the ext4 subsystem
[    0.522018] EXT4-fs (vdb): warning: mounting unchecked fs, running e2fsck is recommended
[    0.547116] EXT4-fs (vdb): mounted filesystem without journal. Opts: 
[    1.049376] mkdir (98) used greatest stack depth: 11672 bytes left
[    1.253757] od (113) used greatest stack depth: 10600 bytes left
[    1.594249] systemd-tmpfiles[114]: [/usr/lib/tmpfiles.d/systemd.conf:11] Unknown group 'utmp'.
[    1.595024] systemd-tmpfiles[114]: [/usr/lib/tmpfiles.d/systemd.conf:19] Unknown user 'systemd-network'.
[    1.595279] systemd-tmpfiles[114]: [/usr/lib/tmpfiles.d/systemd.conf:20] Unknown user 'systemd-network'.
[    1.631482] systemd-tmpfiles[114]: [/usr/lib/tmpfiles.d/systemd.conf:21] Unknown user 'systemd-network'.
[    1.646171] systemd-tmpfiles[114]: [/usr/lib/tmpfiles.d/systemd.conf:25] Unknown group 'systemd-journal'.
[    1.665870] systemd-tmpfiles[114]: [/usr/lib/tmpfiles.d/systemd.conf:26] Unknown group 'systemd-journal'.
[    1.731570] systemd-udevd[115]: starting version 231
qemu-system-s390x: terminating on signal 2
[sharkcz@devel11 libguestfs-1.35.4]$ /usr/bin/qemu-system-s390x -no-reboot -virtioconsole stdio -nodefaults  -nographic  -enable-kvm   -m 500 -kernel /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/kernel     -initrd /home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/initrd -drive file=/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/libguestfs8KFxd9/scratch.1,cache=none,format=raw,if=virtio -drive file=/home/sharkcz/libguestfs/libguestfs-1.35.4/tmp/.guestfs-1001/appliance.d/root,cache=none,if=virtio,format=raw -append 'panic=1 udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory root=/dev/vdb selinux=0 guestfs_verbose=1'
[    0.572873] Freeing unused kernel memory: 2632K (0000000000d23000 - 0000000000fb5000)
[    0.574493] Write protected read-only-after-init data: 4k
[    0.649590] random: fast init done
[    0.691835] EXT4-fs (vdb): mounting ext2 file system using the ext4 subsystem
[    0.728677] EXT4-fs (vdb): warning: mounting unchecked fs, running e2fsck is recommended
[    0.740959] EXT4-fs (vdb): mounted filesystem without journal. Opts: 
[    1.234420] mkdir (96) used greatest stack depth: 11672 bytes left
[    1.526365] od (111) used greatest stack depth: 10600 bytes left
[    1.953861] systemd-tmpfiles[112]: [/usr/lib/tmpfiles.d/systemd.conf:11] Unknown group 'utmp'.
[    1.972949] systemd-tmpfiles[112]: [/usr/lib/tmpfiles.d/systemd.conf:19] Unknown user 'systemd-network'.
[    1.977152] systemd-tmpfiles[112]: [/usr/lib/tmpfiles.d/systemd.conf:20] Unknown user 'systemd-network'.
[    1.978560] systemd-tmpfiles[112]: [/usr/lib/tmpfiles.d/systemd.conf:21] Unknown user 'systemd-network'.
[    1.994194] systemd-tmpfiles[112]: [/usr/lib/tmpfiles.d/systemd.conf:25] Unknown group 'systemd-journal'.
[    1.999812] systemd-tmpfiles[112]: [/usr/lib/tmpfiles.d/systemd.conf:26] Unknown group 'systemd-journal'.
[    2.078689] systemd-udevd[113]: starting version 231
[    3.319612] reboot: Restarting system


My conclusion is: it doesn't like the SCSI device type for the disks.

Comment 5 Richard W.M. Jones 2016-10-06 13:38:37 UTC
In comment 3 what is likely to be happening is that the
appliance kernel is crashing or hanging.  However because
we cannot see the console messages, there's no real clue
exactly what went wrong.

In comment 4 it gets a lot further than I thought, but really
you can't just run the qemu command directly.  See comment 1
for how this can be properly debugged and fixed.

Comment 6 Richard W.M. Jones 2016-10-06 13:40:07 UTC
Which is to say you should fiddle around with src/launch-direct.c
fixing the parameters until the appliance can boot with console
messages enabled.

Comment 7 Dan Horák 2016-10-06 14:19:00 UTC
I was wondering how the output should look like, so I took some hackish steps to get it from the command line. Should I check eg. the x86 build logs or is there other place to look?

Right now I think there are 2 issues - no console output (at all, no kernel messages, etc) and root disk not available because scsi is not understood.

Comment 8 Richard W.M. Jones 2016-10-06 15:22:51 UTC
Created attachment 1207973 [details]
libguestfs-test-tool output

After the qemu command is printed, you'd expect to see kernel
boot messages.  There may also be some firmware messages before
those if s/390x has such a thing.  Attached is the output
on x86_64 from:

LIBGUESTFS_BACKEND=direct libguestfs-test-tool

Comment 9 Richard W.M. Jones 2016-11-01 11:50:09 UTC
Another way to reproduce the bug:

$ virt-install --name=tmp-3b182f5a \
    --arch=s390x --ram=2048 --vcpus=1 \
    --os-type=linux --os-variant=fedora23 \
    --disk /tmp/fedora-24-s390x,size=6,format=raw \
    --serial pty \
    --location=https://dl.fedoraproject.org/pub/fedora-secondary/releases/24/Server/s390x/os/ \
    --nographics --noreboot
WARNING  No --console device added, you likely will not see text install output from the guest.

Starting install...
Retrieving file kernel.img...                               |  13 MB  00:05     
Retrieving file initrd.img...                               |  39 MB  00:16     
Allocating 'fedora-24-s390x'                                | 6.0 GB  00:00     
ERROR    internal error: process exited while connecting to monitor: 2016-11-01T11:47:48.773432Z qemu-system-s390x: -device isa-serial,chardev=charserial0,id=serial0: 'isa-serial' is not a valid device model name
Removing disk 'fedora-24-s390x'                             |    0 B  00:00     
Domain installation does not appear to have been successful.
If it was, you can restart your domain by running:
  virsh --connect qemu:///session start tmp-3b182f5a
otherwise, please restart your installation.


Removing '--serial pty' gives the error:

ERROR    'NoneType' object is not iterable


Using '--serial none' gives the error:

ERROR    unsupported configuration: unknown type presented to host for character device: none

Comment 10 Cole Robinson 2016-11-02 14:37:36 UTC
(In reply to Richard W.M. Jones from comment #9)
> 
> ERROR    internal error: process exited while connecting to monitor:
> 2016-11-01T11:47:48.773432Z qemu-system-s390x: -device
> isa-serial,chardev=charserial0,id=serial0: 'isa-serial' is not a valid
> device model name
> Removing disk 'fedora-24-s390x'                             |    0 B  00:00 

Yeah I guess see if you can find a working qemu invocation for fedora s390 and work backwards from there. There may not be a working command line though... I though I've heard that qemu doesn't emulate the s390 machine/cpu/something that fedora/rhel s390 are built to run on

> 
> Removing '--serial pty' gives the error:
> 
> ERROR    'NoneType' object is not iterable

That's a virt-install issue, fixed upstream now:

commit 7962672c713cf6d35e770f0d00068dee707b6ec9
Author: Cole Robinson <crobinso@redhat.com>
Date:   Wed Nov 2 10:27:14 2016 -0400

    virt-install: Fix error checking extra_args

Comment 11 Richard W.M. Jones 2016-11-02 15:44:59 UTC
I wasn't able to make our images boot.  The closest I got was:

  The Linux kernel requires more recent processor hardware

using the command:

  qemu-system-s390x -M s390-ccw-virtio -nographic -m 1024 -kernel dl.fedoraproject.org/pub/fedora-secondary/releases/24/Server/s390x/os/images/kernel.img -initrd dl.fedoraproject.org/pub/fedora-secondary/releases/24/Server/s390x/os/images/initrd.img -drive file=dl.fedoraproject.org/pub/fedora-secondary/releases/24/Server/s390x/os/images/install.img,format=raw,if=none,id=d1 -device virtio-blk-ccw,drive=d1,id=vd1 -drive file=disk,format=raw,if=none,id=d2 -device virtio-blk-ccw,drive=d2,id=vd2 -netdev user,id=n1 -device virtio-net-ccw,netdev=n1,id=vn1,devno=fe.0.0002

and images from:

  https://dl.fedoraproject.org/pub/fedora-secondary/releases/24/Server/s390x/os/images/

Comment 12 Fedora End Of Life 2017-02-28 10:18:46 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 26 development cycle.
Changing version to '26'.

Comment 13 Thomas Huth 2017-03-16 08:11:55 UTC
(In reply to Richard W.M. Jones from comment #11)
> I wasn't able to make our images boot.  The closest I got was:
> 
>   The Linux kernel requires more recent processor hardware

I think this is likely due to the missing STFLE emulation in QEMU. Patches are currently discussed on the mailing list:

https://lists.gnu.org/archive/html/qemu-devel/2017-03/msg00239.html

Comment 14 Cole Robinson 2017-05-04 18:14:58 UTC
I just played with this a bit. qemu s390 doesn't actually have anything that maps to libvirt's <serial> device, it only has <console> devices, target type=sclp or type=sclplm. Libvirt fills in an appropriate s390 default for <console> XML

So I think for the s390 case, libguestfs needs to s/serial/console/

There's some relevant info here too: https://specs.openstack.org/openstack/nova-specs/specs/kilo/implemented/libvirt-kvm-systemz.html

Comment 15 Richard W.M. Jones 2017-05-17 11:56:11 UTC
I posted a few patches which make this work with libguestfs:

https://www.redhat.com/archives/libguestfs/2017-May/msg00066.html
https://www.redhat.com/archives/libguestfs/2017-May/msg00067.html
https://www.redhat.com/archives/libguestfs/2017-May/msg00068.html

However I still don't believe that libvirt is *right* here
(or perhaps the problem lies with qemu, the kernel or the s/390
architecture itself).  All those #ifdefs represent pointless
complexity IMHO.

Comment 16 Thomas Huth 2017-05-17 13:45:03 UTC
(In reply to Richard W.M. Jones from comment #15)
> I posted a few patches which make this work with libguestfs:
> 
> https://www.redhat.com/archives/libguestfs/2017-May/msg00066.html
> [...]
> However I still don't believe that libvirt is *right* here

Unless you explicitely need a serial port ... can't you simply always use a <console> instead? If I get the libvirt documentation right (https://libvirt.org/formatdomain.html#elementCharConsole), the <console> device will create the right real device automatically for you, depending on the machine type and hypervisor. So on x86, you should get a serial port with it, while on s390x, you should get an sclp console instead.

Comment 17 Markus Zoeller 2017-10-11 11:15:42 UTC
(In reply to Richard W.M. Jones from comment #15)
> I posted a few patches which make this work with libguestfs:
> 
> https://www.redhat.com/archives/libguestfs/2017-May/msg00066.html
> https://www.redhat.com/archives/libguestfs/2017-May/msg00067.html
> https://www.redhat.com/archives/libguestfs/2017-May/msg00068.html
> 
> [...]

What's the current state and outlook of this? OpenStack wants to remove the NBD fallback in favor of always using libguestfs [1] and with this bug here OpenStack on s390x will lose functionality.

References:
[1] https://review.openstack.org/#/c/324720/

Comment 18 Pino Toscano 2017-10-11 11:28:10 UTC
All the patches mentioned above, and a couple of additional fixes, where done in 1.37.14.

An additional fix for the direct backend (commit 5b60dd4eff02f48d344bcdad0d4bad4676ca9168) is available in 1.37.22.

Comment 19 Markus Zoeller 2017-10-11 11:39:18 UTC
(In reply to Pino Toscano from comment #18)
> All the patches mentioned above, and a couple of additional fixes, where
> done in 1.37.14.
> 
> An additional fix for the direct backend (commit
> 5b60dd4eff02f48d344bcdad0d4bad4676ca9168) is available in 1.37.22.

Will this get backported into the Ubuntu 16.04 LTS ? Currently I see version 1.32.2:

root@z-lpar:~# apt-cache policy libguestfs-tools
libguestfs-tools:
  Installed: (none)
  Candidate: 1:1.32.2-4ubuntu2
  Version table:
     1:1.32.2-4ubuntu2 500
        500 http://ports.ubuntu.com xenial/universe s390x Packages

If it doesn't get backported, will it be available in the next LTS release Ubuntu 18.04?

Comment 20 Pino Toscano 2017-10-11 11:52:09 UTC
(In reply to Markus Zoeller from comment #19)
> (In reply to Pino Toscano from comment #18)
> > All the patches mentioned above, and a couple of additional fixes, where
> > done in 1.37.14.
> > 
> > An additional fix for the direct backend (commit
> > 5b60dd4eff02f48d344bcdad0d4bad4676ca9168) is available in 1.37.22.
> 
> Will this get backported into the Ubuntu 16.04 LTS ? Currently I see version
> 1.32.2:
> 
> root@z-lpar:~# apt-cache policy libguestfs-tools
> libguestfs-tools:
>   Installed: (none)
>   Candidate: 1:1.32.2-4ubuntu2
>   Version table:
>      1:1.32.2-4ubuntu2 500
>         500 http://ports.ubuntu.com xenial/universe s390x Packages

Between 1.32.x and 1.37.x there were lots of changes, and backporting it (even if we would) is not an easy option.  Generally speaking though, we do not control what goes in Ubuntu, and from what I know there is no active maintainer of libguestfs in Ubuntu, so I'm afraid there is basically no chance this is going to be backported.

> If it doesn't get backported, will it be available in the next LTS release
> Ubuntu 18.04?

As mentioned above, this depends on the Ubuntu packagers themselves.  Ubuntu usually gets libguestfs directly from Debian, and Debian follows only stable series (the even series, e.g. 1.32, 1.34, etc).

Comment 25 Richard W.M. Jones 2017-11-27 14:02:27 UTC
Closing since this is fixed upstream.