Bug 1726075 - qemu-system-ppc64le can't boot the kernel
Summary: qemu-system-ppc64le can't boot the kernel
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux Advanced Virtualization
Classification: Red Hat
Component: qemu-kvm
Version: 8.1
Hardware: ppc64le
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: 8.1
Assignee: David Gibson
QA Contact: Min Deng
URL:
Whiteboard:
Depends On:
Blocks: TRACKER-bugs-affecting-libguestfs 1732497 1741619
TreeView+ depends on / blocked
 
Reported: 2019-07-02 07:34 UTC by Richard W.M. Jones
Modified: 2019-11-06 07:17 UTC (History)
10 users (show)

Fixed In Version: qemu-kvm-4.1.0-1.module+el8.1.0+3966+4a23dca1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1732497 (view as bug list)
Environment:
Last Closed: 2019-11-06 07:17:18 UTC
Type: Bug
Target Upstream Version:


Attachments (Terms of Use)
build.log (3.52 MB, text/plain)
2019-07-02 07:36 UTC, Richard W.M. Jones
no flags Details
root.log (189.51 KB, text/plain)
2019-07-02 07:36 UTC, Richard W.M. Jones
no flags Details
qemu4.0's log (60.79 KB, text/plain)
2019-07-03 08:35 UTC, Min Deng
no flags Details


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2019:3723 None None None 2019-11-06 07:17:38 UTC

Description Richard W.M. Jones 2019-07-02 07:34:56 UTC
Description of problem:

The kernel crashes when run under qemu-system-ppc64le.

LC_ALL=C \
LD_LIBRARY_PATH=/builddir/build/BUILD/libguestfs-1.40.2/ruby/ext/guestfs:/builddir/build/BUILD/libguestfs-1.40.2/lib/.libs:/builddir/build/BUILD/libguestfs-1.40.2/java/.libs:/builddir/build/BUILD/libguestfs-1.40.2/gobject/.libs \
PATH=/builddir/build/BUILD/libguestfs-1.40.2/v2v:/builddir/build/BUILD/libguestfs-1.40.2/tools:/builddir/build/BUILD/libguestfs-1.40.2/test-tool:/builddir/build/BUILD/libguestfs-1.40.2/sysprep:/builddir/build/BUILD/libguestfs-1.40.2/sparsify:/builddir/build/BUILD/libguestfs-1.40.2/resize:/builddir/build/BUILD/libguestfs-1.40.2/rescue:/builddir/build/BUILD/libguestfs-1.40.2/p2v:/builddir/build/BUILD/libguestfs-1.40.2/make-fs:/builddir/build/BUILD/libguestfs-1.40.2/inspector:/builddir/build/BUILD/libguestfs-1.40.2/get-kernel:/builddir/build/BUILD/libguestfs-1.40.2/fuse:/builddir/build/BUILD/libguestfs-1.40.2/format:/builddir/build/BUILD/libguestfs-1.40.2/fish:/builddir/build/BUILD/libguestfs-1.40.2/erlang:/builddir/build/BUILD/libguestfs-1.40.2/edit:/builddir/build/BUILD/libguestfs-1.40.2/diff:/builddir/build/BUILD/libguestfs-1.40.2/dib:/builddir/build/BUILD/libguestfs-1.40.2/df:/builddir/build/BUILD/libguestfs-1.40.2/customize:/builddir/build/BUILD/libguestfs-1.40.2/cat:/builddir/build/BUILD/libguestfs-1.40.2/builder:/builddir/build/BUILD/libguestfs-1.40.2/align:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/sbin \
HOME=/builddir \
USER=mockbuild \
LOGNAME=mockbuild \
QEMU_AUDIO_DRV=none \
TMPDIR=/builddir/build/BUILD/libguestfs-1.40.2/tmp \
/usr/libexec/qemu-kvm \
-name guest=guestfs-0tqjxlglb9geando,debug-threads=on \
-S \
-object secret,id=masterKey0,format=raw,file=/builddir/.config/libvirt/qemu/lib/domain-1-guestfs-0tqjxlglb9ge/master-key.aes \
-machine pseries-rhel8.0.0,accel=tcg,usb=off,dump-guest-core=off \
-m 1024 \
-realtime mlock=off \
-smp 1,sockets=1,cores=1,threads=1 \
-uuid b059528e-f682-4c83-b606-5112f0c9e8d0 \
-display none \
-no-user-config \
-nodefaults \
-chardev socket,id=charmonitor,fd=19,server,nowait \
-mon chardev=charmonitor,id=monitor,mode=control \
-rtc base=utc,driftfix=slew \
-no-reboot \
-boot strict=on \
-kernel /builddir/build/BUILD/libguestfs-1.40.2/tmp/.guestfs-997/appliance.d/kernel \
-initrd /builddir/build/BUILD/libguestfs-1.40.2/tmp/.guestfs-997/appliance.d/initrd \
-append 'panic=1 console=hvc0 console=ttyS0 edd=off udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=/dev/sdb selinux=0 guestfs_verbose=1 TERM=vt100' \
-device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x1 \
-device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x2 \
-drive file=/builddir/build/BUILD/libguestfs-1.40.2/tmp/libguestfsgebKHW/scratch1.img,format=raw,if=none,id=drive-scsi0-0-0-0,cache=unsafe \
-device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1,write-cache=on \
-drive file=/builddir/build/BUILD/libguestfs-1.40.2/tmp/libguestfsgebKHW/overlay2.qcow2,format=qcow2,if=none,id=drive-scsi0-0-1-0,cache=unsafe \
-device scsi-hd,bus=scsi0.0,channel=0,scsi-id=1,lun=0,drive=drive-scsi0-0-1-0,id=scsi0-0-1-0,write-cache=on \
-chardev socket,id=charserial0,path=/tmp/libguestfsiaCr4Z/console.sock \
-device spapr-vty,chardev=charserial0,id=serial0,reg=0x30000000 \
-chardev socket,id=charchannel0,path=/tmp/libguestfsiaCr4Z/guestfsd.sock \
-device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.libguestfs.channel.0 \
-object rng-random,id=objrng0,filename=/dev/urandom \
-device virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.0,addr=0x3 \
-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
-msg timestamp=on
2019-07-01 23:39:04.231+0000: Domain id=1 is tainted: custom-argv
2019-07-01T23:39:04.353426Z qemu-kvm: warning: TCG doesn't support requested feature, cap-cfpc=workaround
2019-07-01T23:39:04.353459Z qemu-kvm: warning: TCG doesn't support requested feature, cap-sbbc=workaround
2019-07-01T23:39:04.353473Z qemu-kvm: warning: TCG doesn't support requested feature, cap-ibs=workaround
2019-07-01 23:39:23.162+0000: shutting down, reason=crashed

Version-Release number of selected component (if applicable):

kernel 4.18.0-80.4.2.el8_0
qemu-kvm 15:3.1.0-28.module+el8.0.1+3556+b59953c6

How reproducible:

At least once.

Steps to Reproduce:
1. Run libguestfs-test-tool.

Comment 1 Richard W.M. Jones 2019-07-02 07:36:19 UTC
Created attachment 1586544 [details]
build.log

Comment 2 Richard W.M. Jones 2019-07-02 07:36:39 UTC
Created attachment 1586545 [details]
root.log

Comment 5 Laurent Vivier 2019-07-02 16:08:11 UTC
From attachment 1586544 [details]:

[    0.097222] Modules linked in:
[    0.097799] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.18.0-80.4.2.el8_0.ppc64le #1
[    0.098371] NIP:  c00000000000b8a0 LR: c00000000000dc68 CTR: 0000000000007fff
[    0.101959] REGS: c0000000015bb800 TRAP: 0700   Not tainted  (4.18.0-80.4.2.el8_0.ppc64le)
[    0.102481] MSR:  8000000002081033 <SF,VEC,ME,IR,DR,RI,LE>  CR: 44828822  XER: 00000000
[    0.103202] CFAR: c00000000000b884 IRQMASK: 3 
[    0.103202] GPR00: c00000000001fa30 c0000000015bba80 c0000000015bd400 c00000000153f860 
[    0.103202] GPR04: c00000003a57cae0 c00000003fd29378 0000000000000000 00000000058a4b02 
[    0.103202] GPR08: 0000000000000000 0000000000007fff 0000000000000000 0000000000000000 
[    0.103202] GPR12: 0000000000008000 c0000000019f0000 00000000012db278 00000000012db270 
[    0.103202] GPR16: 00000000013a3250 00000000012daba8 000000003dc5ef20 00000000012daed0 
[    0.103202] GPR20: c00000000153ec00 c00000000153f3d0 000000003ec00000 0000000044828822 
[    0.103202] GPR24: c00000000153f860 c00000003a57cae0 000000003ec00000 c000000001105918 
[    0.103202] GPR28: c00000000153f860 c00000000153ec00 c00000003a57be80 c00000000153ec00 
[    0.108322] NIP [c00000000000b8a0] flush_count_cache+0x120/0x2480
[    0.108838] LR [c00000000000dc68] _switch+0x68/0x180
[    0.109222] Call Trace:
[    0.109560] [c0000000015bba80] [c0000000015bbac0] init_stack+0x3ac0/0x4000 (unreliable)
[    0.110232] [c0000000015bbc50] [c00000000001fa30] __switch_to+0x2e0/0x4e0
[    0.110711] [c0000000015bbcb0] [c000000000ccca54] __schedule+0x2c4/0xb20
[    0.111199] [c0000000015bbd80] [c000000000ccd768] _cond_resched+0x78/0xb0
[    0.111729] [c0000000015bbdb0] [c00000000013491c] copy_process.isra.4.part.5+0x9ac/0x14e0
[    0.112300] [c0000000015bbe40] [c000000000135524] _do_fork.constprop.7+0xd4/0x3a0
[    0.112817] [c0000000015bbeb0] [c000000000135d18] kernel_thread+0x38/0x50
[    0.113293] [c0000000015bbed0] [c000000000010390] rest_init+0x98/0xf8
[    0.113754] [c0000000015bbf00] [c00000000102435c] start_kernel+0x630/0x650
[    0.114264] [c0000000015bbf90] [c00000000000ad7c] start_here_common+0x1c/0x520
[    0.114774] Instruction dump:
[    0.115515] 48000005 48000005 48000005 48000005 48000005 4800001c 60000000 60000000 
[    0.116156] 60000000 60000000 60000000 60000000 <7d2803a6> 39207fff 7d2903a6 4c400420 
[    0.117746] ---[ end trace cc56b8036ef3f22b ]---
[    0.118139] 
[    1.119053] Kernel panic - not syncing: Fatal exception

It's fixed by:

commit fa200c95f7f99ce14b8af25ea0be478c722d3cec
Author: Greg Kurz <groug@kaod.org>
Date:   Fri Mar 22 19:03:46 2019 +0100

    target/ppc: Enable "decrement and test CTR" version of bcctr
    
    Even if all ISAs up to v3 indeed mention:
    
        If the "decrement and test CTR" option is specified (BO2=0), the
        instruction form is invalid.
    
    The UMs of all existing 64-bit server class processors say:
    
        If BO[2] = 0, the contents of CTR (before any update) are used as the
        target address and for the test of the contents of CTR to resolve the
        branch. The contents of the CTR are then decremented and written back
        to the CTR.
    
    The linux kernel has spectre v2 mitigation code that relies on a
    BO[2] = 0 variant of bcctr, which is now activated by default on
    spapr, even with TCG. This causes linux guests to panic with
    the default machine type under TCG.
    
    Since any CPU model can provide its own behaviour for invalid forms,
    we could possibly introduce a new instruction flag to handle this.
    In practice, since the behaviour is shared by all 64-bit server
    processors starting with 970 up to POWER9, let's reuse the
    PPC_SEGMENT_64B flag. Caveat: this may have to be fixed later if
    POWER10 introduces a different behaviour.
    
    The existing behaviour of throwing a program interrupt is kept for
    all other CPU models.
    
    Signed-off-by: Greg Kurz <groug@kaod.org>
    Message-Id: <155327782604.1283071.10640596307206921951.stgit@bahia.lan>
    Tested-by: Suraj Jitindar Singh <sjitindarsingh@gmail.com>
    Signed-off-by: David Gibson <david@gibson.dropbear.id.au>

You can check it's actually this problem by trying one of these workarounds:

 ... -M cap-cfpc=broken,cap-sbbc=broken,cap-ibs=broken ...

or with

  ... -M pseries-3.1.0 ...

Thanks,
Laurent

Comment 6 Laurent Vivier 2019-07-02 16:22:55 UTC
This problem can happen only in TCG mode.

Comment 7 Laurent Vivier 2019-07-02 16:32:00 UTC
(In reply to Laurent Vivier from comment #5)
...
> or with
> 
>   ... -M pseries-3.1.0 ...
> 

On RHEL, it will be more like:

  ... -M pseries-rhel7.6.0 ...

The goal is to disable the following warnings:

  qemu-kvm: warning: TCG doesn't support requested feature, cap-cfpc=workaround
  qemu-kvm: warning: TCG doesn't support requested feature, cap-sbbc=workaround
  qemu-kvm: warning: TCG doesn't support requested feature, cap-ibs=workaround

That have been introduced in qemu-kvm-3.1.0-26.module+el8.0.1+3210+5c219ce1 by:

  f21757edc554 target/ppc/spapr: Enable mitigations by default for pseries-4.0 machine type

Comment 9 David Gibson 2019-07-03 00:54:02 UTC
This problem only occurs under TCG, which AFAIK we don't actually support, so I don't think there's any cause for a z-stream backport.

The fix is already in qemu-4.0 upstream, so we'll get it via rebase for RHEL AV 8.1.

Comment 10 Min Deng 2019-07-03 07:05:16 UTC
QE reproduced the similar problem with the following steps,

Build information,
qemu-kvm-4.0.0-4.module+el8.1.0+3523+b348b848.ppc64le
kernel-4.18.0-109.el8.ppc64le

1. tar -xvf libguestfs-1.40.1.tar.gz
2. cd libguestfs-1.40.1
3. ./configure
(Require a lot of dependencies)
4. make
rm po-docs/podfiles; make -C po-docs update-po
make
make quickcheck LIBGUESTFS_BACKEND_SETTINGS=force_tcg \
     LIBGUESTFS_HV=/usr/libexec/qemu-kvm



Actual results,
[root@ibm-p9wr-17 libguestfs-1.40.1]# make quickcheck LIBGUESTFS_BACKEND_SETTINGS=force_tcg      LIBGUESTFS_HV=/usr/libexec/qemu-kvm 
fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
./run test-tool/libguestfs-test-tool 
     ************************************************************
     *                    IMPORTANT NOTICE
     *
     * When reporting bugs, include the COMPLETE, UNEDITED
     * output below in your bug report.
     *
     ************************************************************
LD_LIBRARY_PATH=/home/libguestfs-1.40.1/ruby/ext/guestfs:/home/libguestfs-1.40.1/lib/.libs:/home/libguestfs-1.40.1/java/.libs:/home/libguestfs-1.40.1/gobject/.libs
LIBGUESTFS_BACKEND_SETTINGS=force_tcg
LIBGUESTFS_CACHEDIR=/home/libguestfs-1.40.1/tmp
LIBGUESTFS_HV=/usr/libexec/qemu-kvm
LIBGUESTFS_TMPDIR=/home/libguestfs-1.40.1/tmp
LIBGUESTFS_PATH=/home/libguestfs-1.40.1/appliance
PATH=/home/libguestfs-1.40.1/v2v:/home/libguestfs-1.40.1/tools:/home/libguestfs-1.40.1/test-tool:/home/libguestfs-1.40.1/sysprep:/home/libguestfs-1.40.1/sparsify:/home/libguestfs-1.40.1/resize:/home/libguestfs-1.40.1/rescue:/home/libguestfs-1.40.1/p2v:/home/libguestfs-1.40.1/make-fs:/home/libguestfs-1.40.1/inspector:/home/libguestfs-1.40.1/get-kernel:/home/libguestfs-1.40.1/fuse:/home/libguestfs-1.40.1/format:/home/libguestfs-1.40.1/fish:/home/libguestfs-1.40.1/erlang:/home/libguestfs-1.40.1/edit:/home/libguestfs-1.40.1/diff:/home/libguestfs-1.40.1/dib:/home/libguestfs-1.40.1/df:/home/libguestfs-1.40.1/customize:/home/libguestfs-1.40.1/cat:/home/libguestfs-1.40.1/builder:/home/libguestfs-1.40.1/align:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin
XDG_RUNTIME_DIR=/run/user/0
SELinux: Disabled
guestfs_get_append: (null)
guestfs_get_autosync: 1
guestfs_get_backend: direct
guestfs_get_backend_settings: [force_tcg]
guestfs_get_cachedir: /home/libguestfs-1.40.1/tmp
guestfs_get_hv: /usr/libexec/qemu-kvm
guestfs_get_memsize: 1024
guestfs_get_network: 0
guestfs_get_path: /home/libguestfs-1.40.1/appliance
guestfs_get_pgroup: 0
guestfs_get_program: libguestfs-test-tool
guestfs_get_recovery_proc: 1
guestfs_get_smp: 1
guestfs_get_sockdir: /tmp
guestfs_get_tmpdir: /home/libguestfs-1.40.1/tmp
guestfs_get_trace: 0
guestfs_get_verbose: 1
host_cpu: powerpc64le
Launching appliance, timeout set to 600 seconds.
libguestfs: launch: program=libguestfs-test-tool
libguestfs: launch: version=1.40.1
libguestfs: launch: backend registered: unix
libguestfs: launch: backend registered: uml
libguestfs: launch: backend registered: direct
libguestfs: launch: backend=direct
libguestfs: launch: tmpdir=/home/libguestfs-1.40.1/tmp/libguestfs6NpOnf
libguestfs: launch: umask=0022
libguestfs: launch: euid=0
libguestfs: begin building supermin appliance
libguestfs: run supermin
libguestfs: command: run: /usr/bin/supermin
libguestfs: command: run: \ --build
libguestfs: command: run: \ --verbose
libguestfs: command: run: \ --if-newer
libguestfs: command: run: \ --lock /home/libguestfs-1.40.1/tmp/.guestfs-0/lock
libguestfs: command: run: \ --copy-kernel
libguestfs: command: run: \ -f ext2
libguestfs: command: run: \ --host-cpu powerpc64le
libguestfs: command: run: \ /home/libguestfs-1.40.1/appliance/supermin.d
libguestfs: command: run: \ -o /home/libguestfs-1.40.1/tmp/.guestfs-0/appliance.d
supermin: version: 5.1.19
supermin: rpm: detected RPM version 4.14
supermin: package handler: fedora/rpm
supermin: acquiring lock on /home/libguestfs-1.40.1/tmp/.guestfs-0/lock
supermin: if-newer: output does not need rebuilding
libguestfs: finished building supermin appliance
libguestfs: begin testing qemu features
libguestfs: checking for previously cached test results of /usr/libexec/qemu-kvm, in /home/libguestfs-1.40.1/tmp/.guestfs-0
libguestfs: loading previously cached test results
libguestfs: qemu version: 4.0
libguestfs: qemu mandatory locking: yes
libguestfs: qemu KVM: enabled
libguestfs: finished testing qemu features
libguestfs: command: run: dmesg | grep -Eoh 'lpj=[[:digit:]]+'
libguestfs: read_lpj_from_dmesg: external command exited with error status 1
libguestfs: read_lpj_from_files: no boot messages files are readable
/usr/libexec/qemu-kvm \
    -global virtio-blk-pci.scsi=off \
    -no-user-config \
    -enable-fips \
    -nodefaults \
    -display none \
    -machine pseries,accel=tcg \
    -m 1024 \
    -no-reboot \
    -rtc driftfix=slew \
    -kernel /home/libguestfs-1.40.1/tmp/.guestfs-0/appliance.d/kernel \
    -initrd /home/libguestfs-1.40.1/tmp/.guestfs-0/appliance.d/initrd \
    -object rng-random,filename=/dev/urandom,id=rng0 \
    -device virtio-rng-pci,rng=rng0 \
    -device virtio-scsi-pci,id=scsi \
    -drive file=/home/libguestfs-1.40.1/tmp/libguestfs6NpOnf/scratch1.img,cache=unsafe,format=raw,id=hd0,if=none \
    -device scsi-hd,drive=hd0 \
    -drive file=/home/libguestfs-1.40.1/tmp/.guestfs-0/appliance.d/root,snapshot=on,id=appliance,cache=unsafe,if=none,format=raw \
    -device scsi-hd,drive=appliance \
    -device virtio-serial-pci \
    -serial stdio \
    -chardev socket,path=/tmp/libguestfsu9Mxap/guestfsd.sock,id=channel0 \
    -device virtserialport,chardev=channel0,name=org.libguestfs.channel.0 \
    -append "panic=1 console=hvc0 console=ttyS0 edd=off udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=/dev/sdb selinux=0 guestfs_verbose=1 TERM=xterm-256color"
qemu-kvm: warning: TCG doesn't support requested feature, cap-cfpc=workaround
qemu-kvm: warning: TCG doesn't support requested feature, cap-sbbc=workaround
qemu-kvm: warning: TCG doesn't support requested feature, cap-ibs=workaround
qemu-kvm: warning: global mc146818rtc.lost_tick_policy has invalid class name

Comment 11 Min Deng 2019-07-03 07:07:44 UTC
(In reply to David Gibson from comment #9)
> This problem only occurs under TCG, which AFAIK we don't actually support,
> so I don't think there's any cause for a z-stream backport.
> 
> The fix is already in qemu-4.0 upstream, so we'll get it via rebase for RHEL
> AV 8.1.

I used qemu-kvm-4.0.0-4.module+el8.1.0+3523+b348b848.ppc64le,it is still reproducible.

Comment 12 Laurent Vivier 2019-07-03 07:27:30 UTC
(In reply to Min Deng from comment #11)
> (In reply to David Gibson from comment #9)
> > This problem only occurs under TCG, which AFAIK we don't actually support,
> > so I don't think there's any cause for a z-stream backport.
> > 
> > The fix is already in qemu-4.0 upstream, so we'll get it via rebase for RHEL
> > AV 8.1.
> 
> I used qemu-kvm-4.0.0-4.module+el8.1.0+3523+b348b848.ppc64le,it is still
> reproducible.

Could you describe?
The problem is the kernel crash, not the qemu-kvm warnings.

Comment 13 Min Deng 2019-07-03 08:00:13 UTC
(In reply to Laurent Vivier from comment #12)
> (In reply to Min Deng from comment #11)
> > (In reply to David Gibson from comment #9)
> > > This problem only occurs under TCG, which AFAIK we don't actually support,
> > > so I don't think there's any cause for a z-stream backport.
> > > 
> > > The fix is already in qemu-4.0 upstream, so we'll get it via rebase for RHEL
> > > AV 8.1.
> > 
> > I used qemu-kvm-4.0.0-4.module+el8.1.0+3523+b348b848.ppc64le,it is still
> > reproducible.
> 
> Could you describe?
> The problem is the kernel crash, not the qemu-kvm warnings.

Hi Laurent,
   Well received,I will double confirm it.
   And even if I use qemu4.0,the warning will still be there.Thanks
Best regards
Min

Comment 14 Min Deng 2019-07-03 08:34:47 UTC
I upload completed log of "qemu-kvm-4.0.0-4.module+el8.1.0+3523+b348b848.ppc64le" ,warnings are there but the whole test suite can finish finally,I didn't see crash problem.
Besides that,I will also upload the log of the problem build as soon as possible since I don't have relevant host at this moment.Thanks a lot.

Comment 15 Min Deng 2019-07-03 08:35:28 UTC
Created attachment 1586979 [details]
qemu4.0's log

Comment 16 Min Deng 2019-07-04 03:33:45 UTC
Do a summary for this bug 
1.QE reproduced the bug with steps from comment10 with build qemu-kvm-3.1.0-26.module+el8.0.1+3210+5c219ce1.ppc64le
  Actual results,
The log is following as below,
[    0.069530] NIP:  c00000000000b8a0 LR: c00000000000dc68 CTR: 0000000000007fff
[    0.069530] NIP:  c00000000000b8a0 LR: c00000000000dc68 CTR: 0000000000007fff
[    0.069900] REGS: c00000000165f7f0 TRAP: 0700   Not tainted  (4.18.0-109.el8.ppc64le)
[    0.069900] REGS: c00000000165f7f0 TRAP: 0700   Not tainted  (4.18.0-109.el8.ppc64le)
[    0.070232] MSR:  8000000002081033 <SF,VEC,ME,IR,DR,RI,LE>  CR: 44828822  XER: 00000000
[    0.070232] MSR:  8000000002081033 <SF,VEC,ME,IR,DR,RI,LE>  CR: 44828822  XER: 00000000
[    0.070756] CFAR: c00000000000b884 IRQMASK: 3 
[    0.070756] GPR00: c00000000001fa10 c00000000165fa70 c000000001662300 c0000000015e33e0 
[    0.070756] GPR04: c00000003a576ce0 c00000003fd29478 0000000000000000 0000000003e05274 
[    0.070756] GPR08: 0000000000000000 0000000000007fff 0000000000000000 0000000000000000 
[    0.070756] GPR12: 0000000000008000 c000000001a80000 0000000001356478 0000000001356470 
[    0.070756] GPR16: 0000000001421698 0000000001355da8 000000003dc5ef20 00000000013560d0 
[    0.070756] GPR20: c0000000015e2780 c0000000015e2f50 000000003eb70000 0000000044828822 
[    0.070756] GPR24: c0000000015e33e0 c00000003a576ce0 000000003eb70000 c000000001195998 
[    0.070756] GPR28: c0000000015e33e0 c0000000015e2780 c00000003a576080 c0000000015e2780 
[    0.070756] CFAR: c00000000000b884 IRQMASK: 3 
[    0.070756] GPR00: c00000000001fa10 c00000000165fa70 c000000001662300 c0000000015e33e0 
[    0.070756] GPR04: c00000003a576ce0 c00000003fd29478 0000000000000000 0000000003e05274 
[    0.070756] GPR08: 0000000000000000 0000000000007fff 0000000000000000 0000000000000000 
[    0.070756] GPR12: 0000000000008000 c000000001a80000 0000000001356478 0000000001356470 
[    0.070756] GPR16: 0000000001421698 0000000001355da8 000000003dc5ef20 00000000013560d0 
[    0.070756] GPR20: c0000000015e2780 c0000000015e2f50 000000003eb70000 0000000044828822 
[    0.070756] GPR24: c0000000015e33e0 c00000003a576ce0 000000003eb70000 c000000001195998 
[    0.070756] GPR28: c0000000015e33e0 c0000000015e2780 c00000003a576080 c0000000015e2780 
[    0.075589] NIP [c00000000000b8a0] flush_count_cache+0x120/0x2480
[    0.075589] NIP [c00000000000b8a0] flush_count_cache+0x120/0x2480
[    0.075908] LR [c00000000000dc68] _switch+0x68/0x180
[    0.075908] LR [c00000000000dc68] _switch+0x68/0x180
[    0.076179] Call Trace:
[    0.076179] Call Trace:
[    0.076469] [c00000000165fa70] [c00000000165fab0] init_stack+0x3ab0/0x4000 (unreliable)
[    0.076469] [c00000000165fa70] [c00000000165fab0] init_stack+0x3ab0/0x4000 (unreliable)
[    0.076946] [c00000000165fc40] [c00000000001fa10] __switch_to+0x2e0/0x4e0
[    0.076946] [c00000000165fc40] [c00000000001fa10] __switch_to+0x2e0/0x4e0
[    0.077275] [c00000000165fca0] [c000000000d3bcf4] __schedule+0x2c4/0xb20
[    0.077275] [c00000000165fca0] [c000000000d3bcf4] __schedule+0x2c4/0xb20
[    0.077594] [c00000000165fd70] [c000000000d3ca08] _cond_resched+0x78/0xb0
[    0.077594] [c00000000165fd70] [c000000000d3ca08] _cond_resched+0x78/0xb0
[    0.077986] [c00000000165fda0] [c000000000140240] copy_process.isra.4.part.5+0xcc0/0x16a0
[    0.077986] [c00000000165fda0] [c000000000140240] copy_process.isra.4.part.5+0xcc0/0x16a0
[    0.078363] [c00000000165fe40] [c000000000140cf4] _do_fork.constprop.7+0xd4/0x3a0
[    0.078363] [c00000000165fe40] [c000000000140cf4] _do_fork.constprop.7+0xd4/0x3a0
[    0.078706] [c00000000165feb0] [c0000000001414f8] kernel_thread+0x38/0x50
[    0.078706] [c00000000165feb0] [c0000000001414f8] kernel_thread+0x38/0x50
[    0.079020] [c00000000165fed0] [c000000000010390] rest_init+0x98/0xf8
[    0.079020] [c00000000165fed0] [c000000000010390] rest_init+0x98/0xf8
[    0.079333] [c00000000165ff00] [c0000000010a4354] start_kernel+0x630/0x650
[    0.079333] [c00000000165ff00] [c0000000010a4354] start_kernel+0x630/0x650
[    0.079671] [c00000000165ff90] [c00000000000ad7c] start_here_common+0x1c/0x520
[    0.079671] [c00000000165ff90] [c00000000000ad7c] start_here_common+0x1c/0x520
[    0.080024] Instruction dump:
[    0.080024] Instruction dump:
[    0.080637] 48000005 48000005 48000005 48000005 48000005 4800001c 60000000 60000000 
[    0.080637] 48000005 48000005 48000005 48000005 48000005 4800001c 60000000 60000000 
[    0.081089] 60000000 60000000 60000000 60000000 <7d2803a6> 39207fff 7d2903a6 4c400420 
[    0.081089] 60000000 60000000 60000000 60000000 <7d2803a6> 39207fff 7d2903a6 4c400420 
[    0.082459] ---[ end trace f3bf4a1a785db234 ]---
[    0.082459] ---[ end trace f3bf4a1a785db234 ]---
[    0.082735] 
[    0.082735] 
[    1.083529] Kernel panic - not syncing: Fatal exception
[    1.083529] Kernel panic - not syncing: Fatal exception


2.According to comment14,the problem has been fixed on the build 
  kernel-4.18.0-109.el8.ppc64le
  qemu-kvm-4.0.0-4.module+el8.1.0+3523+b348b848.ppc64le

  Thanks.

Comment 18 Pino Toscano 2019-08-15 14:58:15 UTC
Tried a scratch build with qemu-kvm 15:4.1.0-1.module+el8.1.0+3966+4a23dca1:

LC_ALL=C \
LD_LIBRARY_PATH=/builddir/build/BUILD/libguestfs-1.40.2/ruby/ext/guestfs:/builddir/build/BUILD/libguestfs-1.40.2/lib/.libs:/builddir/build/BUILD/libguestfs-1.40.2/java/.libs:/builddir/build/BUILD/libguestfs-1.40.2/gobject/.libs \
PATH=/builddir/build/BUILD/libguestfs-1.40.2/v2v:/builddir/build/BUILD/libguestfs-1.40.2/tools:/builddir/build/BUILD/libguestfs-1.40.2/test-tool:/builddir/build/BUILD/libguestfs-1.40.2/sysprep:/builddir/build/BUILD/libguestfs-1.40.2/sparsify:/builddir/build/BUILD/libguestfs-1.40.2/resize:/builddir/build/BUILD/libguestfs-1.40.2/rescue:/builddir/build/BUILD/libguestfs-1.40.2/p2v:/builddir/build/BUILD/libguestfs-1.40.2/make-fs:/builddir/build/BUILD/libguestfs-1.40.2/inspector:/builddir/build/BUILD/libguestfs-1.40.2/get-kernel:/builddir/build/BUILD/libguestfs-1.40.2/fuse:/builddir/build/BUILD/libguestfs-1.40.2/format:/builddir/build/BUILD/libguestfs-1.40.2/fish:/builddir/build/BUILD/libguestfs-1.40.2/erlang:/builddir/build/BUILD/libguestfs-1.40.2/edit:/builddir/build/BUILD/libguestfs-1.40.2/diff:/builddir/build/BUILD/libguestfs-1.40.2/dib:/builddir/build/BUILD/libguestfs-1.40.2/df:/builddir/build/BUILD/libguestfs-1.40.2/customize:/builddir/build/BUILD/libguestfs-1.40.2/cat:/builddir/build/BUILD/libguestfs-1.40.2/builder:/builddir/build/BUILD/libguestfs-1.40.2/align:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/sbin \
HOME=/builddir \
USER=mockbuild \
LOGNAME=mockbuild \
XDG_CACHE_HOME=/builddir/.config/libvirt/qemu/lib/domain-1-guestfs-16ztguqhzlym/.cache \
QEMU_AUDIO_DRV=none \
TMPDIR=/builddir/build/BUILD/libguestfs-1.40.2/tmp \
/usr/libexec/qemu-kvm \
-name guest=guestfs-16ztguqhzlym44zc,debug-threads=on \
-S \
-object secret,id=masterKey0,format=raw,file=/builddir/.config/libvirt/qemu/lib/domain-1-guestfs-16ztguqhzlym/master-key.aes \
-machine pseries-rhel8.1.0,accel=tcg,usb=off,dump-guest-core=off \
-m 1024 \
-overcommit mem-lock=off \
-smp 1,sockets=1,cores=1,threads=1 \
-uuid ad480930-73f7-4828-9591-c610440edae5 \
-display none \
-no-user-config \
-nodefaults \
-chardev socket,id=charmonitor,fd=25,server,nowait \
-mon chardev=charmonitor,id=monitor,mode=control \
-rtc base=utc,driftfix=slew \
-no-reboot \
-boot strict=on \
-kernel /builddir/build/BUILD/libguestfs-1.40.2/tmp/.guestfs-997/appliance.d/kernel \
-initrd /builddir/build/BUILD/libguestfs-1.40.2/tmp/.guestfs-997/appliance.d/initrd \
-append 'panic=1 console=hvc0 console=ttyS0 edd=off udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=/dev/sdb selinux=0 guestfs_verbose=1 TERM=vt100' \
-device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x1 \
-device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x2 \
-drive file=/builddir/build/BUILD/libguestfs-1.40.2/tmp/libguestfslqpQwu/scratch1.img,format=raw,if=none,id=drive-scsi0-0-0-0,cache=unsafe \
-device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,device_id=drive-scsi0-0-0-0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1,write-cache=on \
-drive file=/builddir/build/BUILD/libguestfs-1.40.2/tmp/libguestfslqpQwu/overlay2.qcow2,format=qcow2,if=none,id=drive-scsi0-0-1-0,cache=unsafe \
-device scsi-hd,bus=scsi0.0,channel=0,scsi-id=1,lun=0,device_id=drive-scsi0-0-1-0,drive=drive-scsi0-0-1-0,id=scsi0-0-1-0,write-cache=on \
-chardev socket,id=charserial0,path=/tmp/libguestfsKTZUIL/console.sock \
-device spapr-vty,chardev=charserial0,id=serial0,reg=0x30000000 \
-chardev socket,id=charchannel0,path=/tmp/libguestfsKTZUIL/guestfsd.sock \
-device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.libguestfs.channel.0 \
-object rng-random,id=objrng0,filename=/dev/urandom \
-device virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.0,addr=0x3 \
-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
-msg timestamp=on
2019-08-15 14:48:08.676+0000: Domain id=1 is tainted: custom-argv
2019-08-15T14:48:08.773926Z qemu-kvm: warning: TCG doesn't support requested feature, cap-cfpc=workaround
2019-08-15T14:48:08.773971Z qemu-kvm: warning: TCG doesn't support requested feature, cap-sbbc=workaround
2019-08-15T14:48:08.774023Z qemu-kvm: warning: TCG doesn't support requested feature, cap-ibs=workaround
2019-08-15T14:48:08.800131Z qemu-kvm: warning: global mc146818rtc.lost_tick_policy has invalid class name
2019-08-15 14:48:24.646+0000: shutting down, reason=destroyed

Comment 19 Min Deng 2019-08-19 07:49:55 UTC
QE tried the issue on the build qemu-kvm-4.1.0-1.module+el8.1.0+3966+4a23dca1.ppc64le
,found out there's still problem as the following so could you please have a look on them ? 

The same issue will not be reproduced on build 
Build information,
qemu-kvm-core-4.0.0-6.module+el8.1.0+3736+a2aefea3.ppc64le
The test could finish correctly.
make quickcheck LIBGUESTFS_BACKEND_SETTINGS=force_tcg      LIBGUESTFS_HV=/usr/libexec/qemu-kvm
===== TEST FINISHED OK =====


On the build qemu-kvm-4.1.0-1.module+el8.1.0+3966+4a23dca1.ppc64le,the results were as following,please help to have a check in time,thank you.
(workspace) [root@ibm-p9wr-17 libguestfs-1.40.2]# make quickcheck LIBGUESTFS_BACKEND_SETTINGS=force_tcg      LIBGUESTFS_HV=/usr/libexec/qemu-kvm
fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
./run test-tool/libguestfs-test-tool 
     ************************************************************
     *                    IMPORTANT NOTICE
     *
     * When reporting bugs, include the COMPLETE, UNEDITED
     * output below in your bug report.
     *
     ************************************************************
LD_LIBRARY_PATH=/home/libguestfs-1.40.2/ruby/ext/guestfs:/home/libguestfs-1.40.2/lib/.libs:/home/libguestfs-1.40.2/java/.libs:/home/libguestfs-1.40.2/gobject/.libs
LIBGUESTFS_BACKEND_SETTINGS=force_tcg
LIBGUESTFS_CACHEDIR=/home/libguestfs-1.40.2/tmp
LIBGUESTFS_HV=/usr/libexec/qemu-kvm
LIBGUESTFS_TMPDIR=/home/libguestfs-1.40.2/tmp
LIBGUESTFS_PATH=/home/libguestfs-1.40.2/appliance
PATH=/home/libguestfs-1.40.2/v2v:/home/libguestfs-1.40.2/tools:/home/libguestfs-1.40.2/test-tool:/home/libguestfs-1.40.2/sysprep:/home/libguestfs-1.40.2/sparsify:/home/libguestfs-1.40.2/resize:/home/libguestfs-1.40.2/rescue:/home/libguestfs-1.40.2/p2v:/home/libguestfs-1.40.2/make-fs:/home/libguestfs-1.40.2/inspector:/home/libguestfs-1.40.2/get-kernel:/home/libguestfs-1.40.2/fuse:/home/libguestfs-1.40.2/format:/home/libguestfs-1.40.2/fish:/home/libguestfs-1.40.2/erlang:/home/libguestfs-1.40.2/edit:/home/libguestfs-1.40.2/diff:/home/libguestfs-1.40.2/dib:/home/libguestfs-1.40.2/df:/home/libguestfs-1.40.2/customize:/home/libguestfs-1.40.2/cat:/home/libguestfs-1.40.2/builder:/home/libguestfs-1.40.2/align:/home/kar/workspace/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin
XDG_RUNTIME_DIR=/run/user/0
SELinux: Enforcing
guestfs_get_append: (null)
guestfs_get_autosync: 1
guestfs_get_backend: direct
guestfs_get_backend_settings: [force_tcg]
guestfs_get_cachedir: /home/libguestfs-1.40.2/tmp
guestfs_get_hv: /usr/libexec/qemu-kvm
guestfs_get_memsize: 1024
guestfs_get_network: 0
guestfs_get_path: /home/libguestfs-1.40.2/appliance
guestfs_get_pgroup: 0
guestfs_get_program: libguestfs-test-tool
guestfs_get_recovery_proc: 1
guestfs_get_smp: 1
guestfs_get_sockdir: /tmp
guestfs_get_tmpdir: /home/libguestfs-1.40.2/tmp
guestfs_get_trace: 0
guestfs_get_verbose: 1
host_cpu: powerpc64le
Launching appliance, timeout set to 600 seconds.
libguestfs: launch: program=libguestfs-test-tool
libguestfs: launch: version=1.40.2
libguestfs: launch: backend registered: unix
libguestfs: launch: backend registered: uml
libguestfs: launch: backend registered: direct
libguestfs: launch: backend=direct
libguestfs: launch: tmpdir=/home/libguestfs-1.40.2/tmp/libguestfs5mOzvn
libguestfs: launch: umask=0022
libguestfs: launch: euid=0
libguestfs: begin building supermin appliance
libguestfs: run supermin
libguestfs: command: run: /usr/bin/supermin
libguestfs: command: run: \ --build
libguestfs: command: run: \ --verbose
libguestfs: command: run: \ --if-newer
libguestfs: command: run: \ --lock /home/libguestfs-1.40.2/tmp/.guestfs-0/lock
libguestfs: command: run: \ --copy-kernel
libguestfs: command: run: \ -f ext2
libguestfs: command: run: \ --host-cpu powerpc64le
libguestfs: command: run: \ /home/libguestfs-1.40.2/appliance/supermin.d
libguestfs: command: run: \ -o /home/libguestfs-1.40.2/tmp/.guestfs-0/appliance.d
supermin: version: 5.1.19
supermin: rpm: detected RPM version 4.14
supermin: package handler: fedora/rpm
supermin: acquiring lock on /home/libguestfs-1.40.2/tmp/.guestfs-0/lock
supermin: build: /home/libguestfs-1.40.2/appliance/supermin.d
supermin: reading the supermin appliance
supermin: build: visiting /home/libguestfs-1.40.2/appliance/supermin.d/base.tar.gz type gzip base image (tar)
supermin: build: visiting /home/libguestfs-1.40.2/appliance/supermin.d/daemon.tar.gz type gzip base image (tar)
supermin: build: visiting /home/libguestfs-1.40.2/appliance/supermin.d/excludefiles type uncompressed excludefiles
supermin: build: visiting /home/libguestfs-1.40.2/appliance/supermin.d/hostfiles type uncompressed hostfiles
supermin: build: visiting /home/libguestfs-1.40.2/appliance/supermin.d/init.tar.gz type gzip base image (tar)
supermin: build: visiting /home/libguestfs-1.40.2/appliance/supermin.d/packages type uncompressed packages
supermin: build: visiting /home/libguestfs-1.40.2/appliance/supermin.d/udev-rules.tar.gz type gzip base image (tar)
supermin: mapping package names to installed packages
supermin: resolving full list of package dependencies
supermin: build: 178 packages, including dependencies
supermin: build: 32589 files
supermin: build: 7104 files, after matching excludefiles
supermin: build: 7105 files, after adding hostfiles
supermin: build: 7091 files, after removing unreadable files
supermin: build: 7123 files, after munging
supermin: kernel: looking for kernel using environment variables ...
supermin: kernel: looking for kernels in /lib/modules/*/vmlinuz ...
supermin: kernel: picked vmlinuz /lib/modules/4.18.0-134.el8.ppc64le/vmlinuz
supermin: kernel: kernel_version 4.18.0-134.el8.ppc64le
supermin: kernel: modpath /lib/modules/4.18.0-134.el8.ppc64le
supermin: ext2: creating empty ext2 filesystem '/home/libguestfs-1.40.2/tmp/.guestfs-0/appliance.d.jh71vgfr/root'
supermin: ext2: populating from base image
supermin: ext2: copying files from host filesystem
supermin: ext2: copying kernel modules
supermin: ext2: creating minimal initrd '/home/libguestfs-1.40.2/tmp/.guestfs-0/appliance.d.jh71vgfr/initrd'
supermin: ext2: wrote 22 modules to minimal initrd
supermin: renaming /home/libguestfs-1.40.2/tmp/.guestfs-0/appliance.d.jh71vgfr to /home/libguestfs-1.40.2/tmp/.guestfs-0/appliance.d
libguestfs: finished building supermin appliance
libguestfs: begin testing qemu features
libguestfs: checking for previously cached test results of /usr/libexec/qemu-kvm, in /home/libguestfs-1.40.2/tmp/.guestfs-0
libguestfs: loading previously cached test results
libguestfs: qemu version: 4.0
libguestfs: qemu mandatory locking: yes
libguestfs: qemu KVM: enabled
libguestfs: finished testing qemu features
libguestfs: command: run: dmesg | grep -Eoh 'lpj=[[:digit:]]+'
libguestfs: read_lpj_from_dmesg: external command exited with error status 1
libguestfs: read_lpj_from_files: no boot messages files are readable
/usr/libexec/qemu-kvm \
    -global virtio-blk-pci.scsi=off \
    -no-user-config \
    -enable-fips \
    -nodefaults \
    -display none \
    -machine pseries,accel=tcg \
    -m 1024 \
    -no-reboot \
    -rtc driftfix=slew \
    -kernel /home/libguestfs-1.40.2/tmp/.guestfs-0/appliance.d/kernel \
    -initrd /home/libguestfs-1.40.2/tmp/.guestfs-0/appliance.d/initrd \
    -object rng-random,filename=/dev/urandom,id=rng0 \
    -device virtio-rng-pci,rng=rng0 \
    -device virtio-scsi-pci,id=scsi \
    -drive file=/home/libguestfs-1.40.2/tmp/libguestfs5mOzvn/scratch1.img,cache=unsafe,format=raw,id=hd0,if=none \
    -device scsi-hd,drive=hd0 \
    -drive file=/home/libguestfs-1.40.2/tmp/.guestfs-0/appliance.d/root,snapshot=on,id=appliance,cache=unsafe,if=none,format=raw \
    -device scsi-hd,drive=appliance \
    -device virtio-serial-pci \
    -serial stdio \
    -chardev socket,path=/tmp/libguestfsM8yjvT/guestfsd.sock,id=channel0 \
    -device virtserialport,chardev=channel0,name=org.libguestfs.channel.0 \
    -append "panic=1 console=hvc0 console=ttyS0 edd=off udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=/dev/sdb selinux=0 guestfs_verbose=1 TERM=xterm-256color"
qemu-kvm: warning: TCG doesn't support requested feature, cap-cfpc=workaround
qemu-kvm: warning: TCG doesn't support requested feature, cap-sbbc=workaround
qemu-kvm: warning: TCG doesn't support requested feature, cap-ibs=workaround
qemu-kvm: warning: global mc146818rtc.lost_tick_policy has invalid class name


SLOF\x1b[0m\x1b[?25l **********************************************************************
\x1b[1mQEMU Starting
\x1b[0m Build Date = Jul 23 2019 04:40:55
 FW Version = mockbuild@ release 20190703
 Press "s" to enter Open Firmware.

Populating /vdevice methods
Populating /vdevice/vty@71000000
Populating /vdevice/nvram@71000001
Populating /pci@800000020000000
                     00 0000 (D) : 1af4 1005    unknown-legacy-device*
                     00 0800 (D) : 1af4 1004    virtio [ scsi ]
Populating /pci@800000020000000/scsi@1
       SCSI: Looking for devices
          100000000000000 DISK     : "QEMU     QEMU HARDDISK    2.5+"
          101000000000000 DISK     : "QEMU     QEMU HARDDISK    2.5+"
                     00 1000 (D) : 1af4 1003    virtio [ serial ]
No NVRAM common partition, re-initializing...
Scanning USB 
Using default console: /vdevice/vty@71000000
Detected RAM kernel at 400000 (1a7b860 bytes) 
     
  Welcome to Open Firmware

  Copyright (c) 2004, 2017 IBM Corporation All rights reserved.
  This program and the accompanying materials are made available
  under the terms of the BSD License available at
  http://www.opensource.org/licenses/bsd-license.php

Booting from memory...
OF stdout device is: /vdevice/vty@71000000
Preparing to boot Linux version 4.18.0-134.el8.ppc64le (mockbuild@ppc-061.build.eng.bos.redhat.com) (gcc version 8.3.1 20190507 (Red Hat 8.3.1-4) (GCC)) #1 SMP Thu Aug 15 17:39:09 UTC 2019
Detected machine type: 0000000000000101
command line: panic=1 console=hvc0 console=ttyS0 edd=off udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=/dev/sdb selinux=0 guestfs_verbose=1 TERM=xterm-256color
Max number of cores passed to firmware: 2048 (NR_CPUS = 2048)
Calling ibm,client-architecture-support...libguestfs: error: appliance closed the connection unexpectedly, see earlier error messages
libguestfs: child_cleanup: 0x3e420d30: child process died
libguestfs: sending SIGTERM to process 139946
libguestfs: qemu maxrss 153472K
libguestfs: error: guestfs_launch failed, see earlier error messages
libguestfs: closing guestfs handle 0x3e420d30 (state 0)
libguestfs: command: run: rm
libguestfs: command: run: \ -rf /home/libguestfs-1.40.2/tmp/libguestfs5mOzvn
libguestfs: command: run: rm
libguestfs: command: run: \ -rf /tmp/libguestfsM8yjvT
make: *** [Makefile:2946: quickcheck] Error 1

Comment 20 Laurent Vivier 2019-08-19 14:24:24 UTC
(In reply to Min Deng from comment #19)
> QE tried the issue on the build
> qemu-kvm-4.1.0-1.module+el8.1.0+3966+4a23dca1.ppc64le
> ,found out there's still problem as the following so could you please have a
> look on them ? 
> 
> The same issue will not be reproduced on build 
> Build information,
> qemu-kvm-core-4.0.0-6.module+el8.1.0+3736+a2aefea3.ppc64le
> The test could finish correctly.
> make quickcheck LIBGUESTFS_BACKEND_SETTINGS=force_tcg     
> LIBGUESTFS_HV=/usr/libexec/qemu-kvm
> ===== TEST FINISHED OK =====
> 
> 
> On the build qemu-kvm-4.1.0-1.module+el8.1.0+3966+4a23dca1.ppc64le,the
> results were as following,please help to have a check in time,thank you.
> (workspace) [root@ibm-p9wr-17 libguestfs-1.40.2]# make quickcheck
> LIBGUESTFS_BACKEND_SETTINGS=force_tcg     
> LIBGUESTFS_HV=/usr/libexec/qemu-kvm
> fatal: not a git repository (or any parent up to mount point /)
> Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
> ./run test-tool/libguestfs-test-tool 
>      ************************************************************
>      *                    IMPORTANT NOTICE
>      *
>      * When reporting bugs, include the COMPLETE, UNEDITED
>      * output below in your bug report.
>      *
>      ************************************************************
> LD_LIBRARY_PATH=/home/libguestfs-1.40.2/ruby/ext/guestfs:/home/libguestfs-1.
> 40.2/lib/.libs:/home/libguestfs-1.40.2/java/.libs:/home/libguestfs-1.40.2/
> gobject/.libs
> LIBGUESTFS_BACKEND_SETTINGS=force_tcg
> LIBGUESTFS_CACHEDIR=/home/libguestfs-1.40.2/tmp
> LIBGUESTFS_HV=/usr/libexec/qemu-kvm
> LIBGUESTFS_TMPDIR=/home/libguestfs-1.40.2/tmp
> LIBGUESTFS_PATH=/home/libguestfs-1.40.2/appliance
> PATH=/home/libguestfs-1.40.2/v2v:/home/libguestfs-1.40.2/tools:/home/
> libguestfs-1.40.2/test-tool:/home/libguestfs-1.40.2/sysprep:/home/libguestfs-
> 1.40.2/sparsify:/home/libguestfs-1.40.2/resize:/home/libguestfs-1.40.2/
> rescue:/home/libguestfs-1.40.2/p2v:/home/libguestfs-1.40.2/make-fs:/home/
> libguestfs-1.40.2/inspector:/home/libguestfs-1.40.2/get-kernel:/home/
> libguestfs-1.40.2/fuse:/home/libguestfs-1.40.2/format:/home/libguestfs-1.40.
> 2/fish:/home/libguestfs-1.40.2/erlang:/home/libguestfs-1.40.2/edit:/home/
> libguestfs-1.40.2/diff:/home/libguestfs-1.40.2/dib:/home/libguestfs-1.40.2/
> df:/home/libguestfs-1.40.2/customize:/home/libguestfs-1.40.2/cat:/home/
> libguestfs-1.40.2/builder:/home/libguestfs-1.40.2/align:/home/kar/workspace/
> bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin
> XDG_RUNTIME_DIR=/run/user/0
> SELinux: Enforcing
> guestfs_get_append: (null)
> guestfs_get_autosync: 1
> guestfs_get_backend: direct
> guestfs_get_backend_settings: [force_tcg]
> guestfs_get_cachedir: /home/libguestfs-1.40.2/tmp
> guestfs_get_hv: /usr/libexec/qemu-kvm
> guestfs_get_memsize: 1024
> guestfs_get_network: 0
> guestfs_get_path: /home/libguestfs-1.40.2/appliance
> guestfs_get_pgroup: 0
> guestfs_get_program: libguestfs-test-tool
> guestfs_get_recovery_proc: 1
> guestfs_get_smp: 1
> guestfs_get_sockdir: /tmp
> guestfs_get_tmpdir: /home/libguestfs-1.40.2/tmp
> guestfs_get_trace: 0
> guestfs_get_verbose: 1
> host_cpu: powerpc64le
> Launching appliance, timeout set to 600 seconds.
> libguestfs: launch: program=libguestfs-test-tool
> libguestfs: launch: version=1.40.2
> libguestfs: launch: backend registered: unix
> libguestfs: launch: backend registered: uml
> libguestfs: launch: backend registered: direct
> libguestfs: launch: backend=direct
> libguestfs: launch: tmpdir=/home/libguestfs-1.40.2/tmp/libguestfs5mOzvn
> libguestfs: launch: umask=0022
> libguestfs: launch: euid=0
> libguestfs: begin building supermin appliance
> libguestfs: run supermin
> libguestfs: command: run: /usr/bin/supermin
> libguestfs: command: run: \ --build
> libguestfs: command: run: \ --verbose
> libguestfs: command: run: \ --if-newer
> libguestfs: command: run: \ --lock
> /home/libguestfs-1.40.2/tmp/.guestfs-0/lock
> libguestfs: command: run: \ --copy-kernel
> libguestfs: command: run: \ -f ext2
> libguestfs: command: run: \ --host-cpu powerpc64le
> libguestfs: command: run: \ /home/libguestfs-1.40.2/appliance/supermin.d
> libguestfs: command: run: \ -o
> /home/libguestfs-1.40.2/tmp/.guestfs-0/appliance.d
> supermin: version: 5.1.19
> supermin: rpm: detected RPM version 4.14
> supermin: package handler: fedora/rpm
> supermin: acquiring lock on /home/libguestfs-1.40.2/tmp/.guestfs-0/lock
> supermin: build: /home/libguestfs-1.40.2/appliance/supermin.d
> supermin: reading the supermin appliance
> supermin: build: visiting
> /home/libguestfs-1.40.2/appliance/supermin.d/base.tar.gz type gzip base
> image (tar)
> supermin: build: visiting
> /home/libguestfs-1.40.2/appliance/supermin.d/daemon.tar.gz type gzip base
> image (tar)
> supermin: build: visiting
> /home/libguestfs-1.40.2/appliance/supermin.d/excludefiles type uncompressed
> excludefiles
> supermin: build: visiting
> /home/libguestfs-1.40.2/appliance/supermin.d/hostfiles type uncompressed
> hostfiles
> supermin: build: visiting
> /home/libguestfs-1.40.2/appliance/supermin.d/init.tar.gz type gzip base
> image (tar)
> supermin: build: visiting
> /home/libguestfs-1.40.2/appliance/supermin.d/packages type uncompressed
> packages
> supermin: build: visiting
> /home/libguestfs-1.40.2/appliance/supermin.d/udev-rules.tar.gz type gzip
> base image (tar)
> supermin: mapping package names to installed packages
> supermin: resolving full list of package dependencies
> supermin: build: 178 packages, including dependencies
> supermin: build: 32589 files
> supermin: build: 7104 files, after matching excludefiles
> supermin: build: 7105 files, after adding hostfiles
> supermin: build: 7091 files, after removing unreadable files
> supermin: build: 7123 files, after munging
> supermin: kernel: looking for kernel using environment variables ...
> supermin: kernel: looking for kernels in /lib/modules/*/vmlinuz ...
> supermin: kernel: picked vmlinuz /lib/modules/4.18.0-134.el8.ppc64le/vmlinuz
> supermin: kernel: kernel_version 4.18.0-134.el8.ppc64le
> supermin: kernel: modpath /lib/modules/4.18.0-134.el8.ppc64le
> supermin: ext2: creating empty ext2 filesystem
> '/home/libguestfs-1.40.2/tmp/.guestfs-0/appliance.d.jh71vgfr/root'
> supermin: ext2: populating from base image
> supermin: ext2: copying files from host filesystem
> supermin: ext2: copying kernel modules
> supermin: ext2: creating minimal initrd
> '/home/libguestfs-1.40.2/tmp/.guestfs-0/appliance.d.jh71vgfr/initrd'
> supermin: ext2: wrote 22 modules to minimal initrd
> supermin: renaming
> /home/libguestfs-1.40.2/tmp/.guestfs-0/appliance.d.jh71vgfr to
> /home/libguestfs-1.40.2/tmp/.guestfs-0/appliance.d
> libguestfs: finished building supermin appliance
> libguestfs: begin testing qemu features
> libguestfs: checking for previously cached test results of
> /usr/libexec/qemu-kvm, in /home/libguestfs-1.40.2/tmp/.guestfs-0
> libguestfs: loading previously cached test results
> libguestfs: qemu version: 4.0
> libguestfs: qemu mandatory locking: yes
> libguestfs: qemu KVM: enabled
> libguestfs: finished testing qemu features
> libguestfs: command: run: dmesg | grep -Eoh 'lpj=[[:digit:]]+'
> libguestfs: read_lpj_from_dmesg: external command exited with error status 1
> libguestfs: read_lpj_from_files: no boot messages files are readable
> /usr/libexec/qemu-kvm \
>     -global virtio-blk-pci.scsi=off \
>     -no-user-config \
>     -enable-fips \
>     -nodefaults \
>     -display none \
>     -machine pseries,accel=tcg \
>     -m 1024 \
>     -no-reboot \
>     -rtc driftfix=slew \
>     -kernel /home/libguestfs-1.40.2/tmp/.guestfs-0/appliance.d/kernel \
>     -initrd /home/libguestfs-1.40.2/tmp/.guestfs-0/appliance.d/initrd \
>     -object rng-random,filename=/dev/urandom,id=rng0 \
>     -device virtio-rng-pci,rng=rng0 \
>     -device virtio-scsi-pci,id=scsi \
>     -drive
> file=/home/libguestfs-1.40.2/tmp/libguestfs5mOzvn/scratch1.img,cache=unsafe,
> format=raw,id=hd0,if=none \
>     -device scsi-hd,drive=hd0 \
>     -drive
> file=/home/libguestfs-1.40.2/tmp/.guestfs-0/appliance.d/root,snapshot=on,
> id=appliance,cache=unsafe,if=none,format=raw \
>     -device scsi-hd,drive=appliance \
>     -device virtio-serial-pci \
>     -serial stdio \
>     -chardev socket,path=/tmp/libguestfsM8yjvT/guestfsd.sock,id=channel0 \
>     -device virtserialport,chardev=channel0,name=org.libguestfs.channel.0 \
>     -append "panic=1 console=hvc0 console=ttyS0 edd=off udevtimeout=6000
> udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory
> usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=/dev/sdb
> selinux=0 guestfs_verbose=1 TERM=xterm-256color"
> qemu-kvm: warning: TCG doesn't support requested feature, cap-cfpc=workaround
> qemu-kvm: warning: TCG doesn't support requested feature, cap-sbbc=workaround
> qemu-kvm: warning: TCG doesn't support requested feature, cap-ibs=workaround
> qemu-kvm: warning: global mc146818rtc.lost_tick_policy has invalid class name
> 
> 
> SLOF\x1b[0m\x1b[?25l
> **********************************************************************
> \x1b[1mQEMU Starting
> \x1b[0m Build Date = Jul 23 2019 04:40:55
>  FW Version = mockbuild@ release 20190703
>  Press "s" to enter Open Firmware.
> 
> Populating /vdevice methods
> Populating /vdevice/vty@71000000
> Populating /vdevice/nvram@71000001
> Populating /pci@800000020000000
>                      00 0000 (D) : 1af4 1005    unknown-legacy-device*
>                      00 0800 (D) : 1af4 1004    virtio [ scsi ]
> Populating /pci@800000020000000/scsi@1
>        SCSI: Looking for devices
>           100000000000000 DISK     : "QEMU     QEMU HARDDISK    2.5+"
>           101000000000000 DISK     : "QEMU     QEMU HARDDISK    2.5+"
>                      00 1000 (D) : 1af4 1003    virtio [ serial ]
> No NVRAM common partition, re-initializing...
> Scanning USB 
> Using default console: /vdevice/vty@71000000
> Detected RAM kernel at 400000 (1a7b860 bytes) 
>      
>   Welcome to Open Firmware
> 
>   Copyright (c) 2004, 2017 IBM Corporation All rights reserved.
>   This program and the accompanying materials are made available
>   under the terms of the BSD License available at
>   http://www.opensource.org/licenses/bsd-license.php
> 
> Booting from memory...
> OF stdout device is: /vdevice/vty@71000000
> Preparing to boot Linux version 4.18.0-134.el8.ppc64le
> (mockbuild@ppc-061.build.eng.bos.redhat.com) (gcc version 8.3.1 20190507
> (Red Hat 8.3.1-4) (GCC)) #1 SMP Thu Aug 15 17:39:09 UTC 2019
> Detected machine type: 0000000000000101
> command line: panic=1 console=hvc0 console=ttyS0 edd=off udevtimeout=6000
> udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory
> usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=/dev/sdb
> selinux=0 guestfs_verbose=1 TERM=xterm-256color
> Max number of cores passed to firmware: 2048 (NR_CPUS = 2048)
> Calling ibm,client-architecture-support...libguestfs: error: appliance
> closed the connection unexpectedly, see earlier error messages
> libguestfs: child_cleanup: 0x3e420d30: child process died
> libguestfs: sending SIGTERM to process 139946
> libguestfs: qemu maxrss 153472K
> libguestfs: error: guestfs_launch failed, see earlier error messages
> libguestfs: closing guestfs handle 0x3e420d30 (state 0)
> libguestfs: command: run: rm
> libguestfs: command: run: \ -rf /home/libguestfs-1.40.2/tmp/libguestfs5mOzvn
> libguestfs: command: run: rm
> libguestfs: command: run: \ -rf /tmp/libguestfsM8yjvT
> make: *** [Makefile:2946: quickcheck] Error 1

It's another bug: it fails because of the "-no-reboot" parameter.

Since bd94bc06479a "spapr: change default interrupt mode to 'dual'", QEMU resets the machine to select the appropriate interrupt controller. And -no-reboot prevents that.

Comment 21 Min Deng 2019-08-20 04:47:14 UTC
(In reply to Laurent Vivier from comment #20)
> (In reply to Min Deng from comment #19)
> > QE tried the issue on the build
> > qemu-kvm-4.1.0-1.module+el8.1.0+3966+4a23dca1.ppc64le
> > ,found out there's still problem as the following so could you please have a
> > look on them ? 
> > 
> > The same issue will not be reproduced on build 
> > Build information,
> > qemu-kvm-core-4.0.0-6.module+el8.1.0+3736+a2aefea3.ppc64le
> > The test could finish correctly.
> > make quickcheck LIBGUESTFS_BACKEND_SETTINGS=force_tcg     
> > LIBGUESTFS_HV=/usr/libexec/qemu-kvm
> > ===== TEST FINISHED OK =====
> > 
> > 
> > On the build qemu-kvm-4.1.0-1.module+el8.1.0+3966+4a23dca1.ppc64le,the
> > results were as following,please help to have a check in time,thank you.
> > (workspace) [root@ibm-p9wr-17 libguestfs-1.40.2]# make quickcheck
> > LIBGUESTFS_BACKEND_SETTINGS=force_tcg     
> > LIBGUESTFS_HV=/usr/libexec/qemu-kvm
> > fatal: not a git repository (or any parent up to mount point /)
> > Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
> > ./run test-tool/libguestfs-test-tool 
> >      ************************************************************
> >      *                    IMPORTANT NOTICE
> >      *
> >      * When reporting bugs, include the COMPLETE, UNEDITED
> >      * output below in your bug report.
> >      *
> >      ************************************************************
> > LD_LIBRARY_PATH=/home/libguestfs-1.40.2/ruby/ext/guestfs:/home/libguestfs-1.
> > 40.2/lib/.libs:/home/libguestfs-1.40.2/java/.libs:/home/libguestfs-1.40.2/
> > gobject/.libs
> > LIBGUESTFS_BACKEND_SETTINGS=force_tcg
> > LIBGUESTFS_CACHEDIR=/home/libguestfs-1.40.2/tmp
> > LIBGUESTFS_HV=/usr/libexec/qemu-kvm
> > LIBGUESTFS_TMPDIR=/home/libguestfs-1.40.2/tmp
> > LIBGUESTFS_PATH=/home/libguestfs-1.40.2/appliance
> > PATH=/home/libguestfs-1.40.2/v2v:/home/libguestfs-1.40.2/tools:/home/
> > libguestfs-1.40.2/test-tool:/home/libguestfs-1.40.2/sysprep:/home/libguestfs-
> > 1.40.2/sparsify:/home/libguestfs-1.40.2/resize:/home/libguestfs-1.40.2/
> > rescue:/home/libguestfs-1.40.2/p2v:/home/libguestfs-1.40.2/make-fs:/home/
> > libguestfs-1.40.2/inspector:/home/libguestfs-1.40.2/get-kernel:/home/
> > libguestfs-1.40.2/fuse:/home/libguestfs-1.40.2/format:/home/libguestfs-1.40.
> > 2/fish:/home/libguestfs-1.40.2/erlang:/home/libguestfs-1.40.2/edit:/home/
> > libguestfs-1.40.2/diff:/home/libguestfs-1.40.2/dib:/home/libguestfs-1.40.2/
> > df:/home/libguestfs-1.40.2/customize:/home/libguestfs-1.40.2/cat:/home/
> > libguestfs-1.40.2/builder:/home/libguestfs-1.40.2/align:/home/kar/workspace/
> > bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin
> > XDG_RUNTIME_DIR=/run/user/0
> > SELinux: Enforcing
> > guestfs_get_append: (null)
> > guestfs_get_autosync: 1
> > guestfs_get_backend: direct
> > guestfs_get_backend_settings: [force_tcg]
> > guestfs_get_cachedir: /home/libguestfs-1.40.2/tmp
> > guestfs_get_hv: /usr/libexec/qemu-kvm
> > guestfs_get_memsize: 1024
> > guestfs_get_network: 0
> > guestfs_get_path: /home/libguestfs-1.40.2/appliance
> > guestfs_get_pgroup: 0
> > guestfs_get_program: libguestfs-test-tool
> > guestfs_get_recovery_proc: 1
> > guestfs_get_smp: 1
> > guestfs_get_sockdir: /tmp
> > guestfs_get_tmpdir: /home/libguestfs-1.40.2/tmp
> > guestfs_get_trace: 0
> > guestfs_get_verbose: 1
> > host_cpu: powerpc64le
> > Launching appliance, timeout set to 600 seconds.
> > libguestfs: launch: program=libguestfs-test-tool
> > libguestfs: launch: version=1.40.2
> > libguestfs: launch: backend registered: unix
> > libguestfs: launch: backend registered: uml
> > libguestfs: launch: backend registered: direct
> > libguestfs: launch: backend=direct
> > libguestfs: launch: tmpdir=/home/libguestfs-1.40.2/tmp/libguestfs5mOzvn
> > libguestfs: launch: umask=0022
> > libguestfs: launch: euid=0
> > libguestfs: begin building supermin appliance
> > libguestfs: run supermin
> > libguestfs: command: run: /usr/bin/supermin
> > libguestfs: command: run: \ --build
> > libguestfs: command: run: \ --verbose
> > libguestfs: command: run: \ --if-newer
> > libguestfs: command: run: \ --lock
> > /home/libguestfs-1.40.2/tmp/.guestfs-0/lock
> > libguestfs: command: run: \ --copy-kernel
> > libguestfs: command: run: \ -f ext2
> > libguestfs: command: run: \ --host-cpu powerpc64le
> > libguestfs: command: run: \ /home/libguestfs-1.40.2/appliance/supermin.d
> > libguestfs: command: run: \ -o
> > /home/libguestfs-1.40.2/tmp/.guestfs-0/appliance.d
> > supermin: version: 5.1.19
> > supermin: rpm: detected RPM version 4.14
> > supermin: package handler: fedora/rpm
> > supermin: acquiring lock on /home/libguestfs-1.40.2/tmp/.guestfs-0/lock
> > supermin: build: /home/libguestfs-1.40.2/appliance/supermin.d
> > supermin: reading the supermin appliance
> > supermin: build: visiting
> > /home/libguestfs-1.40.2/appliance/supermin.d/base.tar.gz type gzip base
> > image (tar)
> > supermin: build: visiting
> > /home/libguestfs-1.40.2/appliance/supermin.d/daemon.tar.gz type gzip base
> > image (tar)
> > supermin: build: visiting
> > /home/libguestfs-1.40.2/appliance/supermin.d/excludefiles type uncompressed
> > excludefiles
> > supermin: build: visiting
> > /home/libguestfs-1.40.2/appliance/supermin.d/hostfiles type uncompressed
> > hostfiles
> > supermin: build: visiting
> > /home/libguestfs-1.40.2/appliance/supermin.d/init.tar.gz type gzip base
> > image (tar)
> > supermin: build: visiting
> > /home/libguestfs-1.40.2/appliance/supermin.d/packages type uncompressed
> > packages
> > supermin: build: visiting
> > /home/libguestfs-1.40.2/appliance/supermin.d/udev-rules.tar.gz type gzip
> > base image (tar)
> > supermin: mapping package names to installed packages
> > supermin: resolving full list of package dependencies
> > supermin: build: 178 packages, including dependencies
> > supermin: build: 32589 files
> > supermin: build: 7104 files, after matching excludefiles
> > supermin: build: 7105 files, after adding hostfiles
> > supermin: build: 7091 files, after removing unreadable files
> > supermin: build: 7123 files, after munging
> > supermin: kernel: looking for kernel using environment variables ...
> > supermin: kernel: looking for kernels in /lib/modules/*/vmlinuz ...
> > supermin: kernel: picked vmlinuz /lib/modules/4.18.0-134.el8.ppc64le/vmlinuz
> > supermin: kernel: kernel_version 4.18.0-134.el8.ppc64le
> > supermin: kernel: modpath /lib/modules/4.18.0-134.el8.ppc64le
> > supermin: ext2: creating empty ext2 filesystem
> > '/home/libguestfs-1.40.2/tmp/.guestfs-0/appliance.d.jh71vgfr/root'
> > supermin: ext2: populating from base image
> > supermin: ext2: copying files from host filesystem
> > supermin: ext2: copying kernel modules
> > supermin: ext2: creating minimal initrd
> > '/home/libguestfs-1.40.2/tmp/.guestfs-0/appliance.d.jh71vgfr/initrd'
> > supermin: ext2: wrote 22 modules to minimal initrd
> > supermin: renaming
> > /home/libguestfs-1.40.2/tmp/.guestfs-0/appliance.d.jh71vgfr to
> > /home/libguestfs-1.40.2/tmp/.guestfs-0/appliance.d
> > libguestfs: finished building supermin appliance
> > libguestfs: begin testing qemu features
> > libguestfs: checking for previously cached test results of
> > /usr/libexec/qemu-kvm, in /home/libguestfs-1.40.2/tmp/.guestfs-0
> > libguestfs: loading previously cached test results
> > libguestfs: qemu version: 4.0
> > libguestfs: qemu mandatory locking: yes
> > libguestfs: qemu KVM: enabled
> > libguestfs: finished testing qemu features
> > libguestfs: command: run: dmesg | grep -Eoh 'lpj=[[:digit:]]+'
> > libguestfs: read_lpj_from_dmesg: external command exited with error status 1
> > libguestfs: read_lpj_from_files: no boot messages files are readable
> > /usr/libexec/qemu-kvm \
> >     -global virtio-blk-pci.scsi=off \
> >     -no-user-config \
> >     -enable-fips \
> >     -nodefaults \
> >     -display none \
> >     -machine pseries,accel=tcg \
> >     -m 1024 \
> >     -no-reboot \
> >     -rtc driftfix=slew \
> >     -kernel /home/libguestfs-1.40.2/tmp/.guestfs-0/appliance.d/kernel \
> >     -initrd /home/libguestfs-1.40.2/tmp/.guestfs-0/appliance.d/initrd \
> >     -object rng-random,filename=/dev/urandom,id=rng0 \
> >     -device virtio-rng-pci,rng=rng0 \
> >     -device virtio-scsi-pci,id=scsi \
> >     -drive
> > file=/home/libguestfs-1.40.2/tmp/libguestfs5mOzvn/scratch1.img,cache=unsafe,
> > format=raw,id=hd0,if=none \
> >     -device scsi-hd,drive=hd0 \
> >     -drive
> > file=/home/libguestfs-1.40.2/tmp/.guestfs-0/appliance.d/root,snapshot=on,
> > id=appliance,cache=unsafe,if=none,format=raw \
> >     -device scsi-hd,drive=appliance \
> >     -device virtio-serial-pci \
> >     -serial stdio \
> >     -chardev socket,path=/tmp/libguestfsM8yjvT/guestfsd.sock,id=channel0 \
> >     -device virtserialport,chardev=channel0,name=org.libguestfs.channel.0 \
> >     -append "panic=1 console=hvc0 console=ttyS0 edd=off udevtimeout=6000
> > udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory
> > usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=/dev/sdb
> > selinux=0 guestfs_verbose=1 TERM=xterm-256color"
> > qemu-kvm: warning: TCG doesn't support requested feature, cap-cfpc=workaround
> > qemu-kvm: warning: TCG doesn't support requested feature, cap-sbbc=workaround
> > qemu-kvm: warning: TCG doesn't support requested feature, cap-ibs=workaround
> > qemu-kvm: warning: global mc146818rtc.lost_tick_policy has invalid class name
> > 
> > 
> > SLOF\x1b[0m\x1b[?25l
> > **********************************************************************
> > \x1b[1mQEMU Starting
> > \x1b[0m Build Date = Jul 23 2019 04:40:55
> >  FW Version = mockbuild@ release 20190703
> >  Press "s" to enter Open Firmware.
> > 
> > Populating /vdevice methods
> > Populating /vdevice/vty@71000000
> > Populating /vdevice/nvram@71000001
> > Populating /pci@800000020000000
> >                      00 0000 (D) : 1af4 1005    unknown-legacy-device*
> >                      00 0800 (D) : 1af4 1004    virtio [ scsi ]
> > Populating /pci@800000020000000/scsi@1
> >        SCSI: Looking for devices
> >           100000000000000 DISK     : "QEMU     QEMU HARDDISK    2.5+"
> >           101000000000000 DISK     : "QEMU     QEMU HARDDISK    2.5+"
> >                      00 1000 (D) : 1af4 1003    virtio [ serial ]
> > No NVRAM common partition, re-initializing...
> > Scanning USB 
> > Using default console: /vdevice/vty@71000000
> > Detected RAM kernel at 400000 (1a7b860 bytes) 
> >      
> >   Welcome to Open Firmware
> > 
> >   Copyright (c) 2004, 2017 IBM Corporation All rights reserved.
> >   This program and the accompanying materials are made available
> >   under the terms of the BSD License available at
> >   http://www.opensource.org/licenses/bsd-license.php
> > 
> > Booting from memory...
> > OF stdout device is: /vdevice/vty@71000000
> > Preparing to boot Linux version 4.18.0-134.el8.ppc64le
> > (mockbuild@ppc-061.build.eng.bos.redhat.com) (gcc version 8.3.1 20190507
> > (Red Hat 8.3.1-4) (GCC)) #1 SMP Thu Aug 15 17:39:09 UTC 2019
> > Detected machine type: 0000000000000101
> > command line: panic=1 console=hvc0 console=ttyS0 edd=off udevtimeout=6000
> > udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory
> > usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=/dev/sdb
> > selinux=0 guestfs_verbose=1 TERM=xterm-256color
> > Max number of cores passed to firmware: 2048 (NR_CPUS = 2048)
> > Calling ibm,client-architecture-support...libguestfs: error: appliance
> > closed the connection unexpectedly, see earlier error messages
> > libguestfs: child_cleanup: 0x3e420d30: child process died
> > libguestfs: sending SIGTERM to process 139946
> > libguestfs: qemu maxrss 153472K
> > libguestfs: error: guestfs_launch failed, see earlier error messages
> > libguestfs: closing guestfs handle 0x3e420d30 (state 0)
> > libguestfs: command: run: rm
> > libguestfs: command: run: \ -rf /home/libguestfs-1.40.2/tmp/libguestfs5mOzvn
> > libguestfs: command: run: rm
> > libguestfs: command: run: \ -rf /tmp/libguestfsM8yjvT
> > make: *** [Makefile:2946: quickcheck] Error 1
> 
> It's another bug: it fails because of the "-no-reboot" parameter.
> 
> Since bd94bc06479a "spapr: change default interrupt mode to 'dual'", QEMU
> resets the machine to select the appropriate interrupt controller. And
> -no-reboot prevents that.

Hello Laurent,
  QE will file a bug for new issue,by the way,is it proper for QE to verify this bug since new issue affected this verification ? Could you please give some ideas for it ? Thanks in advance.

Best regards,
Min

Comment 22 Min Deng 2019-08-20 05:23:59 UTC
New issues,
Bug 1743477 - Since bd94bc06479a "spapr: change default interrupt mode to 'dual'", QEMU resets the machine to select the appropriate interrupt controller. And -no-reboot prevents that.

Comment 23 Laurent Vivier 2019-08-20 08:25:13 UTC
The way to workaround the problem is to not use the the "dual" mode for the interrupt controller.

You can either use something like "-M accel=tcg,ic-mode=xics" or "-M accel=tcg,ic-mode=xive".

But as it doesn't seem we can provide these parameters to the test environment of libguestfs you must reproduce the problem directly and remove the -no-reboot parameter:

curl -O http://download-ipv4.eng.brq.redhat.com/rhel-8/rel-eng/RHEL-8/latest-RHEL-8.1.0/compose/BaseOS/ppc64le/os/ppc/ppc64/vmlinuz
/usr/libexec/qemu-kvm -M accel=tcg -nodefaults -serial mon:stdio -kernel vmlinuz -nographic 


qemu-system-ppc64: warning: TCG doesn't support requested feature, cap-cfpc=workaround
qemu-system-ppc64: warning: TCG doesn't support requested feature, cap-sbbc=workaround
qemu-system-ppc64: warning: TCG doesn't support requested feature, cap-ibs=workaround


SLOF **********************************************************************
...
[    0.081922] Oops: Exception in kernel mode, sig: 4 [#1]
[    0.082430] LE SMP NR_CPUS=2048 NUMA pSeries
[    0.082796] Modules linked in:
[    0.083239] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.18.0-128.el8.ppc64le #1
[    0.083407] NIP:  c00000000000b8a0 LR: c00000000000dc68 CTR: 0000000000007fff
[    0.083525] REGS: c00000000165f7f0 TRAP: 0700   Not tainted  (4.18.0-128.el8.ppc64le)
[    0.083587] MSR:  8000000002089033 <SF,VEC,EE,ME,IR,DR,RI,LE>  CR: 24828822  XER: 00000000
[    0.083868] CFAR: c00000000000b884 IRQMASK: 1 
[    0.083868] GPR00: c00000000001fa00 c00000000165fa70 c000000001662a00 c0000000015e3860 
[    0.083868] GPR04: c00000001e4a23e0 c00000001ff39478 0000000000000000 0000000004b1b252 
[    0.083868] GPR08: 0000000000000000 0000000000007fff 0000000000000000 0000000000000000 
[    0.083868] GPR12: 0000000000008000 c000000001a80000 0000000001356a60 0000000001356a58 
[    0.083868] GPR16: 0000000001421ec8 0000000001356388 000000001dc5fee0 00000000013566a8 
[    0.083868] GPR20: c0000000015e2c00 c0000000015e33d0 000000001ed80000 0000000024828822 
[    0.083868] GPR24: c0000000015e3860 c00000001e4a23e0 000000001ed80000 c000000001195998 
[    0.083868] GPR28: c0000000015e3860 c0000000015e2c00 c00000001e4a1780 c0000000015e2c00 
[    0.085244] NIP [c00000000000b8a0] flush_count_cache+0x120/0x2480
[    0.085317] LR [c00000000000dc68] _switch+0x68/0x180
[    0.085432] Call Trace:
[    0.085623] [c00000000165fa70] [c00000000165fab0] init_stack+0x3ab0/0x4000 (unreliable)
[    0.085797] [c00000000165fc40] [c00000000001fa00] __switch_to+0x2e0/0x4e0
[    0.085875] [c00000000165fca0] [c000000000d42504] __schedule+0x2c4/0xb20
[    0.085939] [c00000000165fd70] [c000000000d43218] _cond_resched+0x78/0xb0
[    0.086067] [c00000000165fda0] [c000000000140690] copy_process.isra.4.part.5+0xcd0/0x16b0
[    0.086137] [c00000000165fe40] [c000000000141144] _do_fork.constprop.7+0xd4/0x3a0
[    0.086201] [c00000000165feb0] [c000000000141948] kernel_thread+0x38/0x50
[    0.086260] [c00000000165fed0] [c000000000010390] rest_init+0x98/0xf8
[    0.086324] [c00000000165ff00] [c0000000010a4228] start_kernel+0x638/0x658
[    0.086388] [c00000000165ff90] [c00000000000ad7c] start_here_common+0x1c/0x520
[    0.086501] Instruction dump:
[    0.087095] 48000005 48000005 48000005 48000005 48000005 4800001c 60000000 60000000 
[    0.087262] 60000000 60000000 60000000 60000000 <7d2803a6> 39207fff 7d2903a6 4c400420 
[    0.088528] ---[ end trace 2365a3bd11288e2b ]---
[    0.088655] 
[    1.089532] Kernel panic - not syncing: Fatal exception

Comment 24 Richard W.M. Jones 2019-08-20 08:49:10 UTC
> You can either use something like "-M accel=tcg,ic-mode=xics" or "-M accel=tcg,ic-mode=xive".

It'd be nice if the qemu command would just DTRT rather than requiring odd, ever-
changing per-architecture hacks.

Comment 25 Min Deng 2019-08-20 09:46:18 UTC
As there's a new issue Bug 1743477,which will affect bug's verification if QE insist on the original steps.QE made use of the steps from comment23.Thanks for Laurent's big help.
The problem can be reproduced on qemu-kvm-15:3.1.0-26.module+el8.0.1+3210+5c219ce1.ppc64le by the following steps

The problem can be **verified** on qemu-kvm-4.1.0-4.module+el8.1.0+4020+16089f93.ppc64le by the following steps

curl -O http://download-ipv4.eng.brq.redhat.com/rhel-8/rel-eng/RHEL-8/latest-RHEL-8.1.0/compose/BaseOS/ppc64le/os/ppc/ppc64/vmlinuz
curl -O http://download-ipv4.eng.brq.redhat.com/rhel-8/rel-eng/RHEL-8/latest-RHEL-8.1.0/compose/BaseOS/ppc64le/os/ppc/ppc64/initrd.img
/usr/libexec/qemu-kvm -M accel=tcg -nodefaults -serial mon:stdio -kernel vmlinuz -nographic -m 2G -initrd initrd.img

There's no kernel panic problem any more,so the original issue has been fixed and move this bug to be verified.Thanks.

Comment 26 David Gibson 2019-08-21 06:45:50 UTC
> > You can either use something like "-M accel=tcg,ic-mode=xics" or "-M accel=tcg,ic-mode=xive".

> It'd be nice if the qemu command would just DTRT rather than requiring odd, ever-
> changing per-architecture hacks.

Yes, this situation sucks.  Unfortunately, it's going to take me a little while to figure out how to fix it, and there are a bunch of other high priority bugs I have to look at as well.

We're tracking this in bug 1743477.

Comment 28 errata-xmlrpc 2019-11-06 07:17:18 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:3723


Note You need to log in before you can comment on or make changes to this bug.