Bug 1403745

Summary: libvirt picks qemu-system-arm to run ppc64le VM on ppc64le host
Product: [Fedora] Fedora Reporter: Richard W.M. Jones <rjones>
Component: libvirtAssignee: Libvirt Maintainers <libvirt-maint>
Status: CLOSED CURRENTRELEASE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 26CC: agedosier, berrange, clalancette, crobinso, itamar, jdenemar, laine, libvirt-maint, veillard, virt-maint
Target Milestone: ---   
Target Release: ---   
Hardware: ppc64le   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1403785 (view as bug list) Environment:
Last Closed: 2017-05-04 17:47:59 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 910269    
Attachments:
Description Flags
root.log
none
build.log none

Description Richard W.M. Jones 2016-12-12 10:19:07 UTC 
Description of problem:

Running libguestfs-test-tool on ppc64le host gives the error:

Original error from libvirt: internal error: process exited while connecting to monitor: 2016-12-12T10:06:56.792870Z qemu-system-arm: -machine pseries,accel=tcg,usb=off,dump-guest-core=off: unsupported machine type
Use -machine help to list supported machines [code=1 int1=-1]

The problem is that libvirt is using qemu-system-arm to
run the VM, even though we never specified ARM anywhere
and obviously it makes no sense.

Version-Release number of selected component (if applicable):

libvirt-2.5.0-2.fc26.ppc64le
qemu-2:2.8.0-0.2.rc2.fc26.ppc64le

How reproducible:

Unknown.

Steps to Reproduce:
1. Run libguestfs-test-tool on ppc64le host.

Additional info:

build.log & root.log attached.
Comment 1 Richard W.M. Jones 2016-12-12 10:20:42 UTC 
Created attachment 1230740 [details]
root.log
Comment 2 Richard W.M. Jones 2016-12-12 10:22:23 UTC 
Created attachment 1230741 [details]
build.log
Comment 3 Daniel Berrangé 2016-12-12 11:16:22 UTC 
Capabilities from build.log:

<capabilities>
  <host>
    <uuid>98d4d4ab-0bc3-4f8d-b7f6-18394a59acd0</uuid>
    <cpu>
      <arch>ppc64le</arch>
      <model>POWER8</model>
      <vendor>IBM</vendor>
      <topology sockets='1' cores='4' threads='1'/>
      <pages unit='KiB' size='64'/>
    </cpu>
    <power_management/>
    <migration_features>
      <live/>
      <uri_transports>
        <uri_transport>tcp</uri_transport>
        <uri_transport>rdma</uri_transport>
      </uri_transports>
    </migration_features>
    <topology>
      <cells num='1'>
        <cell id='0'>
          <memory unit='KiB'>10452928</memory>
          <pages unit='KiB' size='64'>163327</pages>
          <distances>
            <sibling id='0' value='10'/>
          </distances>
          <cpus num='4'>
            <cpu id='0' socket_id='0' core_id='0' siblings='0'/>
            <cpu id='1' socket_id='0' core_id='8' siblings='1'/>
            <cpu id='2' socket_id='0' core_id='16' siblings='2'/>
            <cpu id='3' socket_id='0' core_id='24' siblings='3'/>
          </cpus>
        </cell>
      </cells>
    </topology>
    <secmodel>
      <model>none</model>
      <doi>0</doi>
    </secmodel>
  </host>
  <guest>
    <os_type>hvm</os_type>
    <arch name='armv7l'>
      <wordsize>32</wordsize>
      <emulator>/usr/bin/qemu-system-arm</emulator>
      <machine maxCpus='1'>integratorcp</machine>
      <machine maxCpus='2'>nuri</machine>
      <machine maxCpus='1'>verdex</machine>
      <machine maxCpus='1'>ast2500-evb</machine>
      <machine maxCpus='2'>smdkc210</machine>
      <machine maxCpus='1'>collie</machine>
      <machine maxCpus='1'>imx25-pdk</machine>
      <machine maxCpus='1'>spitz</machine>
      <machine maxCpus='4'>realview-pbx-a9</machine>
      <machine maxCpus='1'>realview-eb</machine>
      <machine maxCpus='1'>versatilepb</machine>
      <machine maxCpus='1'>realview-pb-a8</machine>
      <machine maxCpus='1'>musicpal</machine>
      <machine maxCpus='1'>z2</machine>
      <machine maxCpus='1'>akita</machine>
      <machine maxCpus='255'>virt-2.7</machine>
      <machine maxCpus='1'>kzm</machine>
      <machine maxCpus='255'>virt-2.8</machine>
      <machine canonical='virt-2.8' maxCpus='255'>virt</machine>
      <machine maxCpus='4'>realview-eb-mpcore</machine>
      <machine maxCpus='1'>sx1</machine>
      <machine maxCpus='1'>sx1-v1</machine>
      <machine maxCpus='255'>virt-2.6</machine>
      <machine maxCpus='1'>cubieboard</machine>
      <machine maxCpus='4'>highbank</machine>
      <machine maxCpus='4'>raspi2</machine>
      <machine maxCpus='1'>netduino2</machine>
      <machine maxCpus='1'>terrier</machine>
      <machine maxCpus='1'>n810</machine>
      <machine maxCpus='1'>mainstone</machine>
      <machine maxCpus='1'>palmetto-bmc</machine>
      <machine maxCpus='4'>sabrelite</machine>
      <machine maxCpus='4'>midway</machine>
      <machine maxCpus='1'>cheetah</machine>
      <machine maxCpus='1'>tosa</machine>
      <machine maxCpus='1'>borzoi</machine>
      <machine maxCpus='1'>versatileab</machine>
      <machine maxCpus='1'>lm3s6965evb</machine>
      <machine maxCpus='1'>n800</machine>
      <machine maxCpus='1'>connex</machine>
      <machine maxCpus='1'>xilinx-zynq-a9</machine>
      <machine maxCpus='4'>vexpress-a9</machine>
      <machine maxCpus='4'>vexpress-a15</machine>
      <machine maxCpus='1'>canon-a1100</machine>
      <machine maxCpus='1'>lm3s811evb</machine>
      <domain type='qemu'/>
    </arch>
    <features>
      <cpuselection/>
      <deviceboot/>
      <disksnapshot default='on' toggle='no'/>
    </features>
  </guest>
  <guest>
    <os_type>hvm</os_type>
    <arch name='aarch64'>
      <wordsize>64</wordsize>
      <emulator>/usr/bin/qemu-system-aarch64</emulator>
      <machine maxCpus='1'>integratorcp</machine>
      <machine maxCpus='2'>nuri</machine>
      <machine maxCpus='1'>verdex</machine>
      <machine maxCpus='1'>ast2500-evb</machine>
      <machine maxCpus='2'>smdkc210</machine>
      <machine maxCpus='1'>collie</machine>
      <machine maxCpus='1'>imx25-pdk</machine>
      <machine maxCpus='1'>spitz</machine>
      <machine maxCpus='4'>realview-pbx-a9</machine>
      <machine maxCpus='1'>realview-eb</machine>
      <machine maxCpus='1'>versatilepb</machine>
      <machine maxCpus='1'>realview-pb-a8</machine>
      <machine maxCpus='1'>musicpal</machine>
      <machine maxCpus='1'>z2</machine>
      <machine maxCpus='1'>akita</machine>
      <machine maxCpus='255'>virt-2.7</machine>
      <machine maxCpus='1'>kzm</machine>
      <machine maxCpus='255'>virt-2.8</machine>
      <machine canonical='virt-2.8' maxCpus='255'>virt</machine>
      <machine maxCpus='4'>realview-eb-mpcore</machine>
      <machine maxCpus='1'>sx1</machine>
      <machine maxCpus='1'>sx1-v1</machine>
      <machine maxCpus='255'>virt-2.6</machine>
      <machine maxCpus='1'>cubieboard</machine>
      <machine maxCpus='4'>highbank</machine>
      <machine maxCpus='4'>raspi2</machine>
      <machine maxCpus='1'>netduino2</machine>
      <machine maxCpus='1'>terrier</machine>
      <machine maxCpus='1'>n810</machine>
      <machine maxCpus='1'>mainstone</machine>
      <machine maxCpus='1'>palmetto-bmc</machine>
      <machine maxCpus='4'>sabrelite</machine>
      <machine maxCpus='4'>midway</machine>
      <machine maxCpus='1'>cheetah</machine>
      <machine maxCpus='1'>tosa</machine>
      <machine maxCpus='1'>borzoi</machine>
      <machine maxCpus='1'>versatileab</machine>
      <machine maxCpus='1'>lm3s6965evb</machine>
      <machine maxCpus='1'>n800</machine>
      <machine maxCpus='1'>connex</machine>
      <machine maxCpus='1'>xilinx-zynq-a9</machine>
      <machine maxCpus='1'>xlnx-ep108</machine>
      <machine maxCpus='4'>vexpress-a9</machine>
      <machine maxCpus='4'>vexpress-a15</machine>
      <machine maxCpus='1'>xlnx-zcu102</machine>
      <machine maxCpus='1'>canon-a1100</machine>
      <machine maxCpus='1'>lm3s811evb</machine>
      <domain type='qemu'/>
    </arch>
    <features>
      <cpuselection/>
      <deviceboot/>
      <disksnapshot default='on' toggle='no'/>
    </features>
  </guest>
  <guest>
    <os_type>hvm</os_type>
    <arch name='i686'>
      <wordsize>32</wordsize>
      <emulator>/usr/bin/qemu-system-i386</emulator>
      <machine maxCpus='255'>pc-i440fx-2.8</machine>
      <machine canonical='pc-i440fx-2.8' maxCpus='255'>pc</machine>
      <machine maxCpus='255'>pc-0.12</machine>
      <machine maxCpus='255'>pc-i440fx-2.4</machine>
      <machine maxCpus='255'>pc-1.3</machine>
      <machine maxCpus='255'>pc-q35-2.7</machine>
      <machine maxCpus='255'>pc-q35-2.6</machine>
      <machine maxCpus='255'>pc-i440fx-1.7</machine>
      <machine maxCpus='255'>pc-i440fx-1.6</machine>
      <machine maxCpus='255'>pc-i440fx-2.7</machine>
      <machine maxCpus='255'>pc-0.11</machine>
      <machine maxCpus='255'>pc-i440fx-2.3</machine>
      <machine maxCpus='255'>pc-0.10</machine>
      <machine maxCpus='255'>pc-1.2</machine>
      <machine maxCpus='255'>pc-i440fx-2.2</machine>
      <machine maxCpus='1'>isapc</machine>
      <machine maxCpus='255'>pc-q35-2.5</machine>
      <machine maxCpus='255'>pc-0.15</machine>
      <machine maxCpus='255'>pc-i440fx-1.5</machine>
      <machine maxCpus='255'>pc-0.14</machine>
      <machine maxCpus='255'>pc-i440fx-2.6</machine>
      <machine maxCpus='255'>pc-i440fx-1.4</machine>
      <machine maxCpus='255'>pc-i440fx-2.5</machine>
      <machine maxCpus='255'>pc-1.1</machine>
      <machine maxCpus='255'>pc-i440fx-2.1</machine>
      <machine maxCpus='288'>pc-q35-2.8</machine>
      <machine canonical='pc-q35-2.8' maxCpus='288'>q35</machine>
      <machine maxCpus='255'>pc-1.0</machine>
      <machine maxCpus='255'>pc-i440fx-2.0</machine>
      <machine maxCpus='255'>pc-q35-2.4</machine>
      <machine maxCpus='255'>pc-0.13</machine>
      <domain type='qemu'/>
    </arch>
    <features>
      <cpuselection/>
      <deviceboot/>
      <disksnapshot default='on' toggle='no'/>
      <acpi default='on' toggle='yes'/>
      <apic default='on' toggle='no'/>
      <pae/>
      <nonpae/>
    </features>
  </guest>
  <guest>
    <os_type>hvm</os_type>
    <arch name='ppc'>
      <wordsize>32</wordsize>
      <emulator>/usr/bin/qemu-system-ppc</emulator>
      <machine maxCpus='1'>g3beige</machine>
      <machine maxCpus='1'>ref405ep</machine>
      <machine maxCpus='1'>virtex-ml507</machine>
      <machine maxCpus='32'>ppce500</machine>
      <machine maxCpus='15'>mpc8544ds</machine>
      <machine maxCpus='1'>bamboo</machine>
      <machine maxCpus='1'>prep</machine>
      <machine maxCpus='1'>mac99</machine>
      <machine maxCpus='1'>taihu</machine>
      <domain type='qemu'/>
    </arch>
    <features>
      <cpuselection/>
      <deviceboot/>
      <disksnapshot default='on' toggle='no'/>
    </features>
  </guest>
  <guest>
    <os_type>hvm</os_type>
    <arch name='ppc64'>
      <wordsize>64</wordsize>
      <emulator>/usr/bin/qemu-system-ppc64</emulator>
      <machine maxCpus='255'>pseries-2.8</machine>
      <machine canonical='pseries-2.8' maxCpus='255'>pseries</machine>
      <machine maxCpus='1'>ref405ep</machine>
      <machine maxCpus='1'>virtex-ml507</machine>
      <machine maxCpus='2048'>powernv</machine>
      <machine maxCpus='32'>ppce500</machine>
      <machine maxCpus='15'>mpc8544ds</machine>
      <machine maxCpus='1'>bamboo</machine>
      <machine maxCpus='1'>g3beige</machine>
      <machine maxCpus='1'>prep</machine>
      <machine maxCpus='1'>mac99</machine>
      <machine maxCpus='255'>pseries-2.6</machine>
      <machine maxCpus='255'>pseries-2.7</machine>
      <machine maxCpus='255'>pseries-2.4</machine>
      <machine maxCpus='255'>pseries-2.5</machine>
      <machine maxCpus='255'>pseries-2.2</machine>
      <machine maxCpus='1'>taihu</machine>
      <machine maxCpus='255'>pseries-2.3</machine>
      <machine maxCpus='255'>pseries-2.1</machine>
      <domain type='qemu'/>
    </arch>
    <features>
      <cpuselection/>
      <deviceboot/>
      <disksnapshot default='on' toggle='no'/>
    </features>
  </guest>
  <guest>
    <os_type>hvm</os_type>
    <arch name='ppcemb'>
      <wordsize>32</wordsize>
      <emulator>/usr/bin/qemu-system-ppcemb</emulator>
      <machine maxCpus='1'>ref405ep</machine>
      <machine maxCpus='1'>virtex-ml507</machine>
      <machine maxCpus='1'>bamboo</machine>
      <machine maxCpus='1'>taihu</machine>
      <domain type='qemu'/>
    </arch>
    <features>
      <cpuselection/>
      <deviceboot/>
      <disksnapshot default='on' toggle='no'/>
    </features>
  </guest>
  <guest>
    <os_type>hvm</os_type>
    <arch name='s390x'>
      <wordsize>64</wordsize>
      <emulator>/usr/bin/qemu-system-s390x</emulator>
      <machine maxCpus='248'>s390-ccw-virtio-2.8</machine>
      <machine canonical='s390-ccw-virtio-2.8' maxCpus='248'>s390-ccw-virtio</machine>
      <machine maxCpus='248'>s390-ccw-virtio-2.7</machine>
      <machine maxCpus='248'>s390-ccw-virtio-2.6</machine>
      <machine maxCpus='248'>s390-ccw-virtio-2.5</machine>
      <machine maxCpus='248'>s390-ccw-virtio-2.4</machine>
      <domain type='qemu'/>
    </arch>
    <features>
      <cpuselection/>
      <deviceboot/>
      <disksnapshot default='on' toggle='no'/>
    </features>
  </guest>
  <guest>
    <os_type>hvm</os_type>
    <arch name='x86_64'>
      <wordsize>64</wordsize>
      <emulator>/usr/bin/qemu-system-x86_64</emulator>
      <machine maxCpus='255'>pc-i440fx-2.8</machine>
      <machine canonical='pc-i440fx-2.8' maxCpus='255'>pc</machine>
      <machine maxCpus='255'>pc-0.12</machine>
      <machine maxCpus='255'>pc-i440fx-2.4</machine>
      <machine maxCpus='255'>pc-1.3</machine>
      <machine maxCpus='255'>pc-q35-2.7</machine>
      <machine maxCpus='255'>pc-q35-2.6</machine>
      <machine maxCpus='255'>pc-i440fx-1.7</machine>
      <machine maxCpus='255'>pc-i440fx-1.6</machine>
      <machine maxCpus='255'>pc-i440fx-2.7</machine>
      <machine maxCpus='255'>pc-0.11</machine>
      <machine maxCpus='255'>pc-i440fx-2.3</machine>
      <machine maxCpus='255'>pc-0.10</machine>
      <machine maxCpus='255'>pc-1.2</machine>
      <machine maxCpus='255'>pc-i440fx-2.2</machine>
      <machine maxCpus='1'>isapc</machine>
      <machine maxCpus='255'>pc-q35-2.5</machine>
      <machine maxCpus='255'>pc-0.15</machine>
      <machine maxCpus='255'>pc-i440fx-1.5</machine>
      <machine maxCpus='255'>pc-0.14</machine>
      <machine maxCpus='255'>pc-i440fx-2.6</machine>
      <machine maxCpus='255'>pc-i440fx-1.4</machine>
      <machine maxCpus='255'>pc-i440fx-2.5</machine>
      <machine maxCpus='255'>pc-1.1</machine>
      <machine maxCpus='255'>pc-i440fx-2.1</machine>
      <machine maxCpus='288'>pc-q35-2.8</machine>
      <machine canonical='pc-q35-2.8' maxCpus='288'>q35</machine>
      <machine maxCpus='255'>pc-1.0</machine>
      <machine maxCpus='255'>pc-i440fx-2.0</machine>
      <machine maxCpus='255'>pc-q35-2.4</machine>
      <machine maxCpus='255'>pc-0.13</machine>
      <domain type='qemu'/>
    </arch>
    <features>
      <cpuselection/>
      <deviceboot/>
      <disksnapshot default='on' toggle='no'/>
      <acpi default='on' toggle='yes'/>
      <apic default='on' toggle='no'/>
    </features>
  </guest>
</capabilities>


It correctly detected 'ppc64le' as the host arch.

There is no <guest> reported with an arch of ppc64le though - only ppc and ppc64 (be).

There domain XML does not request any architecture. So libvirt tries to find an emulator for the host architecture (ppc64le) and fails, so tries to find any emulator and thus finds arm7.

So the key question is why we've not found ppc64le emulators
Comment 4 Daniel Berrangé 2016-12-12 11:22:32 UTC 
When probing caps, libvirt iterates over every arch and tries to find qemu-system-$ARCH.  Now ppc64le is a special case, so we have code to look for qemu-system-ppc64 for the ppc64le arch. This is in virQEMUCapsFindTarget().

Unfortunately looking at the code, it seems we have a regression introduced there by

commit f2e71550d62f08dad93697ec56a6bd1c960b5f03
Author: Jiri Denemark <jdenemar>
Date:   Tue Jun 14 17:25:58 2016 +0200

    qemu: Introduce virQEMUCapsGuestIsNative
    
THis commit replaced

 
-    /* armv7l guests on aarch64 hosts can use the aarch64 target
-     * i686 guests on x86_64 hosts can use the x86_64 target */
-    if ((guestarch == VIR_ARCH_ARMV7L && hostarch == VIR_ARCH_AARCH64) ||
-        (guestarch == VIR_ARCH_I686 && hostarch == VIR_ARCH_X86_64)) {
-        return hostarch;
-    }
+    if (virQEMUCapsGuestIsNative(hostarch, guestarch))
+        guestarch = hostarch;


which causes virQEMUCapsFindTarget to return VIR_ARCH_PPC64LE again, instead of VIR_ARCH_PPC64. So we try to probe a qemu binary that doesn't exist.
Comment 5 Jiri Denemark 2016-12-13 16:22:03 UTC 
Patch sent upstream for review: https://www.redhat.com/archives/libvir-list/2016-December/msg00552.html
Comment 6 Jiri Denemark 2016-12-13 21:19:45 UTC 
This is now fixed upstream by

commit f00c00475fba4bc876e549088ec971bdce28a38b
Refs: v2.5.0-88-gf00c00475
Author:     Jiri Denemark <jdenemar>
AuthorDate: Tue Dec 13 16:58:41 2016 +0100
Commit:     Jiri Denemark <jdenemar>
CommitDate: Tue Dec 13 22:11:33 2016 +0100

    qemu: Fix virQEMUCapsFindTarget on ppc64le

    virQEMUCapsFindTarget is supposed to find an alternative QEMU binary if
    qemu-system-$GUEST_ARCH doesn't exist. The alternative is using host
    architecture when it is compatible with $GUEST_ARCH. But a special
    treatment has to be applied for ppc64le since the QEMU binary is always
    called qemu-system-ppc64.

    Broken by me in v2.2.0-171-gf2e71550d.

    https://bugzilla.redhat.com/show_bug.cgi?id=1403745

    Signed-off-by: Jiri Denemark <jdenemar>
Comment 7 Richard W.M. Jones 2016-12-13 22:57:34 UTC 
I have reenabled the libguestfs tests for %{power64}.  They are only
when libguestfs is built in Rawhide, which is usually every few days
but at the moment is every few weeks b/c holidays etc.
Comment 8 Fedora End Of Life 2017-02-28 10:46:20 UTC 
This bug appears to have been reported against 'rawhide' during the Fedora 26 development cycle.
Changing version to '26'.
Comment 9 Cole Robinson 2017-05-04 17:47:59 UTC 
This has been in f26 for a couple months