Bug 1171501 - Cubietruck: cannot create KVM guests: "kvm_init_vcpu failed: Invalid argument"
Cubietruck: cannot create KVM guests: "kvm_init_vcpu failed: Invalid argument"
Status: CLOSED UPSTREAM
Product: Fedora
Classification: Fedora
Component: libguestfs (Show other bugs)
rawhide
armv7hl Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Richard W.M. Jones
Fedora Extras Quality Assurance
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2014-12-07 14:46 EST by Kashyap Chamarthy
Modified: 2014-12-09 12:35 EST (History)
13 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2014-12-09 12:35:51 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Kashyap Chamarthy 2014-12-07 14:46:21 EST
Description of problem
----------------------

Invoking `libguestfs-test-tool` with 'direct' backend on Fedora-21 running on a 
Cubietruck fails with:

  kvm_init_vcpu failed: Invalid argument


Version
-------

  cubietruck]$ uname -r; rpm -q qemu-system-arm libguestfs
  3.17.4-302.fc21.armv7hl+lpae
  qemu-system-arm-2.1.2-7.fc21.armv7hl
  libguestfs-1.28.4-2.fc21.armv7hl


How reproducible: Consistently.


Steps to Reproduce
------------------

(1) Install Fedora-21 ARM on a Cubietruck. I obtained the image from here:

  http://be.mirror.eurid.eu/fedora/linux/releases/test/21-Beta/Images/armhfp/Fedora-Minimal-armhfp-21_Beta-4-sda.raw.xz

(2) Install libguestfs tools:

  $ yum install libguestfs-tools-c -y

(3) Enable debugging and libguestfs 'direct' backend:

  $ export LIBGUESTFS_DEBUG=1
  $ export LIBGUESTFS_TRACE=1
  $ export LIBGUESTFS_BACKEND=direct

(3) Invoke the 'libguestfs-test-tool':

  $ libguestfs-test-tool


Actual results
--------------

=======================================================================
$ libguestfs-test-tool
libguestfs: trace: set_verbose true
libguestfs: trace: set_verbose = 0
libguestfs: trace: set_backend "direct"
libguestfs: trace: set_backend = 0
libguestfs: trace: set_verbose true
libguestfs: trace: set_verbose = 0
     ************************************************************
     *                    IMPORTANT NOTICE
     *
     * When reporting bugs, include the COMPLETE, UNEDITED
     * output below in your bug report.
     *
     ************************************************************
LIBGUESTFS_DEBUG=1
LIBGUESTFS_BACKEND=direct
LIBGUESTFS_TRACE=1
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin
SELinux: Permissive
libguestfs: trace: add_drive_scratch 104857600
libguestfs: trace: get_tmpdir
libguestfs: trace: get_tmpdir = "/tmp"
libguestfs: trace: disk_create "/tmp/libguestfsDKHlqd/scratch.1" "raw" 104857600
libguestfs: trace: disk_create = 0
libguestfs: trace: add_drive "/tmp/libguestfsDKHlqd/scratch.1" "format:raw" "cachemode:unsafe"
libguestfs: trace: add_drive = 0
libguestfs: trace: add_drive_scratch = 0
libguestfs: trace: get_append
libguestfs: trace: get_append = "NULL"
libguestfs: trace: get_autosync
libguestfs: trace: get_autosync = 1
libguestfs: trace: get_backend
libguestfs: trace: get_backend = "direct"
libguestfs: trace: get_backend_settings
libguestfs: trace: get_backend_settings = []
libguestfs: trace: get_cachedir
libguestfs: trace: get_cachedir = "/var/tmp"
libguestfs: trace: get_direct
libguestfs: trace: get_direct = 0
libguestfs: trace: get_hv
libguestfs: trace: get_hv = "/usr/bin/qemu-system-arm"
libguestfs: trace: get_memsize
libguestfs: trace: get_memsize = 500
libguestfs: trace: get_network
libguestfs: trace: get_network = 0
libguestfs: trace: get_path
libguestfs: trace: get_path = "/usr/lib/guestfs"
libguestfs: trace: get_pgroup
libguestfs: trace: get_pgroup = 0
libguestfs: trace: get_program
libguestfs: trace: get_program = "libguestfs-test-tool"
libguestfs: trace: get_recovery_proc
libguestfs: trace: get_recovery_proc = 1
libguestfs: trace: get_selinux
libguestfs: trace: get_selinux = 0
libguestfs: trace: get_smp
libguestfs: trace: get_smp = 1
libguestfs: trace: get_tmpdir
libguestfs: trace: get_tmpdir = "/tmp"
libguestfs: trace: get_trace
libguestfs: trace: get_trace = 1
libguestfs: trace: get_verbose
libguestfs: trace: get_verbose = 1
guestfs_get_append: (null)
guestfs_get_autosync: 1
guestfs_get_backend: direct
guestfs_get_backend_settings: []
guestfs_get_cachedir: /var/tmp
guestfs_get_direct: 0
guestfs_get_hv: /usr/bin/qemu-system-arm
guestfs_get_memsize: 500
guestfs_get_network: 0
guestfs_get_path: /usr/lib/guestfs
guestfs_get_pgroup: 0
guestfs_get_program: libguestfs-test-tool
guestfs_get_recovery_proc: 1
guestfs_get_selinux: 0
guestfs_get_smp: 1
guestfs_get_tmpdir: /tmp
guestfs_get_trace: 1
guestfs_get_verbose: 1
host_cpu: armv7hl
Launching appliance, timeout set to 600 seconds.
libguestfs: trace: launch
libguestfs: trace: version
libguestfs: trace: version = <struct guestfs_version *>
libguestfs: trace: get_backend
libguestfs: trace: get_backend = "direct"
libguestfs: launch: program=libguestfs-test-tool
libguestfs: launch: version=1.28.4fedora=21,release=2.fc21,libvirt
libguestfs: launch: backend registered: unix
libguestfs: launch: backend registered: uml
libguestfs: launch: backend registered: libvirt
libguestfs: launch: backend registered: direct
libguestfs: launch: backend=direct
libguestfs: launch: tmpdir=/tmp/libguestfsDKHlqd
libguestfs: launch: umask=0022
libguestfs: launch: euid=0
libguestfs: trace: get_backend_setting "force_tcg"
libguestfs: trace: get_backend_setting = NULL (error)
libguestfs: trace: get_cachedir
libguestfs: trace: get_cachedir = "/var/tmp"
libguestfs: [00005ms] begin building supermin appliance
libguestfs: [00005ms] run supermin
libguestfs: command: run: /usr/bin/supermin
libguestfs: command: run: \ --build
libguestfs: command: run: \ --verbose
libguestfs: command: run: \ --if-newer
libguestfs: command: run: \ --lock /var/tmp/.guestfs-0/lock
libguestfs: command: run: \ --copy-kernel
libguestfs: command: run: \ -f ext2
libguestfs: command: run: \ --host-cpu armv7hl
libguestfs: command: run: \ --dtb vexpress*a15-tc1.dtb
libguestfs: command: run: \ /usr/lib/guestfs/supermin.d
libguestfs: command: run: \ -o /var/tmp/.guestfs-0/appliance.d
supermin: version: 5.1.10
supermin: rpm: detected RPM version 4.12
supermin: package handler: fedora/rpm
supermin: acquiring lock on /var/tmp/.guestfs-0/lock
supermin: build: /usr/lib/guestfs/supermin.d
supermin: build: visiting /usr/lib/guestfs/supermin.d/base.tar.gz type gzip base image (tar)
supermin: build: visiting /usr/lib/guestfs/supermin.d/daemon.tar.gz type gzip base image (tar)
supermin: build: visiting /usr/lib/guestfs/supermin.d/excludefiles type uncompressed excludefiles
supermin: build: visiting /usr/lib/guestfs/supermin.d/hostfiles type uncompressed hostfiles
supermin: build: visiting /usr/lib/guestfs/supermin.d/init.tar.gz type gzip base image (tar)
supermin: build: visiting /usr/lib/guestfs/supermin.d/packages type uncompressed packages
supermin: build: visiting /usr/lib/guestfs/supermin.d/udev-rules.tar.gz type gzip base image (tar)
supermin: build: 190 packages, including dependencies
supermin: build: 27373 files
supermin: build: 13117 files, after removing unreadable files
supermin: build: 7198 files, after matching excludefiles
supermin: build: 7203 files, after adding hostfiles
supermin: build: 7227 files, after munging
supermin: kernel: picked kernel vmlinuz-3.17.4-302.fc21.armv7hl
supermin: kernel: picked dtb /boot/dtb-3.17.4-302.fc21.armv7hl/vexpress-v2p-ca15-tc1.dtb
supermin: kernel: picked modules path /lib/modules/3.17.4-302.fc21.armv7hl
supermin: kernel: kernel_version 3.17.4-302.fc21.armv7hl
supermin: kernel: modules /lib/modules/3.17.4-302.fc21.armv7hl
supermin: ext2: creating empty ext2 filesystem '/var/tmp/.guestfs-0/appliance.d.9t7x3qry/root'
warning: Unable to get device geometry for /var/tmp/.guestfs-0/appliance.d.9t7x3qry/root
supermin: ext2: populating from base image
supermin: ext2: copying files from host filesystem
supermin: ext2: copying kernel modules
supermin: ext2: creating minimal initrd '/var/tmp/.guestfs-0/appliance.d.9t7x3qry/initrd'
supermin: ext2: wrote 23 modules to minimal initrd
supermin: renaming /var/tmp/.guestfs-0/appliance.d.9t7x3qry to /var/tmp/.guestfs-0/appliance.d
libguestfs: [189597ms] finished building supermin appliance
libguestfs: [189597ms] begin testing qemu features
libguestfs: command: run: /usr/bin/qemu-system-arm
libguestfs: command: run: \ -display none
libguestfs: command: run: \ -help
libguestfs: command: run: /usr/bin/qemu-system-arm
libguestfs: command: run: \ -display none
libguestfs: command: run: \ -version
libguestfs: qemu version 2.1
libguestfs: command: run: /usr/bin/qemu-system-arm
libguestfs: command: run: \ -display none
libguestfs: command: run: \ -M vexpress-a15
libguestfs: command: run: \ -machine accel=kvm:tcg
libguestfs: command: run: \ -device ?
libguestfs: [192330ms] finished testing qemu features
libguestfs: trace: get_backend_setting "gdb"
libguestfs: trace: get_backend_setting = NULL (error)
[192340ms] /usr/bin/qemu-system-arm \
    -global virtio-blk-device.scsi=off \
    -nodefconfig \
    -enable-fips \
    -nodefaults \
    -display none \
    -M vexpress-a15 \
    -machine accel=kvm:tcg \
    -m 500 \
    -no-reboot \
    -rtc driftfix=slew \
    -global kvm-pit.lost_tick_policy=discard \
    -kernel /var/tmp/.guestfs-0/appliance.d/kernel \
    -dtb /var/tmp/.guestfs-0/appliance.d/dtb \
    -initrd /var/tmp/.guestfs-0/appliance.d/initrd \
    -device virtio-scsi-device,id=scsi \
    -drive file=/tmp/libguestfsDKHlqd/scratch.1,cache=unsafe,format=raw,id=hd0,if=none \
    -device scsi-hd,drive=hd0 \
    -drive file=/var/tmp/.guestfs-0/appliance.d/root,snapshot=on,id=appliance,cache=unsafe,if=none \
    -device scsi-hd,drive=appliance \
    -device virtio-serial-device \
    -serial stdio \
    -chardev socket,path=/tmp/libguestfsDKHlqd/guestfsd.sock,id=channel0 \
    -device virtserialport,chardev=channel0,name=org.libguestfs.channel.0 \
    -append 'panic=1 mem=500M console=ttyAMA0 udevtimeout=6000 udev.event-timeout=6000 no_timer_check acpi=off printk.time=1 cgroup_disable=memory root=/dev/sdb selinux=0 guestfs_verbose=1 TERM=vt102'
kvm_init_vcpu failed: Invalid argument
libguestfs: error: appliance closed the connection unexpectedly, see earlier error messages
libguestfs: child_cleanup: 0xb8afa768: child process died
libguestfs: sending SIGTERM to process 8430
libguestfs: error: /usr/bin/qemu-system-arm exited with error status 1, see debug messages above
libguestfs: error: guestfs_launch failed, see earlier error messages
libguestfs: trace: launch = -1 (error)
libguestfs-test-tool: failed to launch appliance
libguestfs: trace: close
libguestfs: closing guestfs handle 0xb8afa768 (state 0)
libguestfs: command: run: rm
libguestfs: command: run: \ -rf /tmp/libguestfsDKHlqd
=======================================================================


Expected results
----------------

`libguestfs-test-tool` should invoke successfully


Additional info
---------------

Before running the `libguestfs-test-tool`, I flushed the buffers, to ensure I had enough memory:

$ free && sync && echo 3 > /proc/sys/vm/drop_caches && free
             total       used       free     shared    buffers     cached
Mem:       2064920    1559580     505340        232      57248    1361652
-/+ buffers/cache:     140680    1924240
Swap:            0          0          0
             total       used       free     shared    buffers     cached
Mem:       2064920     129520    1935400        232        928      39180
-/+ buffers/cache:      89412    1975508
Swap:            0          0          0
Comment 1 Kashyap Chamarthy 2014-12-07 14:50:17 EST
From the inter-webs, seems like this is a reasonably popular error for ARM and KVM. I stumbled on a thread (Tue Dec 10 09:39:09 UTC 2013) 
on fedora-arm list:

  https://lists.fedoraproject.org/pipermail/arm/2013-December/007155.html
  uboot with hyp mode support now available for the cubietruck


Quoting Richard Jones from the above email:

    "I can confirm this works at least as far as running libguestfs
    appliances using KVM.  I had to use the attached hack to make qemu
    create Cortex-A7 guests, otherwise you get the following error":

    kvm_init_vcpu failed: Invalid argument



    -------------------------------------------------
    diff --git a/target-arm/kvm.c b/target-arm/kvm.c
    index 6e5cd36..48b2773 100644
    --- a/target-arm/kvm.c
    +++ b/target-arm/kvm.c
    @@ -86,7 +86,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
         struct kvm_reg_list *rlp;
         ARMCPU *cpu = ARM_CPU(cs);
 
    -    init.target = KVM_ARM_TARGET_CORTEX_A15;
    +    init.target = KVM_ARM_TARGET_CORTEX_A7; /* XXX HACK */
         memset(init.features, 0, sizeof(init.features));
         ret = kvm_vcpu_ioctl(cs, KVM_ARM_VCPU_INIT, &init);
         if (ret) {
Comment 2 Richard W.M. Jones 2014-12-09 03:26:58 EST
For some reason I thought this had been fixed upstream, but
now that I've finally got my CT working again, I see that I
am still carrying that patch in my custom qemu.

diff --git a/target-arm/cpu.c b/target-arm/cpu.c
index 5ce7350..04d69d1 100644
--- a/target-arm/cpu.c
+++ b/target-arm/cpu.c
@@ -858,7 +858,7 @@ static void cortex_a15_initfn(Object *obj)
     set_feature(&cpu->env, ARM_FEATURE_DUMMY_C15_REGS);
     set_feature(&cpu->env, ARM_FEATURE_CBAR_RO);
     set_feature(&cpu->env, ARM_FEATURE_LPAE);
-    cpu->kvm_target = QEMU_KVM_ARM_TARGET_CORTEX_A15;
+    cpu->kvm_target = QEMU_KVM_ARM_TARGET_CORTEX_A7;
     cpu->midr = 0x412fc0f1;
     cpu->reset_fpsid = 0x410430f0;
     cpu->mvfr0 = 0x10110222;

So that's the answer really, it's a qemu bug.  Actually it looks as
if qemu contains some code to try to get the host CPU type, but it
doesn't work, or maybe we need to pass a -cpu option ...
Comment 3 Kashyap Chamarthy 2014-12-09 12:31:45 EST
This is fixed in libguestfs with this commit (committed in upstream git):

https://github.com/libguestfs/libguestfs/commit/6468b9d60172bcef76b36c14b72f9a1913d94954
arm: Use -cpu host on KVM.


With the above commit applied, KVM guest (a libguestfs appliance) successfully boots on Cubietruck:

. . .
[01433ms] /bin/qemu-system-arm \
    -global virtio-blk-device.scsi=off \
    -nodefconfig \
    -enable-fips \
    -nodefaults \
    -display none \
    -M vexpress-a15 \
    -cpu host \
    -machine accel=kvm:tcg \
    -m 500 \
    -no-reboot \
    -rtc driftfix=slew \
    -global kvm-pit.lost_tick_policy=discard \
    -kernel /home/kashyapc/src/libguestfs/tmp/.guestfs-1000/appliance.d/kernel \
    -dtb /home/kashyapc/src/libguestfs/tmp/.guestfs-1000/appliance.d/dtb \
    -initrd /home/kashyapc/src/libguestfs/tmp/.guestfs-1000/appliance.d/initrd \
    -device virtio-scsi-device,id=scsi \
    -drive 
file=/home/kashyapc/src/libguestfs/tmp/libguestfsRMfHzO/scratch.1,cache=unsafe,format=raw,id=hd0,if=none
 \
    -device scsi-hd,drive=hd0 \
    -drive 
file=/home/kashyapc/src/libguestfs/tmp/.guestfs-1000/appliance.d/root,snapshot=on,id=appliance,cache=unsafe,if=none
 \
    -device scsi-hd,drive=appliance \
    -device virtio-serial-device \
    -serial stdio \
    -chardev 
socket,path=/home/kashyapc/src/libguestfs/tmp/libguestfsRMfHzO/guestfsd.sock,id=channel0
 \
    -device virtserialport,chardev=channel0,name=org.libguestfs.channel.0 \
    -append 'panic=1 mem=500M console=ttyAMA0 udevtimeout=6000 
udev.event-timeout=6000 no_timer_check acpi=off printk.time=1 
cgroup_disable=memory root=/dev/sdb selinux=0 guestfs_verbose=1 TERM=vt102'
. . .



Some more context on this upstream QEMU mailing list thread:

http://lists.nongnu.org/archive/html/qemu-devel/2014-12/msg01093.html
Comment 4 Richard W.M. Jones 2014-12-09 12:35:51 EST
Yup, all good now so let's close this bug.

Note You need to log in before you can comment on or make changes to this bug.