RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1661976 - qemu-kvm: failed to set irq for PMU
Summary: qemu-kvm: failed to set irq for PMU
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: qemu-kvm
Version: 8.0
Hardware: aarch64
OS: Unspecified
high
high
Target Milestone: rc
: 8.0
Assignee: Virtualization Maintenance
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-12-25 05:48 UTC by Li Shuang
Modified: 2020-03-04 08:28 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-12-27 13:32:49 UTC
Type: Bug
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)

Description Li Shuang 2018-12-25 05:48:39 UTC 
Description of problem:
# /usr/libexec/qemu-kvm -name debug -drive file=/home/debug_el8.aarch64.qcow2,if=none,id=drive-virtio-disk1,media=disk,cache=none,snapshot=off,format=qcow2,aio=native -device virtio-blk-pci,drive=drive-virtio-disk1,id=virtio-disk1,bootindex=0 -cpu host -drive file=/usr/share/AAVMF/AAVMF_CODE.fd,if=pflash,format=raw,unit=0,readonly=on -drive file=/var/lib/libvirt/qemu/nvram/debug_VARS.fd,if=pflash,format=raw,unit=1  -netdev tap,id=hostnet0,vhost=on,script=/etc/br_wan_ifup,downscript=/etc/br_wan_ifdn,ifname=tap_debug_0 -device virtio-net-pci,netdev=hostnet0,id=virtio-net-pci0,mac=00:e4:a3:83:14:e3 -netdev tap,id=hostnet1,vhost=on,script=/etc/br_lan_ifup,downscript=/etc/br_lan_ifdn,ifname=tap_debug_1 -device virtio-net-pci,netdev=hostnet1,id=virtio-net-pci1,mac=00:a7:c8:b3:76:b2 -netdev tap,id=hostnet2,vhost=on,script=/etc/br_lan_ifup,downscript=/etc/br_lan_ifdn,ifname=tap_debug_2 -device virtio-net-pci,netdev=hostnet2,id=virtio-net-pci2,mac=00:c1:2b:b5:f0:e2 -device pcie-root-port,port=0xb,chassis=4,id=pci.4,bus=pcie.0 -device pcie-root-port,port=0xc,chassis=5,id=pci.5,bus=pcie.0 -serial file:/home/debug_el8.console -serial pty -vnc :10 -qmp tcp:0:11010,server,nowait  -smp 2,cores=1,threads=1,sockets=2 -m 4096
qemu-kvm: -serial pty: char device redirected to /dev/pts/2 (label serial1)
qemu-kvm: PMU: KVM_SET_DEVICE_ATTR: Invalid argument
qemu-kvm: failed to set irq for PMU
Aborted (core dumped)

# tail -5 /var/log/messages 
Dec 25 00:44:15 qualcomm-amberwing-rep-04 kvm[30901]: 0 guests now active
Dec 25 00:44:15 qualcomm-amberwing-rep-04 systemd-coredump[30889]: Process 30855 (qemu-kvm) of user 0 dumped core.#012#012Stack trace of thread 30855:#012#0  0x0000ffff88ad2d0c raise (libc.so.6)#012#1  0x0000ffff88ac08e8 abort (libc.so.6)#012#2  0x0000aaaae1087300 kvm_arm_pmu_set_irq (qemu-kvm)#012#3  0x0000aaaae107efd8 machvirt_init (qemu-kvm)#012#4  0x0000aaaae1152570 machine_run_board_init (qemu-kvm)#012#5  0x0000aaaae0fcfe74 main (qemu-kvm)#012#6  0x0000ffff88ac0d24 __libc_start_main (libc.so.6)#012#7  0x0000aaaae0fd1ccc _start (qemu-kvm)#012#8  0x0000aaaae0fd1ccc _start (qemu-kvm)
Dec 25 00:44:49 qualcomm-amberwing-rep-04 restraintd[4676]: *** Current Time: Tue Dec 25 00:44:49 2018 Localwatchdog at:  * Disabled! *
Dec 25 00:45:49 qualcomm-amberwing-rep-04 restraintd[4676]: *** Current Time: Tue Dec 25 00:45:49 2018 Localwatchdog at:  * Disabled! *
Dec 25 00:46:49 qualcomm-amberwing-rep-04 restraintd[4676]: *** Current Time: Tue Dec 25 00:46:49 2018 Localwatchdog at:  * Disabled! *


Version-Release number of selected component (if applicable):
RHEL-8.0-20181220.1 ==> kernel-4.18.0-56.el8
qemu-kvm-core-2.12.0-49.module+el8+2586+bf759444.aarch64


How reproducible:
sometimes


Steps to Reproduce:
1. install packages and prepare test environment
# yum install kernel-kernel-networking-common -y
# yum install python3-pexpect -y
# cd /mnt/tests/kernel/networking/
# source common/vm/vm.sh
# vinit
# vstart debug
(here we will find that the vstart failed)
2. run qemu-kvm manually
# /usr/libexec/qemu-kvm -name debug -drive file=/home/debug_el8.aarch64.qcow2,if=none,id=drive-virtio-disk1,media=disk,cache=none,snapshot=off,format=qcow2,aio=native -device virtio-blk-pci,drive=drive-virtio-disk1,id=virtio-disk1,bootindex=0 -cpu host -drive file=/usr/share/AAVMF/AAVMF_CODE.fd,if=pflash,format=raw,unit=0,readonly=on -drive file=/var/lib/libvirt/qemu/nvram/debug_VARS.fd,if=pflash,format=raw,unit=1  -netdev tap,id=hostnet0,vhost=on,script=/etc/br_wan_ifup,downscript=/etc/br_wan_ifdn,ifname=tap_debug_0 -device virtio-net-pci,netdev=hostnet0,id=virtio-net-pci0,mac=00:e4:a3:83:14:e3 -netdev tap,id=hostnet1,vhost=on,script=/etc/br_lan_ifup,downscript=/etc/br_lan_ifdn,ifname=tap_debug_1 -device virtio-net-pci,netdev=hostnet1,id=virtio-net-pci1,mac=00:a7:c8:b3:76:b2 -netdev tap,id=hostnet2,vhost=on,script=/etc/br_lan_ifup,downscript=/etc/br_lan_ifdn,ifname=tap_debug_2 -device virtio-net-pci,netdev=hostnet2,id=virtio-net-pci2,mac=00:c1:2b:b5:f0:e2 -device pcie-root-port,port=0xb,chassis=4,id=pci.4,bus=pcie.0 -device pcie-root-port,port=0xc,chassis=5,id=pci.5,bus=pcie.0 -serial file:/home/debug_el8.console -serial pty -vnc :10 -qmp tcp:0:11010,server,nowait  -smp 2,cores=1,threads=1,sockets=2 -m 4096


Actual results:
VM start failed by qemu-kvm on aarch64.


Expected results:
VM start succeed.


Additional info:
We can reproduce this issue on the following systems:
netqe-arm-02.knqe.lab.eng.bos.redhat.com
hpe-apollo-cn99xx-07.khw3.lab.eng.bos.redhat.com
qualcomm-amberwing-rep-04.khw3.lab.eng.bos.redhat.com
Comment 1 Andrew Jones 2018-12-27 13:32:49 UTC 
On machines with GICv3 that don't support GICv2 guests you must have '-machine gic-version=3' on the QEMU command line. Also, the whole command line in comment 0 looks weird. Why isn't libvirt being used, which will ensure it's correct?
Comment 2 Adam Williamson 2020-03-04 00:44:02 UTC 
For the record, just ran into this on Fedora 31. There *are* use cases for running qemu without libvirt. I run the Fedora openQA instance; openQA is a test system which runs tests in qemu virtual machines, which it runs directly, not via libvirt.

Thanks to finding this bug I was able to configure it to use `-machine virt,gic-version=max` which makes qemu boot on both our older and newer aarch64 worker hosts, just figured I'd add a note here in case it helps anyone else.
Comment 3 Andrew Jones 2020-03-04 08:28:28 UTC 
(In reply to Adam Williamson from comment #2)
> For the record, just ran into this on Fedora 31. There *are* use cases for
> running qemu without libvirt.

This is a RHEL8 bug so there are *no* supported use cases for running qemu without libvirt.

> I run the Fedora openQA instance; openQA is a
> test system which runs tests in qemu virtual machines, which it runs
> directly, not via libvirt.

I'm glad to hear that openQA is getting run on AArch64 Fedora, but it sounds like openQA needs to be patched to better determine how to generate AArch64 QEMU command lines.

> 
> Thanks to finding this bug I was able to configure it to use `-machine
> virt,gic-version=max` which makes qemu boot on both our older and newer
> aarch64 worker hosts, just figured I'd add a note here in case it helps
> anyone else.

We do use gic-version=max for QEMU testing quite a bit, but users still need to be a bit cautious, as guests that were working with gicv2 compatibility on gicv3 hosts that support gicv2 compatibility will be magically changed to gicv3 guests. That's possibly not something the user expects. Also, on TCG or with KVM and kernel_irqchip=off, max=3, but the emulation for 3 isn't currently complete, so it won't completely work.

That said, upstream recently got patches to generate better error messages, as the PMU stuff is pretty confusing. And, there are patches on the list to autodetect gic-version=3 in certain cases, which should help avoid the problem if those patches get merged. Still, I think openQA should learn how to probe host gic support and then generate explicit, correct command lines like libvirt does.
Note You need to log in before you can comment on or make changes to this bug.