1058321 – qemu-kvm-rhev: Qemu: Q35: hw: pci: use after free triggered via guest [rhel-7.2]

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1058321 - qemu-kvm-rhev: Qemu: Q35: hw: pci: use after free triggered via guest [rhel-7.2]

Summary: qemu-kvm-rhev: Qemu: Q35: hw: pci: use after free triggered via guest [rhel-7.2]

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Linux 7
Classification:	Red Hat
Component:	qemu-kvm-rhev
Sub Component:
Version:	7.2
Hardware:	Unspecified
OS:	Unspecified
Priority:	unspecified
Severity:	unspecified
Target Milestone:	rc
Target Release:	---
Assignee:	Marcel Apfelbaum
QA Contact:	Virtualization Bugs
Docs Contact:
URL:
Whiteboard:
Depends On:	983344
Blocks:	CVE-2014-3471 1227278
TreeView+	depends on / blocked

Reported:	2014-01-27 14:38 UTC by Markus Armbruster
Modified:	2015-12-04 16:14 UTC (History)
CC List:	10 users (show)
Fixed In Version:	qemu-kvm-rhev-2.3.0
Doc Type:	Release Note
Doc Text:
Clone Of:
Environment:
Last Closed:	2015-12-04 16:14:17 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Product Errata	RHBA-2015:2546	0	normal	SHIPPED_LIVE	qemu-kvm-rhev bug fix and enhancement update	2015-12-04 21:11:56 UTC

Description Markus Armbruster 2014-01-27 14:38:37 UTC

Description of problem:
When I unplug a virtio-blk-pci device sitting in a PCIe slot of q35's
xio3130-downstream bridge, the guest kernel warns.

Version-Release number of selected component (if applicable):
At least qemu-kvm-1.5.3-43.el7, older versions crash (bug 983344)
Guest kernel-3.10.0-71.el7.x86_64

How reproducible:
Always

Steps to Reproduce:
1. Boot a RHEL-7 guest with an additional, unused virtio-blk-pci
device connected to PCIe.  This requires q35.  Relevant part of
command line
-M q35 -device ioh3420,bus=pcie.0,id=root.2,slot=3 -device x3130-upstream,bus=root.2,id=upstream2 -device xio3130-downstream,bus=upstream2,id=downstream2,chassis=3 -drive if=none,id=foo,file=tmp.qcow2 -device virtio-blk-pci,id=bar,bus=downstream2,drive=foo
2. When the guest is up, unplug with "device_del bar"

Actual results:
Unplug succeeds, but guest kernel warns (details below).

Expected results:
Unplug succeeds, guest kernel doesn't warn.

Additional info:
Also observed with current upstream QEMU.

Older guest kernels crash, details at
https://bugzilla.redhat.com/show_bug.cgi?id=983344#c14

Guest dmesg:
[   37.674257] ------------[ cut here ]------------
[   37.674296] WARNING: at drivers/virtio/virtio.c:158 virtio_dev_remove+0x74/0x80 [virtio]()
[   37.674301] Modules linked in: ip6t_rpfilter ip6t_REJECT ipt_REJECT xt_conntrack ebtable_nat ebtable_broute bridge stp llc ebtable_filter ebtables ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw ip6table_filter ip6_tables iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw iptable_filter ip_tables sg kvm_amd kvm pcspkr serio_raw i2c_i801 lpc_ich mfd_core mperf shpchp xfs libcrc32c sr_mod cdrom cirrus virtio_net virtio_blk syscopyarea sysfillrect sysimgblt drm_kms_helper ahci ttm libahci drm libata virtio_pci virtio_ring virtio i2c_core dm_mirror dm_region_hash dm_log dm_mod
[   37.674374] CPU: 0 PID: 22 Comm: kworker/0:1 Not tainted 3.10.0-71.el7.x86_64 #1
[   37.674380] Hardware name: Red Hat KVM, BIOS Bochs 01/01/2011
[   37.674393] Workqueue: pciehp-0 pciehp_power_thread
[   37.674397]  0000000000000009 ffff8802443dbb30 ffffffff815bd8c4 ffff8802443dbb68
[   37.674405]  ffffffff81059c61 ffff880241b0c400 ffff880241b0c408 ffffffffa0024000
[   37.674411]  ffff880244312098 0000000000000000 ffff8802443dbb78 ffffffff81059d3a
[   37.674418] Call Trace:
[   37.674431]  [<ffffffff815bd8c4>] dump_stack+0x19/0x1b
[   37.674443]  [<ffffffff81059c61>] warn_slowpath_common+0x61/0x80
[   37.674454]  [<ffffffff81059d3a>] warn_slowpath_null+0x1a/0x20
[   37.674465]  [<ffffffffa00220e4>] virtio_dev_remove+0x74/0x80 [virtio]
[   37.674476]  [<ffffffff8139535f>] __device_release_driver+0x7f/0xf0
[   37.674484]  [<ffffffff813953f3>] device_release_driver+0x23/0x30
[   37.674491]  [<ffffffff81394b88>] bus_remove_device+0x108/0x180
[   37.674498]  [<ffffffff81391485>] device_del+0x135/0x1d0
[   37.674505]  [<ffffffff8139153e>] device_unregister+0x1e/0x60
[   37.674516]  [<ffffffffa00224b6>] unregister_virtio_device+0x16/0x30 [virtio]
[   37.674527]  [<ffffffffa004c56b>] virtio_pci_remove+0x2b/0x70 [virtio_pci]
[   37.674537]  [<ffffffff812d252b>] pci_device_remove+0x3b/0xb0
[   37.674546]  [<ffffffff8139535f>] __device_release_driver+0x7f/0xf0
[   37.674553]  [<ffffffff813953f3>] device_release_driver+0x23/0x30
[   37.674560]  [<ffffffff81394b88>] bus_remove_device+0x108/0x180
[   37.674567]  [<ffffffff81391485>] device_del+0x135/0x1d0
[   37.674576]  [<ffffffff812cc064>] pci_stop_bus_device+0x94/0xa0
[   37.674583]  [<ffffffff812cc152>] pci_stop_and_remove_bus_device+0x12/0x20
[   37.674591]  [<ffffffff812e4bd8>] pciehp_unconfigure_device+0xa8/0x1b0
[   37.674599]  [<ffffffff812e4538>] pciehp_disable_slot+0x68/0x200
[   37.674607]  [<ffffffff812e4753>] pciehp_power_thread+0x83/0xf0
[   37.674616]  [<ffffffff8107862b>] process_one_work+0x17b/0x460
[   37.674623]  [<ffffffff810793db>] worker_thread+0x11b/0x400
[   37.674631]  [<ffffffff810792c0>] ? rescuer_thread+0x3e0/0x3e0
[   37.674638]  [<ffffffff8107fb90>] kthread+0xc0/0xd0
[   37.674646]  [<ffffffff8107fad0>] ? kthread_create_on_node+0x110/0x110
[   37.674653]  [<ffffffff815cd66c>] ret_from_fork+0x7c/0xb0
[   37.674659]  [<ffffffff8107fad0>] ? kthread_create_on_node+0x110/0x110
[   37.674665] ---[ end trace f5de3b0770382ce3 ]---

Comment 6 Yanhui Ma 2015-06-19 03:37:10 UTC

Reproduce:
Version of components:
qemu-kvm-1.5.3-43.el7
Guest kernel-3.10.0-71.el7.x86_64

steps:
1. Boot a RHEL-7 guest with an additional, unused virtio-blk-pci
device connected to PCIe.  

/usr/libexec/qemu-kvm -M q35 -m 4G -cpu Opteron_G3 -smp 4,sockets=4,cores=1,threads=1,maxcpus=4 -spice port=5931,disable-ticketing -monitor stdio -qmp tcp:0:6666,server,nowait -device ioh3420,bus=pcie.0,id=root.10,slot=1 -device x3130-upstream,bus=root.10,id=upstream10 -device xio3130-downstream,bus=upstream10,id=downstream10,chassis=0 -drive if=none,id=foo,file=/home/test.qcow2 -device virtio-blk-pci,id=bar,bus=downstream10,drive=foo -drive file=/home/rhel70-64-virtio-scsi.qcow2,if=none,id=drive-data-disk1,cache=writethrough,format=qcow2,werror=stop,rerror=stop -device virtio-scsi-pci,id=scsi1,addr=0x13 -device scsi-hd,drive=drive-data-disk1,bus=scsi1.0,id=data-disk1,bootindex=0 -netdev tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup -device e1000,netdev=hostnet0,id=virtio-net-pci0,mac=00:24:21:7f:b6:11,bus=pcie.0,addr=0x9

2. When the guest is up, unplug with "device_del bar"

Actual results:
Unplug succeeds, but guest kernel warns (details below).
[  203.171453] pciehp 0000:02:00.0:pcie24: Card not present on Slot(0)
[  203.174783] ------------[ cut here ]------------
[  203.175309] WARNING: at drivers/virtio/virtio.c:158 virtio_dev_remove+0x74/0x80 [virtio]()
[  203.176139] Modules linked in: fuse ip6t_rpfilter ip6t_REJECT ipt_REJECT xt_conntrack ebtable_nat ebtable_broute bridge stp llc ebtable_filter ebtables ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw ip6table_filter ip6_tables iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw iptable_filter ip_tables sg i2c_i801 lpc_ich serio_raw shpchp kvm mfd_core mperf pcspkr uinput nfsd auth_rpcgss nfs_acl lockd xfs libcrc32c sd_mod crct10dif_generic crc_t10dif crct10dif_common cirrus syscopyarea sysfillrect sysimgblt drm_kms_helper virtio_scsi virtio_blk ttm drm ahci libahci e1000 virtio_pci libata virtio_ring virtio i2c_core sunrpc dm_mirror dm_region_hash dm_log dm_mod
[  203.184857] CPU: 0 PID: 43 Comm: kworker/0:1 Not tainted 3.10.0-71.el7.x86_64 #1
[  203.185602] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
[  203.186186] Workqueue: pciehp-0 pciehp_power_thread
[  203.186682]  0000000000000009 ffff88017966fb30 ffffffff815bd8c4 ffff88017966fb68
[  203.188574]  ffffffff81059c61 ffff8801723f1800 ffff8801723f1808 ffffffffa00ad000
[  203.189404]  ffff8801795b0098 0000000000000000 ffff88017966fb78 ffffffff81059d3a
[  203.190238] Call Trace:
[  203.190505]  [<ffffffff815bd8c4>] dump_stack+0x19/0x1b
[  203.191040]  [<ffffffff81059c61>] warn_slowpath_common+0x61/0x80
[  203.191664]  [<ffffffff81059d3a>] warn_slowpath_null+0x1a/0x20
[  203.192272]  [<ffffffffa00ab0e4>] virtio_dev_remove+0x74/0x80 [virtio]
[  203.192933]  [<ffffffff8139535f>] __device_release_driver+0x7f/0xf0
[  203.193574]  [<ffffffff813953f3>] device_release_driver+0x23/0x30
[  203.194195]  [<ffffffff81394b88>] bus_remove_device+0x108/0x180
[  203.194784]  [<ffffffff81391485>] device_del+0x135/0x1d0
[  203.195324]  [<ffffffff8139153e>] device_unregister+0x1e/0x60
[  203.195899]  [<ffffffffa00ab4b6>] unregister_virtio_device+0x16/0x30 [virtio]
[  203.196622]  [<ffffffffa008d56b>] virtio_pci_remove+0x2b/0x70 [virtio_pci]
[  203.197318]  [<ffffffff812d252b>] pci_device_remove+0x3b/0xb0
[  203.197891]  [<ffffffff8139535f>] __device_release_driver+0x7f/0xf0
[  203.198522]  [<ffffffff813953f3>] device_release_driver+0x23/0x30
[  203.199140]  [<ffffffff81394b88>] bus_remove_device+0x108/0x180
[  203.199723]  [<ffffffff81391485>] device_del+0x135/0x1d0
[  203.200264]  [<ffffffff812cc064>] pci_stop_bus_device+0x94/0xa0
[  203.200855]  [<ffffffff812cc152>] pci_stop_and_remove_bus_device+0x12/0x20
[  203.201559]  [<ffffffff812e4bd8>] pciehp_unconfigure_device+0xa8/0x1b0
[  203.202242]  [<ffffffff812e4538>] pciehp_disable_slot+0x68/0x200
[  203.202851]  [<ffffffff812e4753>] pciehp_power_thread+0x83/0xf0
[  203.203460]  [<ffffffff8107862b>] process_one_work+0x17b/0x460
[  203.204067]  [<ffffffff810793db>] worker_thread+0x11b/0x400
[  203.204626]  [<ffffffff810792c0>] ? rescuer_thread+0x3e0/0x3e0
[  203.205247]  [<ffffffff8107fb90>] kthread+0xc0/0xd0
[  203.205740]  [<ffffffff8107fad0>] ? kthread_create_on_node+0x110/0x110
[  203.206415]  [<ffffffff815cd66c>] ret_from_fork+0x7c/0xb0
[  203.206966]  [<ffffffff8107fad0>] ? kthread_create_on_node+0x110/0x110
[  203.207638] ---[ end trace 34b69ce9c6a3d31a ]---

As above show, this bz has been reproduce.


===================
Verify:
Version of components:
qemu-kvm-rhev-2.3.0-1.el7.x86_64
Guest kernel-3.10.0-71.el7.x86_64

Steps as above show, after step 2, unplug succeeds, no guest kernel warns
So this bz has been verified.

Comment 11 errata-xmlrpc 2015-12-04 16:14:17 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2546.html

Note You need to log in before you can comment on or make changes to this bug.