Bug 1457920 - Hot-unplugging PCI Express virtio devices causes driver reload on Windows Server 2016
Summary: Hot-unplugging PCI Express virtio devices causes driver reload on Windows Ser...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Linux Advanced Virtualization
Classification: Red Hat
Component: qemu-kvm
Version: 8.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: rc
: ---
Assignee: Vadim Rozenfeld
QA Contact: jingzhao
URL:
Whiteboard:
Depends On:
Blocks: 1744438
TreeView+ depends on / blocked
 
Reported: 2017-06-01 14:17 UTC by Ladi Prosek
Modified: 2021-12-17 17:24 UTC (History)
12 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-03-18 01:23:41 UTC
Type: Bug
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1455488 0 high CLOSED [Virtio-win][vioser][ovmf] Guest occured BSOD when hotunplug virtio-serial-pci. 2021-02-22 00:41:40 UTC
Red Hat Issue Tracker RHELPLAN-26562 0 None None None 2021-12-17 17:24:19 UTC

Internal Links: 1455488

Description Ladi Prosek 2017-06-01 14:17:25 UTC
Description of problem:
Hot-unplugging a PCIe virtio device results in Windows shutting down and unloading the driver, almost immediately followed by reloading and initializing the driver again, and then unloading it for good a few seconds later.

If the timing is right, the device's BARs may get unmapped as the driver is initializing, exercising otherwise rarely hit error code paths (see bug 1455488 for one such case).

Version-Release number of selected component (if applicable):
kernel-3.10.0-671.el7.x86_64
qemu-kvm-rhev-2.9.0-5.el7.x86_64
virtio-win-prewhql-0.1-137

Tested with virtio-serial and balloon, the observed behavior is the same. The scope is likely all virtio devices or even all PCIe devices.

How reproducible:
100%

Steps to Reproduce:
1. Boot a VM with

  -device ioh3420,bus=pcie.0,id=root1.0 \
  -device virtio-balloon,bus=root1.0,id=balloon0 \

2. device_del balloon0
3. Watch the driver shut down, reinitialize, and shut down again with DbgView for example.

Actual results:
Windows unloads, loads, and unloads the driver.

Expected results:
Windows unloads the driver.

Additional info:

Connecting the device to a downstream port doesn't seem to make a difference:

  -device ioh3420,bus=pcie.0,id=root1.0 \
  -device x3130-upstream,bus=root1.0,id=upstream1 \
  -device xio3130-downstream,bus=upstream1,id=downstream1,chassis=1 \
  -device virtio-balloon,bus=downstream1,id=balloon0 \

Comment 2 Ladi Prosek 2017-06-01 14:25:52 UTC
CC Marcel to
1) Tell me that I'm doing it wrong.
2) Or take a guess on what might be causing this (a wild guess is fine too).

I have opened this as a qemu-kvm-rhev bug because virtio-win drivers do not directly participate in hot-(un)-plug. PnP logic is generally handled by Windows and it's unlikely that we would be able to change it.

Comment 3 Marcel Apfelbaum 2017-06-05 16:38:39 UTC
(In reply to Ladi Prosek from comment #2)
> CC Marcel to
> 1) Tell me that I'm doing it wrong.
> 2) Or take a guess on what might be causing this (a wild guess is fine too).
> 
> I have opened this as a qemu-kvm-rhev bug because virtio-win drivers do not
> directly participate in hot-(un)-plug. PnP logic is generally handled by
> Windows and it's unlikely that we would be able to change it.

Hi Ladi,
Sorry it took me some time for the first response.
From the QEMU point of view we are sending basic PCIe hot plug
events, nothing special there. (a bug is always possible, of course...)

I tried with win10 (tomorrow I'll get a win16 guest from Gal).
I opened windbbg and did a few hotplug/unplug operations and I saw nothing
special in Windbg (I especially didn't see the drivers load/unload). I selected all options for debug of course.

I have virtio 1.9 drivers.

The steps you mentioned are just fine, but I suggest to try starting
the hot-unplug sequence from windows itself. In this way you
are sure Windows does not use the device at hot-unplugged time.

Can you guide me on how to see the driver loading/unloading?

Comment 4 Ladi Prosek 2017-06-06 11:31:42 UTC
Thanks Marcel,

No worries, this is not a critical issue, it's unlikely that it would be considered a RHEL 7.4 blocker for RHEL 7.4, even if it ends up being a QEMU bug.

Please try it with virtio-net. The NetKVM driver version 100.73.104.12600 extracted from virtio-win-1.9.0-3.el7.noarch.rpm has pretty verbose debug output compiled in. Apologies for not including this info in comment 0.

My command line now has

  -netdev user,id=hostnet1 \
  -device virtio-net,netdev=hostnet1,bus=root1.0,id=virtio-net0 \

After
(qemu) device_del virtio-net0

You should see something like this in windbg or DbgView:

00000001        13.37042046     [ParaNdis6_Pause]=>
<driver shutting down>
00000070        13.63768864     [ParaNdis6_Unload]=>

then a gap of 100-150 milliseconds and:

00000072        13.74919987     [ParaNdis_DebugInitialize] Crash callback registered
00000073        13.74920845     [DriverEntry]=>
00000074        13.74921322     Aug 11 2016 04:05:29built 6.30
<driver initializing, then shutting down again>

and finally

00000175        15.62887096     [ParaNdis6_Unload]=>


Starting the hot-unplug sequence from Windows (i.e. "ejecting" the device) results in the same symptoms.

Thanks!
Ladi

Comment 10 Ademar Reis 2020-02-05 22:43:57 UTC
QEMU has been recently split into sub-components and as a one-time operation to avoid breakage of tools, we are setting the QEMU sub-component of this BZ to "General". Please review and change the sub-component if necessary the next time you review this BZ. Thanks


Note You need to log in before you can comment on or make changes to this bug.