Bug 1455122
| Summary: | [Virtio-win][pvpanic] Driver is uninstalled after triggering BSOD. | |||
|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | xiagao | |
| Component: | virtio-win | Assignee: | Amnon Ilan <ailan> | |
| virtio-win sub component: | virtio-win-prewhql | QA Contact: | Virtualization Bugs <virt-bugs> | |
| Status: | CLOSED NOTABUG | Docs Contact: | ||
| Severity: | medium | |||
| Priority: | medium | CC: | ailan, coli, hachen, juzhang, lijin, lveyde, michen, phou, sbonazzo, virt-bugs, wyu, xiagao, yvugenfi | |
| Version: | 7.4 | Keywords: | TestOnly | |
| Target Milestone: | rc | |||
| Target Release: | --- | |||
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | ||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | ||
| Doc Text: | Story Points: | --- | ||
| Clone Of: | ||||
| : | 1500268 (view as bug list) | Environment: | ||
| Last Closed: | 2018-02-26 11:53:07 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | ||||
| Bug Blocks: | 1473046, 1500268 | |||
qemu cmd line: /usr/libexec/qemu-kvm -name 137PNCW10S646T1 -enable-kvm -m 3G -smp 4 -uuid 2ce4aa36-83af-4294-9519-dd14f8c69927 -nodefconfig -nodefaults -rtc base=localtime,driftfix=slew -boot order=cd,menu=on -device piix3-usb-uhci,id=usb -drive file=136SCSW10S648OH,if=none,id=drive-ide0-0-0,format=raw,serial=mike_cao,cache=none -device ide-drive,bus=ide.0,drive=drive-ide0-0-0,id=ide0-0-0 -drive file=en_windows_server_2016_x64_dvd_9718492.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,script=/etc/qemu-ifup,downscript=no,id=hostnet0 -device e1000,netdev=hostnet0,id=net0,mac=00:52:0a:17:54:4b -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=isa_serial0 -device usb-tablet,id=input0 -vnc 0.0.0.0:0 -vga std -M pc -monitor stdio -qmp tcp:0:1234,server,nowait -cdrom virtio-win-prewhql-137.iso -device pvpanic,id=pvpanic0,ioport=0x0505 -no-shutdown I can reproduce the driver disappearing, but only if I don't reboot between the driver install and the BSOD. 1. Driver is installed 2. BSOD is triggered (I tried NMI and NotMyFault) 3. VM comes up with the driver missing It's not limited to PVPanic and it's also not limited to driver installs. If I disable a USB controller in Device Manager, for example, and then trigger BSOD without reboot, the VM comes back up with the controller enabled. Adding a reboot between the two fixes it. So it looks like some driver metadata is not preserved. Perhaps the caching policy for registry writes lets them stay in memory for a long time and they have simply not been flushed when we crash the system. Before we go any further, can you please double-check that you can really hit this *with* a reboot after the driver is installed? Thanks! (In reply to Ladi Prosek from comment #3) > I can reproduce the driver disappearing, but only if I don't reboot between > the driver install and the BSOD. > > 1. Driver is installed > 2. BSOD is triggered (I tried NMI and NotMyFault) > 3. VM comes up with the driver missing > > It's not limited to PVPanic and it's also not limited to driver installs. If > I disable a USB controller in Device Manager, for example, and then trigger > BSOD without reboot, the VM comes back up with the controller enabled. > Adding a reboot between the two fixes it. > > So it looks like some driver metadata is not preserved. Perhaps the caching > policy for registry writes lets them stay in memory for a long time and they > have simply not been flushed when we crash the system. > > Before we go any further, can you please double-check that you can really > hit this *with* a reboot after the driver is installed? Thanks! Ladi hi, Sorry to reply so late. I tried ten times with a reboot after pvpanic driver is installed,but did not hit this issue. BR, xiagao (In reply to xiagao from comment #5) > (In reply to Ladi Prosek from comment #3) > > I can reproduce the driver disappearing, but only if I don't reboot between > > the driver install and the BSOD. > > > > 1. Driver is installed > > 2. BSOD is triggered (I tried NMI and NotMyFault) > > 3. VM comes up with the driver missing > > > > It's not limited to PVPanic and it's also not limited to driver installs. If > > I disable a USB controller in Device Manager, for example, and then trigger > > BSOD without reboot, the VM comes back up with the controller enabled. > > Adding a reboot between the two fixes it. > > > > So it looks like some driver metadata is not preserved. Perhaps the caching > > policy for registry writes lets them stay in memory for a long time and they > > have simply not been flushed when we crash the system. > > > > Before we go any further, can you please double-check that you can really > > hit this *with* a reboot after the driver is installed? Thanks! > > Ladi hi, > Sorry to reply so late. > I tried ten times with a reboot after pvpanic driver is installed,but did > not hit this issue. No worries and thank you for doing that. Based on this information I'd say this BZ should be closed. While it is unfortunate that a driver install may not stick after a crash (even more so for a driver which is primarily concerned with crashing), this is not specific to our drivers and would probably qualify as a Windows issue. I wonder what happens when virtio-win drivers are installed by the official RHV guest installer (together with the guest agents and such). If we observe the same behavior, it might be worth looking for workarounds. (In reply to Ladi Prosek from comment #6) > > No worries and thank you for doing that. Based on this information I'd say > this BZ should be closed. > > While it is unfortunate that a driver install may not stick after a crash > (even more so for a driver which is primarily concerned with crashing), this > is not specific to our drivers and would probably qualify as a Windows issue. > > I wonder what happens when virtio-win drivers are installed by the official > RHV guest installer (together with the guest agents and such). If we observe > the same behavior, it might be worth looking for workarounds. I tried with rhev-guest-tools and got the same behavior sometimes. Pkg version: rhev-guest-tools-iso-4.1-5 Steps: 1. Driver is installed (without reboot) 2. BSOD is triggered (I tried NotMyFault) 3. VM comes up with the driver missing, vioser, netkvm and vioscsi hit the issue. How reproducible: netkvm:3/3 vioser:2/3 vioscsi:1/1 balloon:1/2 Looks like there's a support article on this: https://support.microsoft.com/en-us/help/2784761 "... A newly installed driver may no longer appear to be installed, and will need to be reinstalled" RegFlushKey is the recommended workaround. I wonder if rhev-guest-tools shouldn't be calling RegFlushKey(HKEY_LOCAL_MACHINE) then after the installer is done installing drivers. It would be a small reliability improvement in exchange for a small perf hit. Lev, what do you think? (In reply to Ladi Prosek from comment #8) > Looks like there's a support article on this: > > https://support.microsoft.com/en-us/help/2784761 > > "... A newly installed driver may no longer appear to be installed, and will > need to be reinstalled" > > RegFlushKey is the recommended workaround. I wonder if rhev-guest-tools > shouldn't be calling RegFlushKey(HKEY_LOCAL_MACHINE) then after the > installer is done installing drivers. It would be a small reliability > improvement in exchange for a small perf hit. > > Lev, what do you think? Personally I think that it's a low priority issue, but please open an RFE and we'll check what we can do about it, prioritize etc. I have opened bug 1500268 to track the RFE. Apologies for the delay. Adding TestOnly as there doesn't appear to be anything to do about this in the driver itself. (In reply to Ladi Prosek from comment #3) > I can reproduce the driver disappearing, but only if I don't reboot between > the driver install and the BSOD. > > 1. Driver is installed > 2. BSOD is triggered (I tried NMI and NotMyFault) > 3. VM comes up with the driver missing > > It's not limited to PVPanic and it's also not limited to driver installs. If > I disable a USB controller in Device Manager, for example, and then trigger > BSOD without reboot, the VM comes back up with the controller enabled. > Adding a reboot between the two fixes it. > > So it looks like some driver metadata is not preserved. Perhaps the caching > policy for registry writes lets them stay in memory for a long time and they > have simply not been flushed when we crash the system. It is not the question of cache. This is a "standard" Windows behaviour (I know it happens on physical machines as well). If there is crash after drivers are installed (without reboot) - Windows will remove this driver installation. > > Before we go any further, can you please double-check that you can really > hit this *with* a reboot after the driver is installed? Thanks! (In reply to Yan Vugenfirer from comment #12) > It is not the question of cache. This is a "standard" Windows behaviour (I > know it happens on physical machines as well). If there is crash after > drivers are installed (without reboot) - Windows will remove this driver > installation. RegFlushKey really fixes it for me. I've just re-checked, with both a newly installed driver and by disabling an already installed driver. In both cases the changes were preserved if I called RegFlushKey before crashing the system. This was on Windows Server 2016 in case it makes a difference. (In reply to Ladi Prosek from comment #13) > (In reply to Yan Vugenfirer from comment #12) > > It is not the question of cache. This is a "standard" Windows behaviour (I > > know it happens on physical machines as well). If there is crash after > > drivers are installed (without reboot) - Windows will remove this driver > > installation. > > RegFlushKey really fixes it for me. I've just re-checked, with both a newly > installed driver and by disabling an already installed driver. In both cases > the changes were preserved if I called RegFlushKey before crashing the > system. > > This was on Windows Server 2016 in case it makes a difference. In this case, the call to RegFlushKey can be added to guest tools installer. (In reply to Yan Vugenfirer from comment #14) > (In reply to Ladi Prosek from comment #13) > > (In reply to Yan Vugenfirer from comment #12) > > > It is not the question of cache. This is a "standard" Windows behaviour (I > > > know it happens on physical machines as well). If there is crash after > > > drivers are installed (without reboot) - Windows will remove this driver > > > installation. > > > > RegFlushKey really fixes it for me. I've just re-checked, with both a newly > > installed driver and by disabling an already installed driver. In both cases > > the changes were preserved if I called RegFlushKey before crashing the > > system. > > > > This was on Windows Server 2016 in case it makes a difference. > > In this case, the call to RegFlushKey can be added to guest tools installer. Yup, bug 1500268 tracks the feature request. Try 10 times with rhev-guest-tools-iso-4.1-5 version, can't reproduce this issue. Try 10 times with rhv-guest-tools-iso-4.2-2 version, did not hit it,either. Steps: 1. Driver is installed with RHEV-toolsSetup (without reboot) 2. BSOD is triggered (I tried NotMyFault) 3. Check all drivers. They are not missing. In Comment 7 ,drivers are not installed with "RHEV-toolsSetup" of rhev-guest-tools-iso, they are installed in device manager with "Drivers\xxx" of rhev-guest-tools-iso,such as Drivers\Balloon. Hi Amnon, According to comment#17 and comment#18, this issue could not be reproduced via guest tools installer. I'd like to close this bug as not-a-bug if you agree, as it's not drivers' issues. (In reply to lijin from comment #19) > Hi Amnon, > > According to comment#17 and comment#18, this issue could not be reproduced > via guest tools installer. > > I'd like to close this bug as not-a-bug if you agree, as it's not drivers' > issues. Yes, I agree |
Description of problem: Driver is uninstalled after triggering BSOD. Version-Release number of selected component (if applicable): kernel-3.10.0-668.el7.x86_64 qemu-kvm-rhev-2.9.0-5.el7.x86_64 virtio-win-prewhql-0.1-137 How reproducible: 2/7 Steps to Reproduce: 1.Start VM with pvpanic and install virtio-pvpanic driver. CLI:...-device pvpanic,id=pvpanic0,ioport=0x050 -no-shutdown 2.Configure regisiter for trigger BSoD by using NMI and reboot guest. 3.Generate BSoD manually by NMI. {"execute":"inject-nmi"} 4.Monitor QMP monitor started in step3 when process of dump collecting to 100% 5.check the guest status 6.boot guest again and check driver info in device manager. Actual results: after step 4, qmp receive nothing. after step 5, {"return": {"status": "guest-panicked", "singlestep": false, "running": true}} after step 6, pvpanic driver is uninstalled. Expected results: after step 4, QMP should receive following event:{"timestamp": {"seconds": 1432178991, "microseconds": 216984}, "event": "GUEST_PANICKED", "data": {"action": "pause"}} {"timestamp": {"seconds": 1432178991, "microseconds": 217446}, "event": "STOP"} after step 5, {"return": {"status": "guest-panicked", "singlestep": false, "running": false}} after step 6, pvpanic driver is installed.