Bug 1524837

Summary: virtio input device still in the live xml after hotunplug from the guest with q35 machine type
Product: Red Hat Enterprise Linux 7 Reporter: yafu <yafu>
Component: libvirtAssignee: Ján Tomko <jtomko>
Status: CLOSED ERRATA QA Contact: Meina Li <meili>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.5CC: dyuan, fjin, jdenemar, lmen, lmiksik, rbalakri, salmy, xuzhang, zpeng
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: libvirt-3.9.0-7.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-04-10 11:02:08 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description yafu 2017-12-12 08:45:03 UTC
Description of problem:
virtio input device still in the live xml after hotunplug from the guest with q35 machine type

Version-Release number of selected component (if applicable):
libvirt-3.9.0-5.el7.x86_64
qemu-kvm-rhev-2.10.0-12.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1.Start a guest with q35 machine type and passthrough input device:
#virsh dumpxml ovmf
 <os>
    <type arch='x86_64' machine='pc-q35-rhel7.5.0'>hvm</type>
    <boot dev='hd'/>
  </os>
...
 <input type='passthrough' bus='virtio'>
      <source evdev='/dev/input/event4'/>
      <alias name='input3'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
    </input>


2.Prepare the input device xml:
#cat input.xml
 <input type='passthrough' bus='virtio'>
      <source evdev='/dev/input/event4'/>
      <alias name='input3'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
    </input>

3.Hotunplug the input device from guest:
#virsh detach-device ovmf input.xml
Device detached successfully

4.Check the input device in the live xml:
#virsh dumpxml ovmf | grep -A5 passthrough
 <input type='passthrough' bus='virtio'>
      <source evdev='/dev/input/event4'/>
      <alias name='input3'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
    </input>

5.Do mangedsave and restore:
#virsh managedsave iommu1
Domain ovmf state saved by libvirt

#virsh start iommu1
error: Failed to start domain ovmf
error: internal error: qemu unexpectedly closed the monitor: 2017-12-12T07:09:37.170761Z qemu-kvm: State blocked by non-migratable device '0000:00:01.1:00.0/virtio-input-host'
2017-12-12T07:09:37.170806Z qemu-kvm: load of migration failed: Invalid argument

Actual results:
virtio input device still in the live xml after hotunplug from the guest with q35 machine type

Expected results:
virtio input device should be deleted in the live xml after hotunplug form the guest with q35 machine type

Additional info:
1.It works well with i440fx machine type.

2.Could see qemu delete the input device successfully in the libvirtd log
#cat /var/log/libvirt/libvirtd.log | grep -i del
2017-12-12 08:20:52.691+0000: 1551: info : qemuMonitorIOProcess:439 : QEMU_MONITOR_IO_PROCESS: mon=0x7fb470042250 buf={"timestamp": {"seconds": 1513066852, "microseconds": 691055}, "event": "DEVICE_DELETED", "data": {"path": "/machine/peripheral/input3/virtio-backend"}}
2017-12-12 08:20:52.691+0000: 1551: debug : qemuMonitorJSONIOProcessLine:193 : Line [{"timestamp": {"seconds": 1513066852, "microseconds": 691055}, "event": "DEVICE_DELETED", "data": {"path": "/machine/peripheral/input3/virtio-backend"}}]
2017-12-12 08:20:52.691+0000: 1551: debug : virJSONValueFromString:1723 : string={"timestamp": {"seconds": 1513066852, "microseconds": 691055}, "event": "DEVICE_DELETED", "data": {"path": "/machine/peripheral/input3/virtio-backend"}}
2017-12-12 08:20:52.691+0000: 1551: info : qemuMonitorJSONIOProcessLine:208 : QEMU_MONITOR_RECV_EVENT: mon=0x7fb470042250 event={"timestamp": {"seconds": 1513066852, "microseconds": 691055}, "event": "DEVICE_DELETED", "data": {"path": "/machine/peripheral/input3/virtio-backend"}}
2017-12-12 08:20:52.691+0000: 1551: debug : qemuMonitorEmitEvent:1327 : mon=0x7fb470042250 event=DEVICE_DELETED
2017-12-12 08:20:52.691+0000: 1551: debug : qemuMonitorJSONIOProcessEvent:179 : handle DEVICE_DELETED handler=0x7fb456c2d240 data=0x5625093a3a80
2017-12-12 08:20:52.691+0000: 1551: debug : qemuMonitorJSONHandleDeviceDeleted:999 : missing device in device deleted event
2017-12-12 08:20:52.692+0000: 1551: info : qemuMonitorIOProcess:439 : QEMU_MONITOR_IO_PROCESS: mon=0x7fb470042250 buf={"timestamp": {"seconds": 1513066852, "microseconds": 692470}, "event": "DEVICE_DELETED", "data": {"device": "input3", "path": "/machine/peripheral/input3"}}
2017-12-12 08:20:52.692+0000: 1551: debug : qemuMonitorJSONIOProcessLine:193 : Line [{"timestamp": {"seconds": 1513066852, "microseconds": 692470}, "event": "DEVICE_DELETED", "data": {"device": "input3", "path": "/machine/peripheral/input3"}}]
2017-12-12 08:20:52.692+0000: 1551: debug : virJSONValueFromString:1723 : string={"timestamp": {"seconds": 1513066852, "microseconds": 692470}, "event": "DEVICE_DELETED", "data": {"device": "input3", "path": "/machine/peripheral/input3"}}
2017-12-12 08:20:52.692+0000: 1551: info : qemuMonitorJSONIOProcessLine:208 : QEMU_MONITOR_RECV_EVENT: mon=0x7fb470042250 event={"timestamp": {"seconds": 1513066852, "microseconds": 692470}, "event": "DEVICE_DELETED", "data": {"device": "input3", "path": "/machine/peripheral/input3"}}
2017-12-12 08:20:52.692+0000: 1551: debug : qemuMonitorEmitEvent:1327 : mon=0x7fb470042250 event=DEVICE_DELETED
2017-12-12 08:20:52.692+0000: 1551: debug : qemuMonitorJSONIOProcessEvent:179 : handle DEVICE_DELETED handler=0x7fb456c2d240 data=0x562509396d30
2017-12-12 08:20:52.692+0000: 1551: debug : qemuMonitorEmitDeviceDeleted:1546 : mon=0x7fb470042250
2017-12-12 08:20:52.692+0000: 1551: debug : qemuProcessHandleDeviceDeleted:1381 : Device input3 removed from domain 0x7fb43c2f9d60 ovmf
2017-12-12 08:20:52.692+0000: 3239: debug : processDeviceDeletedEvent:4321 : Removing device input3 from domain 0x7fb43c2f9d60 ovmf

Comment 2 Ján Tomko 2017-12-14 09:52:47 UTC
The part removing the device if the DEVICE_DELETED event is received after the API ends was not implemented:
https://www.redhat.com/archives/libvir-list/2017-December/msg00505.html

I have no idea why it takes longer on a q35 machine.

Comment 3 Ján Tomko 2017-12-14 15:45:22 UTC
Pushed upstream as:
commit 8d51042ef873d2a868d505f0eff32346ba041a95
Author:     Ján Tomko <jtomko>
CommitDate: 2017-12-14 16:36:03 +0100

    qemu: remove input device after receiving the event
    
    Also call qemuDomainRemoveInputDevice if we receive the
    event after the Detach API ends.
    
    Commit 67486bb failed to include this.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1524837
    Signed-off-by: Ján Tomko <jtomko>
    Reviewed-by: Erik Skultety <eskultet>

git describe: v3.10.0-71-g8d51042ef

Comment 6 Meina Li 2018-01-15 09:26:42 UTC
Verified on libvirt-3.9.0-7.el7.x86_64 and qemu-kvm-rhev-2.10.0-15.el7.x86_64.

Scenario 1: hot-unplug after cold-plug

1.Start a guest with q35 machine type and passthrough input device:
#virsh dumpxml ovmf
 <os>
    <type arch='x86_64' machine='pc-q35-rhel7.5.0'>hvm</type>
    <boot dev='hd'/>
  </os>
...
 <input type='passthrough' bus='virtio'>
      <source evdev='/dev/input/event4'/>
      <alias name='input3'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </input>


2.Prepare the input device xml:
#cat input.xml
 <input type='passthrough' bus='virtio'>
      <source evdev='/dev/input/event4'/>
      <alias name='input3'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </input>

3.Hot-unplug the input device from guest:
#virsh detach-device rhel7 input.xml
Device detached successfully
# virsh dumpxml lmn | grep input -a4
...
    <input type='tablet' bus='usb'>
      <alias name='input0'/>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'>
      <alias name='input1'/>
    </input>
    <input type='keyboard' bus='ps2'>
      <alias name='input2'/>
    </input>
    <graphics type='spice' port='5900' autoport='yes' listen='127.0.0.1'>
 ...

4.Do mangedsave and restore:
#virsh managedsave rhel7
Domain rhel7 state saved by libvirt
#virsh start rhel7
Domain rhel7 started
# virsh dumpxml lmn | grep input -a4
...
    <input type='tablet' bus='usb'>
      <alias name='input0'/>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'>
      <alias name='input1'/>
    </input>
    <input type='keyboard' bus='ps2'>
      <alias name='input2'/>
    </input>
    <graphics type='spice' port='5900' autoport='yes' listen='127.0.0.1'>
 ...

Scenario 2: hot-unplug after hotplug

1.Start a guest with q35 machine type.

2. Prepare the input device xml:
#cat input.xml
 <input type='passthrough' bus='virtio'>
      <source evdev='/dev/input/event4'/>
      <alias name='input3'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </input>

3. Hotplug the input device.
# virsh attach-device lmn input.xml 
Device attached successfully
# virsh dumpxml rhel7 | grep passthrough -a4
    </input>
    <input type='keyboard' bus='ps2'>
      <alias name='input2'/>
    </input>
    <input type='passthrough' bus='virtio'>
      <source evdev='/dev/input/event4'/>
      <alias name='input3'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </input>

4. Hot-unplug the input device from guest:
#virsh detach-device rhel7 input.xml
Device detached successfully
# virsh dumpxml lmn | grep input -a4
...
    <input type='tablet' bus='usb'>
      <alias name='input0'/>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'>
      <alias name='input1'/>
    </input>
    <input type='keyboard' bus='ps2'>
      <alias name='input2'/>
    </input>
    <graphics type='spice' port='5900' autoport='yes' listen='127.0.0.1'>
 ...

5. Do mangedsave and restore:
#virsh managedsave rhel7
Domain rhel7 state saved by libvirt
#virsh start rhel7
Domain rhel7 started
# virsh dumpxml lmn | grep input -a4
...
    <input type='tablet' bus='usb'>
      <alias name='input0'/>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'>
      <alias name='input1'/>
    </input>
    <input type='keyboard' bus='ps2'>
      <alias name='input2'/>
    </input>
    <graphics type='spice' port='5900' autoport='yes' listen='127.0.0.1'>
 ...

Both test on seabios and ovmf, and the result is expected, move it to be verified.

Comment 10 errata-xmlrpc 2018-04-10 11:02:08 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:0704