Bug 1175113
| Summary: | pci-bridge should behave the same when adding devices from cli or at hotplug time | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Sibiao Luo <sluo> | ||||
| Component: | qemu-kvm-rhev | Assignee: | Marcel Apfelbaum <marcel> | ||||
| Status: | CLOSED ERRATA | QA Contact: | yduan | ||||
| Severity: | low | Docs Contact: | |||||
| Priority: | low | ||||||
| Version: | 7.1 | CC: | ailan, aliang, chayang, coli, hhuang, jasowang, jinzhao, juzhang, laine, marcel, michen, mrezanin, mst, pbonzini, virt-maint, xfu, xuwei, yduan, yfu | ||||
| Target Milestone: | rc | ||||||
| Target Release: | --- | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | qemu-kvm-rhev-2.9.0-1.el7 | Doc Type: | If docs needed, set a value | ||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2017-08-01 23:27:12 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
(In reply to Sibiao Luo from comment #0) > 3.hot-plug the device to the pci-bridge specified to slot 0 > {"execute": "__com.redhat_drive_add", "arguments": {"id":"drive0", "file": > "/home/my-data-disk.raw", "format": "raw" }} > {"return": {}} > {"execute": "device_add", "arguments": {"bus": "bridge1", "driver": > "virtio-blk-pci", "drive": "drive0", "id": "disk0", "addr": "0x0" }} > {"return": {}} > {"execute": "query-block"} > {"return": [{..., {"io-status": "ok", "device": "ide1-cd0", "locked": false, > "removable": true, "tray_open": false, "type": "unknown"}, {"device": > "floppy0", "locked": false, "removable": true, "tray_open": false, "type": > "unknown"}, {"device": "sd0", "locked": false, "removable": true, > "tray_open": false, "type": "unknown"},...}]} {"return": [{...,{"io-status": "ok", "device": "drive0", "locked": false, "removable": false, "inserted": {"iops_rd": 0, "detect_zeroes": "off", "image": {"virtual-size": 10737418240, "filename": "/home/my-data-disk.raw", "format": "raw", "actual-size": 10737422336, "dirty-flag": false}, "iops_wr": 0, "ro": false, "backing_file_depth": 0, "drv": "raw", "iops": 0, "bps_wr": 0, "encrypted": false, "bps": 0, "bps_rd": 0, "file": "/home/my-data-disk.raw", "encryption_key_missing": false}, "type": "unknown"}]} In two ways to hotplug the same device to same pci_bridge, different hotplug handlers were called.
1) add device in qemu commandline:
shpc_device_hotplug_cb()
2) hotplug by device_add in monitor:
piix4_device_plug_cb()
We use standard hot plug controller to add commandline pci devices, 'slot 0' is only reserved by standard hot plug controller. After guest starts up, we use acpi to handle pci hotplug, 'slot 0' is available for acpi pci hotplug. So it's not a BUG. This is not a BUG, but the bridge should behave the same for hotplug/command line. We will have a look on 7.3 (In reply to Marcel Apfelbaum from comment #4) > This is not a BUG, but the bridge should behave the same for hotplug/command > line. Are you saying that pci-bridge should allow plugging devices into slot 0 in both cases? If so, then isn't the fact that it doesn't allow it a bug? Or are you saying that use of slot 0 on a pci-bridge should never be allowed? Either way, your two sentences seem to contradict each other. Note that libvirt only allows uses of slots 1 - 31 on a pci-bridge. This may be slightly wasteful of resources, but consistency is more important (and I must confess that after experiencing that a device couldn't be connected to slot 0 on the commandline, it never even occurred to me that it might work with an device_add command.) (In reply to Laine Stump from comment #11) > (In reply to Marcel Apfelbaum from comment #4) > > This is not a BUG, but the bridge should behave the same for hotplug/command > > line. > > Are you saying that pci-bridge should allow plugging devices into slot 0 in > both cases? No. > If so, then isn't the fact that it doesn't allow it a bug? No, slot 0 is in use during machine boot by the SHPC (hot-plug controller) > Or > are you saying that use of slot 0 on a pci-bridge should never be allowed? Yes. > Either way, your two sentences seem to contradict each other. > I am sorry if I wasn't clear. > Note that libvirt only allows uses of slots 1 - 31 on a pci-bridge. This may > be slightly wasteful of resources, but consistency is more important (and I > must confess that after experiencing that a device couldn't be connected to > slot 0 on the commandline, it never even occurred to me that it might work > with an device_add command.) Once the machine is booted, the guest firmware/OS can choose what kind of PCI hotplug will use: - if the acpi hotplug takes over all the available slots are "fair play". - if the guest OS/firmware is using the shpc controller, slot 0 is occupied. I agree the consistency is more important. Using only the slots 1-31 is an acceptable solution. Thanks, Marcel I posted a patch upstream: https://lists.gnu.org/archive/html/qemu-devel/2016-11/msg00377.html that solves the problem by disabling the shpc component for newer machine types. Please see the upstream discussion that follows the question if the shpc component is used by other archs besides x86. The answers seems to be the shpc is not needed be default, I will try to push this upstream. Thanks, Marcel I should add that (as discussed offline) even once qemu turns off shpc by default, libvirt will continue to forbid use of slot 0 on pci-bridge until it's able to query the default value of either the shpc option, or the range of usable slots for each PCI controller device, for individual machinetypes. At that point libvirt will begin taking advantage of the extra slot when appropriate. (The alternative would have been to expose yet another option to libvirt config, which I prefer to avoid since it usually just adds to user confusion) V2 posted upstream and is expected to be merged by MST soon When verifying this bug with latest qemu-kvm-rhev, hotplug virtio-blk device to pci-bridge slot 0 failed with error info "[ 127.974047] virtio-pci: probe of 0000:01:00.0 failed with error -12" in guest "dmesg". The whole dmesg info is as attachment. Version-Release number of selected component (if applicable): Host: # uname -r && rpm -q qemu-kvm-rhev 3.10.0-648.el7.x86_64 qemu-kvm-rhev-2.9.0-1.el7.x86_64 Guest: # uname -r 3.10.0-648.el7.x86_64 Hi marcel, Whether need to open a new bug? Thanks in advance! yduan Created attachment 1273522 [details]
dmesg_info
The issue seen here is actually https://bugzilla.redhat.com/show_bug.cgi?id=1434706(In reply to yduan from comment #18) > Created attachment 1273522 [details] > dmesg_info (In reply to yduan from comment #18) > Created attachment 1273522 [details] > dmesg_info The issue seen here is actually: - https://bugzilla.redhat.com/show_bug.cgi?id=1434706 No need for a new BZ. In order to check if the slot 0 is hot-pluggable, add a bridge with a device on another slot. Then it should work. For more details please see the mentioned BZ. Thanks, Marcel Reproduced with qemu-kvm-rhev-2.8.0-6.el7.x86_64. Verified with qemu-kvm-rhev-2.9.0-1.el7.x86_64 according to Comment 20. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2017:2392 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2017:2392 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2017:2392 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2017:2392 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2017:2392 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2017:2392 |
Description of problem: Launch a KVM attach with a device to the pci-bridge specified to slot 0, qemu will quit with a "Unsupported PCI slot 0 for standard hotplug controller. Valid slots are between 1 and 31." warning message. But if you hot-plug the device to the pci-bridge specified to slot 0 which can successfully. Version-Release number of selected component (if applicable): host info: # uname -r && rpm -q qemu-kvm-rhev 3.10.0-217.el7.x86_64 qemu-kvm-rhev-2.1.2-17.el7.x86_64 guest info: rhel6.6-z, 2.6.32-504.6.1.el6.x86_64 How reproducible: 100% Steps to Reproduce: 1.launch a KVM attach with a device to the pci-bridge specified to slot 0. e.g:/usr/libexec/qemu-kvm -machine type=pc,dump-guest-core=off -S -cpu SandyBridge -enable-kvm -m 4096 -smp 4,sockets=2,cores=2,threads=1 -no-kvm-pit-reinjection -device pci-bridge,bus=pci.0,id=bridge1,chassis_nr=1,addr=0x3...-drive file=/home/my-data-disk.raw,if=none,id=drive0,format=raw,cache=none,aio=native -device virtio-blk-pci,bus=bridge1,addr=0x0,drive=drive0,id=disk0 2.launch a KVM with a pci-bridge. e.g:/usr/libexec/qemu-kvm -machine type=pc,dump-guest-core=off -S -cpu SandyBridge -enable-kvm -m 4096 -smp 4,sockets=2,cores=2,threads=1 -no-kvm-pit-reinjection -device pci-bridge,bus=pci.0,id=bridge1,chassis_nr=1,addr=0x3 3.hot-plug the device to the pci-bridge specified to slot 0 {"execute": "__com.redhat_drive_add", "arguments": {"id":"drive0", "file": "/home/my-data-disk.raw", "format": "raw" }} {"return": {}} {"execute": "device_add", "arguments": {"bus": "bridge1", "driver": "virtio-blk-pci", "drive": "drive0", "id": "disk0", "addr": "0x0" }} {"return": {}} {"execute": "query-block"} {"return": [{..., {"io-status": "ok", "device": "ide1-cd0", "locked": false, "removable": true, "tray_open": false, "type": "unknown"}, {"device": "floppy0", "locked": false, "removable": true, "tray_open": false, "type": "unknown"}, {"device": "sd0", "locked": false, "removable": true, "tray_open": false, "type": "unknown"},...}]} 4. check the guest and qemu monitor. # ls -lh /dev/vd* # fdisk -l Actual results: after step 1, qemu will quit with a "Unsupported PCI slot 0 for standard hotplug controller. Valid slots are between 1 and 31." warning message. QEMU 2.1.2 monitor - type 'help' for more information (qemu) qemu-kvm: -device virtio-blk-pci,bus=bridge1,addr=0x0,drive=drive0,id=disk0: Unsupported PCI slot 0 for standard hotplug controller. Valid slots are between 1 and 31. qemu-kvm: -device virtio-blk-pci,bus=bridge1,addr=0x0,drive=drive0,id=disk0: Device 'virtio-blk-pci' could not be initialized after step 3, it can hotplug successfully. after step 4, the disk can be detected in guest correctly. Expected results: It should fail to hotplug device to the pci-bridge specified the PCI slot 0. Additional info: