Bug 1449012
| Summary: | "error -11" in guest dmesg when boot VM with some usb devices | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | yduan | ||||||||
| Component: | qemu-kvm-rhev | Assignee: | Igor Mammedov <imammedo> | ||||||||
| Status: | CLOSED NOTABUG | QA Contact: | hachen <hachen> | ||||||||
| Severity: | medium | Docs Contact: | |||||||||
| Priority: | medium | ||||||||||
| Version: | 7.4 | CC: | aliang, chayang, coli, hhuang, jinzhao, juzhang, knoel, kraxel, michen, qzhang, virt-maint, xfu, xuma, yduan, yuhuang | ||||||||
| Target Milestone: | rc | ||||||||||
| Target Release: | --- | ||||||||||
| Hardware: | x86_64 | ||||||||||
| OS: | Linux | ||||||||||
| Whiteboard: | |||||||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||||||
| Doc Text: | Story Points: | --- | |||||||||
| Clone Of: | Environment: | ||||||||||
| Last Closed: | 2017-06-14 09:58:20 UTC | Type: | Bug | ||||||||
| Regression: | --- | Mount Type: | --- | ||||||||
| Documentation: | --- | CRM: | |||||||||
| Verified Versions: | Category: | --- | |||||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||||
| Embargoed: | |||||||||||
| Attachments: |
|
||||||||||
Created attachment 1277203 [details]
dmesg
It is likely more about usb device other than pc-dimm. When boot guest with 20 pc-dimms and no usb device, didn't hit the issue. # /usr/libexec/qemu-kvm -m 4G,slots=256,maxmem=50G \ -drive file=rhel74-1-1.qcow2,if=none,format=qcow2,id=drive0 -device virtio-blk-pci,id=blk0,drive=drive0 \ -monitor stdio -vnc :0 \ -netdev tap,id=tap0 -device virtio-net-pci,netdev=tap0,id=net0 \ -numa node -monitor unix:/tmp/monitor3,server,nowait \ `for i in $(seq 20); do echo -n "-object memory-backend-ram,id=mem$i,size=128M -device pc-dimm,id=dimm$i,memdev=mem$i,slot=$i " ; done` No error in guest dmesg. I don't think we should only focus on usb device. I retested this case: 1. It is not reproducible when either usb-tablet device or pc-dimm device is in the command line. 2. I met this problem only when boot VM with both usb-tablet device and some pc-dimm devices. So I think it may be a problem about interaction between usb-tablet and pc-dimm devices. Hmm, doesn't reproduce here. Can you retest with latest qemu-kvm-rhev build? Hi Gerd, This problem can be reproduced with latest qemu-kvm-rhev-2.9.0-9.el7.x86_64. I met this problem when boot a pre-installed guest. When I newly install a guest using commands as attachment, there is no any error message. Then, I add more pc-dimm device in the command line, for example, " -object memory-backend-ram,id=mem18,size=128M -device pci-dimm,id=dimm18,memdev=mem18,slot=18 -object memory-backend-ram,id=mem19,size=128M -device pci-dimm,id=dimm19,memdev=mem19,slot=19 -object memory-backend-ram,id=mem20,size=128M -device pci-dimm,id=dimm20,memdev=mem20,slot=20", boot the guest and error info appears again. Thanks, yduan Additional info: "dmesg | grep error" in guest with latest qemu-kvm-rhev-2.9.0-9.el7.x86_64: [ 8.041113] usb 1-1: device descriptor read/64, error -11 [ 8.245134] usb 1-1: device descriptor read/64, error -11 [ 8.552121] usb 1-1: device descriptor read/64, error -11 [ 8.756121] usb 1-1: device descriptor read/64, error -11 [ 9.363143] usb 1-1: device not accepting address 2, error -11 [ 9.868118] usb 1-1: device not accepting address 2, error -11 [ 10.098109] usb 1-1: device descriptor read/64, error -11 [ 10.303131] usb 1-1: device descriptor read/64, error -11 [ 10.611118] usb 1-1: device descriptor read/64, error -11 [ 10.815158] usb 1-1: device descriptor read/64, error -11 [ 11.421124] usb 1-1: device not accepting address 5, error -11 [ 11.926124] usb 1-1: device not accepting address 6, error -11 Whole dmesg info is attached as "dmesg9". Created attachment 1286281 [details]
dmesg9
From the full log: [ 10.507108] usb 1-1: new high-speed USB device number 4 using ehci-pci [ 10.508123] nommu_map_single: overflow 18ede6150+8 of device mask ffffffff [ 10.508898] nommu_map_single: overflow 18ede6150+8 of device mask ffffffff [ 10.508900] nommu_map_single: overflow 18ede6150+8 of device mask ffffffff [ 10.611118] usb 1-1: device descriptor read/64, error -11 [ 10.712109] nommu_map_single: overflow 18ede6218+8 of device mask ffffffff [ 10.712948] nommu_map_single: overflow 18ede6218+8 of device mask ffffffff [ 10.713492] nommu_map_single: overflow 18ede6218+8 of device mask ffffffff [ 10.815158] usb 1-1: device descriptor read/64, error -11 EHCI supports 32bit DMA only, and for some reason the linux kernel thinks it is fine to allocate memory from the DIMMs (which are mapped above 4G and thus are unreachable for 32bit DMA, at least without iommu). The issue in comment 0 is that CLI is missing -numa option and started with 2Gb of initial RAM. For Linux to be happy one has to start VM with 4Gb RAM or more or add -numa option. Libvirt adds -numa option when memory hotplug is used /pc-dimm/ Current upstream guest kernel behaviour: For 32-bit drivers in kernel to work when there is memory above 4Gb, kernel needs to allocate SWIOTLB. However if VM is started with all memory below 4Gb, kernel doesn't allocate SWIOTLB by default which breaks 32-bit drivers if there is hotpluggble RAM (which is above 4Gb range on QEMU). Possible solutions: 1. to make guest happy we need to tell it that there is/might be more memory except of initial startup memory. To do so use QEMU's -numa option, which will make QEMU generate SRAT ACPI table for guest, where max possible address will be specified and as result guest kernel will know that it needs to allocate SWIOTLB even if there is only 2Gb of initial memory. 2. use q35 machine type with iommu enabled, that way guest will use emulated iommu to remap DMA operations instead of default 'nommu'. So closing it as not a bug. |
Created attachment 1277202 [details] Command Description of problem: "error -11" in guest dmesg when boot VM with some pc-dimm devices. Version-Release number of selected component (if applicable): Host: # rpm -q qemu-kvm-rhev qemu-kvm-rhev-2.9.0-3.el7.x86_64 # uname -r 3.10.0-663.el7.x86_64 Guest: 3.10.0-663.el7.x86_64 How reproducible: 100% Steps to Reproduce: 1.Boot a guest with some pc-dimm devices: /usr/libexec/qemu-kvm \ -machine pc \ -m 2048,slots=256,maxmem=64G \ -smp 2,maxcpus=2,sockets=1,cores=2,threads=1 \ -cpu SandyBridge,enforce \ ...... -device usb-ehci,id=usb1 \ -device usb-tablet,id=usb-tablet1 \ ...... -object memory-backend-ram,id=mem0,size=128M \ -device pc-dimm,id=dimm0,memdev=mem0,slot=0 \ ...... -object memory-backend-ram,id=mem17,size=128M \ -device pc-dimm,id=dimm17,memdev=mem17,slot=17 \ Whole CMD as attachment. 2."dmesg | grep error" in guest Actual results: # dmesg | grep error [ 1.486866] usb 1-1: can't read configurations, error -11 [ 1.712804] usb 1-1: device descriptor read/all, error -11 [ 1.828779] usb 1-1: device descriptor read/8, error -11 [ 1.941829] usb 1-1: device descriptor read/8, error -11 [ 2.158776] usb 1-1: device descriptor read/8, error -11 [ 2.272361] usb 1-1: device descriptor read/8, error -11 Expected results: No error message. Additional info: 1.Reproducible with qemu-kvm-rhev-2.8.0-6.el7.x86_64 2.Mouse is unusable occasionally. 3.Whole dmesg info as attachment.