Created attachment 1277203 [details]
dmesg

It is likely more about usb device other than pc-dimm. When boot guest with 20 pc-dimms and no usb device, didn't hit the issue. 

# /usr/libexec/qemu-kvm -m 4G,slots=256,maxmem=50G \

-drive file=rhel74-1-1.qcow2,if=none,format=qcow2,id=drive0 -device virtio-blk-pci,id=blk0,drive=drive0 \

-monitor stdio -vnc :0  \

-netdev tap,id=tap0 -device virtio-net-pci,netdev=tap0,id=net0 \

-numa node -monitor unix:/tmp/monitor3,server,nowait \

`for i in $(seq 20); do echo -n "-object memory-backend-ram,id=mem$i,size=128M -device pc-dimm,id=dimm$i,memdev=mem$i,slot=$i " ; done`


No error in guest dmesg.

I don't think we should only focus on usb device.

I retested this case:
1. It is not reproducible when either usb-tablet device or pc-dimm device is in the command line.
2. I met this problem only when boot VM with both usb-tablet device and some pc-dimm devices.

So I think it may be a problem about interaction between usb-tablet and pc-dimm devices.

Hmm, doesn't reproduce here.
Can you retest with latest qemu-kvm-rhev build?

Hi Gerd,

  This problem can be reproduced with latest qemu-kvm-rhev-2.9.0-9.el7.x86_64.

  I met this problem when boot a pre-installed guest.
  When I newly install a guest using commands as attachment, there is no any error message.
  Then, I add more pc-dimm device in the command line, for example, " -object memory-backend-ram,id=mem18,size=128M -device pci-dimm,id=dimm18,memdev=mem18,slot=18 -object memory-backend-ram,id=mem19,size=128M -device pci-dimm,id=dimm19,memdev=mem19,slot=19 -object memory-backend-ram,id=mem20,size=128M -device pci-dimm,id=dimm20,memdev=mem20,slot=20", boot the guest and error info appears again.

Thanks,
yduan

Additional info:

"dmesg | grep error" in guest with latest qemu-kvm-rhev-2.9.0-9.el7.x86_64:

[    8.041113] usb 1-1: device descriptor read/64, error -11
[    8.245134] usb 1-1: device descriptor read/64, error -11
[    8.552121] usb 1-1: device descriptor read/64, error -11
[    8.756121] usb 1-1: device descriptor read/64, error -11
[    9.363143] usb 1-1: device not accepting address 2, error -11
[    9.868118] usb 1-1: device not accepting address 2, error -11
[   10.098109] usb 1-1: device descriptor read/64, error -11
[   10.303131] usb 1-1: device descriptor read/64, error -11
[   10.611118] usb 1-1: device descriptor read/64, error -11
[   10.815158] usb 1-1: device descriptor read/64, error -11
[   11.421124] usb 1-1: device not accepting address 5, error -11
[   11.926124] usb 1-1: device not accepting address 6, error -11

Whole dmesg info is attached as "dmesg9".

Created attachment 1286281 [details]
dmesg9

From the full log:

[   10.507108] usb 1-1: new high-speed USB device number 4 using ehci-pci
[   10.508123] nommu_map_single: overflow 18ede6150+8 of device mask ffffffff
[   10.508898] nommu_map_single: overflow 18ede6150+8 of device mask ffffffff
[   10.508900] nommu_map_single: overflow 18ede6150+8 of device mask ffffffff
[   10.611118] usb 1-1: device descriptor read/64, error -11
[   10.712109] nommu_map_single: overflow 18ede6218+8 of device mask ffffffff
[   10.712948] nommu_map_single: overflow 18ede6218+8 of device mask ffffffff
[   10.713492] nommu_map_single: overflow 18ede6218+8 of device mask ffffffff
[   10.815158] usb 1-1: device descriptor read/64, error -11

EHCI supports 32bit DMA only, and for some reason the linux kernel thinks it is fine to allocate memory from the DIMMs (which are mapped above 4G and thus are unreachable for 32bit DMA, at least without iommu).

The issue in comment 0 is that  CLI is missing -numa option and started with 2Gb of initial RAM. For Linux to be happy one has to start VM with 4Gb RAM or more or
add -numa option. Libvirt adds -numa option when memory hotplug is used /pc-dimm/

Current upstream guest kernel behaviour:
For 32-bit drivers in kernel to work when there is memory above 4Gb,
kernel needs to allocate SWIOTLB. However if VM is started with all
memory below 4Gb, kernel doesn't allocate SWIOTLB by default which
breaks 32-bit drivers if there is hotpluggble RAM (which is above 4Gb range on QEMU).

Possible solutions:
 1. to make guest happy we need to tell it that there is/might be more memory
    except of initial startup memory. To do so use QEMU's -numa option, which 
    will make QEMU generate SRAT ACPI table for guest, where max possible address
    will be specified and as result guest kernel will know that it needs to
    allocate SWIOTLB even if there is only 2Gb of initial memory.
 2. use q35 machine type with iommu enabled, that way guest will use emulated
    iommu to remap DMA operations instead of default 'nommu'.

So closing it as not a bug.