Bug 2024406
Summary: | Libvirt multiqueue support for vDPA | |||
---|---|---|---|---|
Product: | Red Hat Enterprise Linux 9 | Reporter: | jason wang <jasowang> | |
Component: | libvirt | Assignee: | Jonathon Jongsma <jjongsma> | |
libvirt sub component: | Networking | QA Contact: | yalzhang <yalzhang> | |
Status: | CLOSED ERRATA | Docs Contact: | ||
Severity: | medium | |||
Priority: | medium | CC: | aadam, chhu, dzheng, egallen, jdenemar, jjongsma, jsuchane, leiyang, lulu, pezhang, pvlasin, virt-maint, xuzhang, yalzhang, yanqzhan | |
Version: | 9.0 | Keywords: | FutureFeature, Triaged | |
Target Milestone: | rc | |||
Target Release: | --- | |||
Hardware: | Unspecified | |||
OS: | Unspecified | |||
Whiteboard: | ||||
Fixed In Version: | libvirt-8.0.0-8.el9_0 | Doc Type: | If docs needed, set a value | |
Doc Text: | Story Points: | --- | ||
Clone Of: | ||||
: | 2065381 (view as bug list) | Environment: | ||
Last Closed: | 2022-05-17 12:45:52 UTC | Type: | Feature Request | |
Regression: | --- | Mount Type: | --- | |
Documentation: | --- | CRM: | ||
Verified Versions: | Category: | --- | ||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
Cloudforms Team: | --- | Target Upstream Version: | 8.2.0 | |
Embargoed: | ||||
Bug Depends On: | 2055451, 2059427, 2096180, 2133877 | |||
Bug Blocks: | 2065381 |
Description
jason wang
2021-11-18 03:02:24 UTC
*** Bug 2057748 has been marked as a duplicate of this bug. *** The additional support for mtu setting (as described in bug 2057748) can be done as part of this requirement or can be done in a separate bug. Hi Jonathon, Please help to check bug 2068999 - Guest doesn't show correct queues number after Hot plug vdap with multiple queues. I can also reproduce the bug. Thank you! Start vm with vdpa interface set with multiqueue # rpm -q libvirt qemu-kvm kernel iproute libvirt-8.0.0-7.el9_0.x86_64 qemu-kvm-6.2.0-11.el9_0.1.x86_64 kernel-5.14.0-70.4.1.el9_0.x86_64 iproute-5.15.0-2.2.el9_0.x86_64 guest kernel: 5.14.0-70.2.1.el9_0.x86_64 1. Initialize the vdpa device # ./ovs_init.sh 0000:5e:00.0 4 2. Prepare vm with device as below: <vcpu placement='static'>10</vcpu> ... <interface type='vdpa'> <mac address='00:11:22:33:44:00'/> <source dev='/dev/vhost-vdpa-0'/> <model type='virtio'/> <driver queues='8'/> <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </interface> 3. Start the vm: # virsh start rhel # virsh domiflist rhel Interface Type Source Model MAC -------------------------------------------------------------------- - vdpa /dev/vhost-vdpa-0 virtio 00:11:22:33:44:00 login the vm, and check the queues: [root@localhost ~]# ethtool -l enp1s0 Channel parameters for enp1s0: Pre-set maximums: RX: n/a TX: n/a Other: n/a Combined: 8 Current hardware settings: RX: n/a TX: n/a Other: n/a Combined: 8 4. Configure the ip address of the enp94s0f0np0_br port on the host, and the ip address of the vm interface, test the connectivity: on guest: [root@localhost ~]# ping 100.100.100.100 -c 2 PING 100.100.100.100 (100.100.100.100) 56(84) bytes of data. 64 bytes from 100.100.100.100: icmp_seq=1 ttl=64 time=0.219 ms 64 bytes from 100.100.100.100: icmp_seq=2 ttl=64 time=0.314 ms --- 100.100.100.100 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1001ms rtt min/avg/max/mdev = 0.219/0.266/0.314/0.047 ms 5. Reboot the vm, after reboot, guest works well except there is an error log in /var/log/libvirt/qemu/rhel.log, it is a qemu bug tracked bug 2055955 # cat /var/log/libvirt/qemu/rhel.log ...... 2022-03-28T07:43:26.349926Z qemu-kvm: vhost VQ 16 ring restore failed: -22: Invalid argument (22) 6. suspend and resume the vm, will also got the error messages as above(should be the same qemu bug). hotplug and unplug vdpa type interface with multiqueue 1.Start 1 vm with 1 interface: # virsh start rhel viDomain 'rhel' started # virsh domiflist rhel Interface Type Source Model MAC ------------------------------------------------------------- vnet3 network default virtio 52:54:00:9b:b1:85 2. prepare a vdpa interface xml with multiqueue: # cat interface.xml <interface type='vdpa'> <mac address='00:11:22:33:44:11'/> <source dev='/dev/vhost-vdpa-1'/> <model type='virtio'/> <driver queues='8'/> </interface> # virsh attach-device rhel interface.xml Device attached successfully 3. check on vm: [root@localhost ~]# ethtool -l enp4s0 Channel parameters for enp4s0: Pre-set maximums: RX: n/a TX: n/a Other: n/a Combined: 1 Current hardware settings: RX: n/a TX: n/a Other: n/a Combined: 1 The queue size is 1, which is not expected. This issue is reported as bug 2068999 Catch the related libvirtd.log as below: 2022-03-28 08:07:05.035+0000: 2230: info : qemuMonitorSend:914 : QEMU_MONITOR_SEND_MSG: mon=0x7f4fe0086b90 msg={"execute":"device_add","arguments":{"driver":"virtio-net-pci","netdev":"hostnet1","id":"net1","mac":"00:11:22:33:44:11","bus":"pci.4","addr":"0x0"},"id":"libvirt-416"} fd=-1 2022-03-28 08:07:05.035+0000: 4613: info : virObjectRef:402 : OBJECT_REF: obj=0x7f4fe0086b90 2022-03-28 08:07:05.035+0000: 4613: info : qemuMonitorIOWrite:402 : QEMU_MONITOR_IO_WRITE: mon=0x7f4fe0086b90 buf={"execute":"device_add","arguments":{"driver":"virtio-net-pci","netdev":"hostnet1","id":"net1","mac":"00:11:22:33:44:11","bus":"pci.4","addr":"0x0"},"id":"libvirt-416"} len=170 ret=170 errno=0 There should be elements like this in the command: "mq":true,"vectors":18 This is reported on libvirt bug 2068999 4. hotunplug works well. # virsh detach-device rhel interface.xml Device detached successfully 5. hotplug and hotunplug again, works well except the issue in step 3. As the mac address and queue size set by libvirt will not take effects, it is suggested to use the same mac address and queue size set by vdpa tool. Should we mentions it somewhere in the document? Please help to confirm, Thank you! Unfortunately it seems that we missed something in libvirt. I will propose a patch upstream very soon. Test scenarios in comment 33 and comment 34, PASS. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (new packages: libvirt), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2022:2390 |