Bug 1518884

Summary: [vhost-user] No network access if virtio-net is configured with mq=on but guest doesn't ack MQ
Product: Red Hat Enterprise Linux 7 Reporter: Ladi Prosek <lprosek>
Component: dpdkAssignee: Open vSwitch development team <ovs-team>
Status: CLOSED CURRENTRELEASE QA Contact: Jean-Tsung Hsiao <jhsiao>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.5CC: aconole, atragler, fleitner, juzhang, lprosek, maxime.coquelin, tredaelli
Target Milestone: rcKeywords: Extras
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openvswitch-2.9.0-0.6.20171212git6625e43.el7fdb Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-09-11 14:07:47 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1522700    
Attachments:
Description Flags
Patches backport for v17.11 none

Description Ladi Prosek 2017-11-29 17:19:05 UTC 
Description of problem:
Issue reported on the ipxe-devel mailing list:
http://lists.ipxe.org/pipermail/ipxe-devel/2017-November/005886.html

iPXE is unable to transmit and receive packets through a vhost-user interface if virtio-net is configured with mq=on. It looks like DPDK simply won't consider the interface 'ready' until after all queues have been initialized.

http://dpdk.org/browse/dpdk/tree/lib/librte_vhost/vhost_user.c#n713

iPXE has a simple virtio-net driver without MQ support so it always initializes only queues 0 and 1.

Version-Release number of selected component (if applicable):
Recent upstream QEMU (commit 5e19aed)
Recent OVS (commit a7ce5b8)
DPDK 17.05.2
Recent upstream iPXE (commit b5e0b50)

How reproducible:
100%

Steps to Reproduce:
$ git clone git://git.ipxe.org/ipxe.git
$ cd ipxe/src
$ make bin/1af41000.rom DEBUG=virtio-net:2
$ ln -s bin/1af41000.rom efi-virtio.rom

Then run QEMU without changing the current directory (i.e. should
still be .../ipxe/src):

qemu-system-x86_64 \
-machine pc,accel=kvm -m 128M -boot strict=on -device cirrus-vga \
-monitor stdio \
-object memory-backend-file,id=mem,size=128M,mem-path=/dev/hugepages,share=on \
-numa node,memdev=mem \
-chardev socket,id=char1,path=/var/run/openvswitch/vhost-user0 \
-netdev type=vhost-user,id=mynet1,chardev=char1,vhostforce \
-device virtio-net-pci,mac=00:00:00:00:00:01,netdev=mynet1,bootindex=0

You'll see a bunch of "enqueuing iobuf" debug messages on the screen,
followed by at least "tx complete". Maybe also "rx complete" depending
on what /var/run/openvswitch/vhost-user0 is connected to.

Now if you enable multiqueue by replacing the last two lines with:

-netdev type=vhost-user,id=mynet1,chardev=char1,vhostforce,queues=16 \
-device virtio-net-pci,mac=00:00:00:00:00:01,netdev=mynet1,mq=on,vectors=34,bootindex=0

you'll see only "enqueuing iobuf" without any completion, indicating
that the host is not processing packets placed in the tx virtqueue by
iPXE.

Actual results:
Buffers are not "completed", no packets are transmitted or received.

Expected results:
iPXE has network connectivity.
Comment 2 Laszlo Ersek 2017-12-04 12:12:54 UTC 
(In reply to Ladi Prosek from comment #0)

> iPXE has a simple virtio-net driver without MQ support so it always
> initializes only queues 0 and 1.

Not sure if it matters, but the same applies to OVMF's VirtioNetDxe driver.
Comment 3 Maxime Coquelin 2017-12-12 10:21:50 UTC 
Hi Matteo,

I have posted a DPDK patch upstream to fix this issue.
Last revision may be found here, but is not applied upstream yet:
http://dpdk.org/ml/archives/dev/2017-December/083502.html

I will notify you once it is applied upstream.

Regards,
Maxime
Comment 4 Matteo Croce 2018-02-05 23:44:18 UTC 
Hi Maxime,

was the patch merged as 07f8db29b8833378dd506f3e197319f8b669aed9 ?

Regards,
Matteo
Comment 5 Maxime Coquelin 2018-02-06 08:32:12 UTC 
Hi Matteo,

(In reply to Matteo Croce from comment #4) 
> was the patch merged as 07f8db29b8833378dd506f3e197319f8b669aed9 ?

There are in total 4 patches to backport, two from my initial series, and another 
series fixing SPDK  support by Stefan.

Stefan series is accepted, but didn't land yet into upstream master.
It will be in next 18.02-rc3 release.

Do you want me to provide you a backport of the 4 patches?
If yes, should I attach it to the bz, or sent it to you directly by e-mail?

Cheers,
Maxime
Comment 7 Maxime Coquelin 2018-02-06 17:29:44 UTC 
Created attachment 1392231 [details]
Patches backport for v17.11

Hi Matteo,

Please find the backport for v17.11.
Actually, there are 6 patches, not 4 as I mentioned previously.

Regards,
Maxime