Bug 1999420

I've posted patches for the first three problems here:

https://listman.redhat.com/archives/libvir-list/2021-September/msg00471.html

But the fourth one looks bogus to me. I'm failing to find a path in the code where empty capabilities list would be generated for vdpa.

Merged upstream:

7bfb102ce4 schemas: Allow <unique_id/> to be zero
0ef02498ee schemas: Provide grammar for scsi_generic capabilities
3134555955 schemas: Make <type/> for capscsi type optional

v7.7.0-240-g7bfb102ce4

@hhan can you please try whether vdpa_vdpa0 case still reproduces with latest master? As I'm saying in comment 1 I can't find a way in the source that would produce such XML.

(In reply to Michal Privoznik from comment #3)
> @hhan can you please try whether vdpa_vdpa0 case still reproduces
> with latest master? As I'm saying in comment 1 I can't find a way in the
For the vdpa0 xml in comment0, the xml is still failed to be validated, version libvirt v7.7.0-298-g55178b5c9e:
➜  ~ virt-xml-validate vdpa.xml
vdpa.xml:8: element capability: Relax-NG validity error : Invalid sequence in interleave
vdpa.xml:8: element capability: Relax-NG validity error : Expecting an element chardev, got nothing
vdpa.xml:8: element capability: Relax-NG validity error : Element capability failed to validate content
vdpa.xml fails to validate
> source that would produce such XML.
I cannot produce the vdap XML now, too. The reason is after modprobe vdpa, vhost_vdpa, vdpa_sim, there is no simulated vdpa device.
I will ask the vdpa developer for help.

BTW, the patches for scsi node devices work well

Hi Jason,
I find an issue on vdpa_sim. I have loaded vdpa, vhost_vdpa, vdpa_sim, but there is no vdpa device in my host.

Version:
kernel-5.15.0-0.rc2.18.fc36.x86_64

Steps:1
1. The kernel config:
➜  ~ grep VDPA /boot/config-`uname -r`
CONFIG_VIRTIO_VDPA=m
CONFIG_VDPA=m
CONFIG_VDPA_SIM=m
CONFIG_VDPA_SIM_NET=m
# CONFIG_VDPA_SIM_BLOCK is not set
# CONFIG_VDPA_USER is not set
CONFIG_MLX5_VDPA=y
CONFIG_MLX5_VDPA_NET=m
CONFIG_VP_VDPA=m
CONFIG_VHOST_VDPA=m

2. Following the steps of "Configuring the host" in https://www.redhat.com/en/blog/hands-vdpa-what-do-you-do-when-you-aint-got-hardware
➜  ~ modprobe vdpa
➜  ~ modprobe vhost_vdpa
➜  ~ modprobe vdpa_sim

Then check the vdpa device:
➜  ~ ls -l  /sys/bus/vdpa/drivers
total 0
drwxr-xr-x. 2 root root 0 Sep 24 08:21 vhost_vdpa
➜  ~ ls -l /sys/bus/vdpa/devices/vdpa0/
ls: cannot access '/sys/bus/vdpa/devices/vdpa0/': No such file or directory
➜  ~ ls -l /dev/ |grep vdpa

No vdpa device here.


The results of dmesg and lsmod:
➜  ~ dmesg|grep vdpa
➜  ~ lsmod|grep vdpa
vdpa_sim               16384  0
vringh                 32768  1 vdpa_sim
vhost_vdpa             24576  0
vhost                  53248  1 vhost_vdpa
vhost_iotlb            16384  4 vdpa_sim,vringh,vhost_vdpa,vhost
vdpa                   20480  2 vdpa_sim,vhost_vdpa
irqbypass              16384  2 vhost_vdpa,kvm


Could you please help with this issue? And how to produce an simulated vdpa device?

In the new version of the kernel, the simulator device is created via vdpa tool (integreated with iproute2).

You need to use that to create vdpa device:

vdpa dev add mgmtdev vdpasim_net name foo2 [1]

Thanks

[1] https://lore.kernel.org/netdev/20210105103203.82508-1-parav@nvidia.com/

Test on latest kernel and libvirt, the vdpa device can pass the validation:

# rpm -q kernel libvirt qemu-kvm
kernel-5.14.0-1.3.1.el9.x86_64
libvirt-7.6.0-2.el9.x86_64
qemu-kvm-6.0.0-12.el9.x86_64

1) For vdpa simulator:
# modprobe vdpa
# modprobe vhost_vdpa
# modprobe vdpa_sim
# modprobe vdpa_sim_net

# vdpa mgmtdev show 
vdpasim_net: 
  supported_classes net

# vdpa dev add name vdpa0 mgmtdev vdpasim_net

#  ls -l /sys/bus/vdpa/devices/vdpa0/
total 0
lrwxrwxrwx. 1 root root    0 Sep 26 08:50 driver -> ../../bus/vdpa/drivers/vhost_vdpa
drwxr-xr-x. 2 root root    0 Sep 26 08:50 power
lrwxrwxrwx. 1 root root    0 Sep 26 08:50 subsystem -> ../../bus/vdpa
-rw-r--r--. 1 root root 4096 Sep 26 08:49 uevent
drwxr-xr-x. 3 root root    0 Sep 26 08:50 vhost-vdpa-0

# virsh nodedev-list --tree
computer
  |
  +- net_lo_00_00_00_00_00_00
  +- net_vnet0_fe_54_00_aa_1c_ef
...
  +- pci_0000_d7_16_5
  +- vdpa_vdpa0

# virsh nodedev-dumpxml vdpa_vdpa0
<device>
  <name>vdpa_vdpa0</name>
  <path>/sys/devices/vdpa0</path>
  <parent>computer</parent>
  <driver>
    <name>vhost_vdpa</name>
  </driver>
  <capability type='vdpa'>
    <chardev>/dev/vhost-vdpa-0</chardev>
  </capability>
</device>

# virsh nodedev-dumpxml vdpa_vdpa0 > sim.xml

# virt-xml-validate sim.xml 
sim.xml validates

2) for vdpa device, test on host with 'Mellanox Technologies MT2892 Family [ConnectX-6 Dx]'
set up 2 vdpa device, then check by virsh commands:

# ll /dev/vhost-vdpa*
crw-------. 1 root root 510, 0 Sep 26 08:15 /dev/vhost-vdpa-0
crw-------. 1 root root 510, 1 Sep 26 08:15 /dev/vhost-vdpa-1

# virsh nodedev-list --tree 
computer
  |
...
  +- pci_0000_5d_02_0
  |   |
  |   +- pci_0000_5e_00_0
  |   |   |
  |   |   +- net_enp94s0f0np0_b8_ce_f6_4b_6f_10
  |   |   +- net_eth0_96_f2_9b_a6_23_59
  |   |   +- net_eth1_6a_e3_1d_ac_fc_41
  |   |     
  |   +- pci_0000_5e_00_1
  |   |   |
  |   |   +- net_enp94s0f1np1_b8_ce_f6_4b_6f_11
  |   |     
  |   +- pci_0000_5e_00_2
  |   |   |
  |   |   +- net_enp94s0f0v0_1e_56_20_0f_d4_f2
  |   |   +- vdpa_vdpa0
  |   |     
  |   +- pci_0000_5e_00_3
  |       |
  |       +- net_enp94s0f0v1_6a_f6_21_b2_2a_ce
  |       +- vdpa_vdpa1

# virsh nodedev-dumpxml vdpa_vdpa0 > vdpa.xml

# virsh nodedev-dumpxml vdpa_vdpa0 
<device>
  <name>vdpa_vdpa0</name>
  <path>/sys/devices/pci0000:5d/0000:5d:02.0/0000:5e:00.2/vdpa0</path>
  <parent>pci_0000_5e_00_2</parent>
  <driver>
    <name>vhost_vdpa</name>
  </driver>
  <capability type='vdpa'>
    <chardev>/dev/vhost-vdpa-0</chardev>
  </capability>
</device>

# virt-xml-validate vdpa.xml
vdpa.xml validates

Perfect, so we can leave vdpa case out. And since the other cases are now fixed upstream, I'm moving this to POST.

7bfb102ce4 schemas: Allow <unique_id/> to be zero
0ef02498ee schemas: Provide grammar for scsi_generic capabilities
3134555955 schemas: Make <type/> for capscsi type optional

v7.7.0-240-g7bfb102ce4

Verified on libvirt-7.8.0-1.el9.x86_64:
Simulate different types of scsi node devices and check validate the xmls of nodedev

for i in {0..5};do
    modprobe scsi_debug ptype=$i
    sleep 1
    for node in $(virsh nodedev-list);do virsh nodedev-dumpxml $node > /tmp/test; virt-xml-validate /tmp/test; if [ $? -ne 0 ];then virsh nodedev-dumpxml $node>$node;fi;done
    modprobe -r scsi_debug
done

No failure of validation.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (new packages: libvirt), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:2390