+++ This bug was initially created as a clone of Bug #1965140 +++

Description of problem:
RFE: Support to define automatically start mdev device after reboot by 'virsh nodedev-define'

Version-Release number of selected component (if applicable):
libvirt-daemon-7.3.0-1.module+el8.5.0+11004+f4810536.x86_64

How reproducible:
100%

Steps to Reproduce:
1.mdevctl can define mdev device automatically start after reboot:
mdevctl define --uuid=8e21b099-b7a0-4c79-ad9e-6744362e2bee --parent=0000:84:00.0 --type=nvidia-12 -a

2.#mdevctl list -d
8e21b099-b7a0-4c79-ad9e-6744362e2bee 0000:84:00.0 nvidia-12 auto

3.

Actual results:


Expected results:
Libvirt should also support define mdev device automatically start after reboot, for now, mdev device defined by 'virsh nodedev-define' is the inactive state after reboot.

Additional info:

--- Additional comment from RHEL Program Management on 2021-05-27 00:45:08 UTC ---

The keyword FutureFeature has been added. If this bug is not a FutureFeature, please remove from the Summary field any strings containing "RFE, rfe, FutureFeature, FEAT, Feat, feat".

--- Additional comment from Jonathon Jongsma on 2021-05-27 13:20:21 UTC ---

In progress

--- Additional comment from  on 2021-06-01 11:08:25 UTC ---

I think it's worth mentioning that the documentation around this seems to need some help with, should we open another BZ for this?

I believe there's no documentation available about nodedev-define/undefine.
- nothing on manpage https://gitlab.com/libvirt/libvirt/-/blob/master/docs/manpages/virsh.rst#nodedev-commands
- nothing on official docs https://gitlab.com/libvirt/libvirt/-/blob/master/docs/formatnode.html.in

Actually, the official docs seem to explicitly mention that only nodedev-list and nodedev-dumpxml are supported for mdev types:
https://gitlab.com/libvirt/libvirt/-/blob/master/docs/drvnodedev.html.in#L206

"The support of mediated device's framework in libvirt's node device driver covers the following features:

list available mediated devices on the host (Since 3.4.0)
display device details (Since 3.4.0)"

The documentation then goes on to describe the CCW manual mediated device setup. However, not all steps are necessary anymore, the following snippet will define / start a device after the css device has been bound (directly or via driver override) to vfio_ccw:
# cat mdev.xml
<device>
  <name>mdev_c189ac13_b340_4caa_9c73_e18beb671df0</name>
  <path>/sys/devices/css0/0.0.26aa/c189ac13-b340-4caa-9c73-e18beb671df0</path>
  <parent>css_0_0_26aa</parent>
  <driver>
    <name>vfio_mdev</name>
  </driver>
  <capability type='mdev'>
    <type id='vfio_ccw-io'/>
    <uuid>c189ac13-b340-4caa-9c73-e18beb671df0</uuid>
    <iommuGroup number='0'/>
  </capability>
</device>
# virsh nodedev-list |grep mdev
(empty)
# virsh nodedev-define mdev.xml
Node device 'mdev_c189ac13_b340_4caa_9c73_e18beb671df0' defined from 'mdev.xml'
# virsh nodedev-list --inactive
mdev_c189ac13_b340_4caa_9c73_e18beb671df0
# virsh nodedev-start mdev_c189ac13_b340_4caa_9c73_e18beb671df0
Device mdev_c189ac13_b340_4caa_9c73_e18beb671df0 started
# virsh nodedev-list |grep mdev
mdev_c189ac13_b340_4caa_9c73_e18beb671df0

--- Additional comment from yafu on 2021-06-15 07:07:08 UTC ---

(In reply to smitterl from comment #3)
> I think it's worth mentioning that the documentation around this seems to
> need some help with, should we open another BZ for this?
> 
> I believe there's no documentation available about nodedev-define/undefine.
> - nothing on manpage
> https://gitlab.com/libvirt/libvirt/-/blob/master/docs/manpages/virsh.
> rst#nodedev-commands
> - nothing on official docs
> https://gitlab.com/libvirt/libvirt/-/blob/master/docs/formatnode.html.in
> 
> Actually, the official docs seem to explicitly mention that only
> nodedev-list and nodedev-dumpxml are supported for mdev types:
> https://gitlab.com/libvirt/libvirt/-/blob/master/docs/drvnodedev.html.in#L206
> 
> "The support of mediated device's framework in libvirt's node device driver
> covers the following features:
> 
> list available mediated devices on the host (Since 3.4.0)
> display device details (Since 3.4.0)"
> 
> The documentation then goes on to describe the CCW manual mediated device
> setup. However, not all steps are necessary anymore, the following snippet
> will define / start a device after the css device has been bound (directly
> or via driver override) to vfio_ccw:
> # cat mdev.xml
> <device>
>   <name>mdev_c189ac13_b340_4caa_9c73_e18beb671df0</name>
>  
> <path>/sys/devices/css0/0.0.26aa/c189ac13-b340-4caa-9c73-e18beb671df0</path>
>   <parent>css_0_0_26aa</parent>
>   <driver>
>     <name>vfio_mdev</name>
>   </driver>
>   <capability type='mdev'>
>     <type id='vfio_ccw-io'/>
>     <uuid>c189ac13-b340-4caa-9c73-e18beb671df0</uuid>
>     <iommuGroup number='0'/>
>   </capability>
> </device>
> # virsh nodedev-list |grep mdev
> (empty)
> # virsh nodedev-define mdev.xml
> Node device 'mdev_c189ac13_b340_4caa_9c73_e18beb671df0' defined from
> 'mdev.xml'
> # virsh nodedev-list --inactive
> mdev_c189ac13_b340_4caa_9c73_e18beb671df0
> # virsh nodedev-start mdev_c189ac13_b340_4caa_9c73_e18beb671df0
> Device mdev_c189ac13_b340_4caa_9c73_e18beb671df0 started
> # virsh nodedev-list |grep mdev
> mdev_c189ac13_b340_4caa_9c73_e18beb671df0

Hi Sebastian, 

There is existing bug for the man page:
Bug 1964223 - No info in 'man virsh' about 'virsh nodedev-define/nodedev-undefine', 'virsh nodedev-start', 'virsh nodedev-list --inactive/--all'

--- Additional comment from John Ferlan on 2021-08-10 15:18:43 UTC ---

There's been no recent update, moving from 8.5.0 to the backlog to be re-evaluated during 8.6.0 planning