Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 2107031

Summary: Cannot create node device for parent mtty via nodedev api
Product: Red Hat Enterprise Linux 9 Reporter: Guo, Zhiyi <zhguo>
Component: libvirtAssignee: Jonathon Jongsma <jjongsma>
libvirt sub component: CLI & API QA Contact: yalzhang <yalzhang>
Status: CLOSED CURRENTRELEASE Docs Contact:
Severity: medium    
Priority: low CC: jjongsma, jsuchane, pkrempa, smitterl, virt-maint, yalzhang
Version: 9.1Keywords: TestOnly, Triaged
Target Milestone: rcFlags: pm-rhel: mirror+
Target Release: ---   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-12-19 08:00:01 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Guo, Zhiyi 2022-07-14 07:31:28 UTC 
Description of problem:
Cannot create nodedev for mtty device.

Version-Release number of selected component (if applicable):
libvirt-8.5.0-1.el9.x86_64


How reproducible:
100%

Steps to Reproduce:
0. Load mtty kernel module (added in [1]).
1. # cat mtty1.xml
<device>
	<parent>mtty</parent>
	<capability type="mdev">
		<type id="mtty-1"/>
		<uuid>d36d7d0f-cf3d-4fef-bb9c-ed393954996b</uuid>
	</capability>
</device>


2. # virsh nodedev-create mtty1.xml 


Actual results:
error: Failed to create node device from mtty1.xml
error: unsupported configuration: invalid parent device 'mtty'

Expected results:
The device is created successfully and can be used (started, destroyed, undefined).

Additional info:
1. The mtty device allows us to test mediated devices without depending on real hardware.
2. Choosing 'mtty' as the parent's name on libvirt side seems the most simple and natural choice.
3. We expected it should also be listed via
# virsh nodedev-list --cap mdev_types

Which currently doesn't return the device parent although mdevctl knows about it and can create devices:
# mdevctl types
mtty
  mtty-1
    Available instances: 23
    Device API: vfio-pci
    Name: Single port serial
  mtty-2
    Available instances: 11
    Device API: vfio-pci
    Name: Dual port serial

# mdevctl create -t mtty-1 -p mtty
abccc375-f6e4-4ed0-8388-e9ac8c4dd78

[1]https://bugzilla.redhat.com/show_bug.cgi?id=2071955
Comment 1 smitterl 2022-07-14 08:03:46 UTC 
# cat mtty1.xml
<device>
	<parent>mtty</parent>
	<capability type="mdev">
		<type id="mtty-1"/>
		<uuid>d36d7d0f-cf3d-4fef-bb9c-ed393954996b</uuid>
	</capability>
</device>


# virsh nodedev-create mtty1.xml 
error: Failed to create node device from mtty1.xml
error: unsupported configuration: invalid parent device 'mtty'
Comment 2 Peter Krempa 2022-07-14 08:24:14 UTC 
So, is this not only about the query but also about the creation? If that's so the summary should mention that.
Comment 3 Jaroslav Suchanek 2022-07-14 14:31:59 UTC 
Jonathon, can you please check it? Thanks.
Comment 4 Jonathon Jongsma 2022-07-15 17:07:33 UTC 
Hmm, indeed. This is unfortunate. The reason that the device xml for creating the new mdev does not work is because the <parent>$devicename</parent> xml element expects $devicename to be the name of a libvirt node device. And the 'mtty' device is not recognized by libvirt as a node device.

The issue is that the mtty parent device doesn't really fit into any of the standard device types (e.g. PCI, USB, etc). libvirt queries udev for the list of devices on the host and only handles those devices that fit into specific types that libvirt knows how to handle. The full list of supported device types corresponds to the 'cap' argument of nodedev-list and can be found here: https://gitlab.com/libvirt/libvirt/-/blob/d531b9c724187e31bb72b5878d5c0435d3c0c33e/src/conf/node_device_conf.h#L50

If a device does not fit one of those device types, libvirt doesn't know how to handle it, so it is ignored.

When I insert the mtty kernel module (modprobe mtty), I get the following udev events:

KERNEL[6441.374935] add      /bus/mdev/drivers/mtty (drivers)
ACTION=add
DEVPATH=/bus/mdev/drivers/mtty
SUBSYSTEM=drivers
SEQNUM=5815

KERNEL[6441.374999] add      /class/mtty (class)
ACTION=add
DEVPATH=/class/mtty
SUBSYSTEM=class
SEQNUM=5816

KERNEL[6441.375036] add      /devices/virtual/mtty/mtty (mtty)
ACTION=add
DEVPATH=/devices/virtual/mtty/mtty
SUBSYSTEM=mtty
SEQNUM=5817

KERNEL[6441.375082] change   /devices/virtual/mtty/mtty (mtty)
ACTION=change
DEVPATH=/devices/virtual/mtty/mtty
SUBSYSTEM=mtty
MDEV_STATE=registered
SEQNUM=5818

KERNEL[6441.375126] add      /module/mtty (module)
ACTION=add
DEVPATH=/module/mtty
SUBSYSTEM=module
SEQNUM=5819

UDEV  [6441.386693] add      /module/mtty (module)
ACTION=add
DEVPATH=/module/mtty
SUBSYSTEM=module
SEQNUM=5819
USEC_INITIALIZED=6441379252

UDEV  [6441.386824] add      /devices/virtual/mtty/mtty (mtty)
ACTION=add
DEVPATH=/devices/virtual/mtty/mtty
SUBSYSTEM=mtty
SEQNUM=5817
USEC_INITIALIZED=6441378351

UDEV  [6441.386871] add      /bus/mdev/drivers/mtty (drivers)
ACTION=add
DEVPATH=/bus/mdev/drivers/mtty
SUBSYSTEM=drivers
SEQNUM=5815
USEC_INITIALIZED=6441377388

UDEV  [6441.386914] add      /class/mtty (class)
ACTION=add
DEVPATH=/class/mtty
SUBSYSTEM=class
SEQNUM=5816
USEC_INITIALIZED=6441377842

UDEV  [6441.392205] change   /devices/virtual/mtty/mtty (mtty)
ACTION=change
DEVPATH=/devices/virtual/mtty/mtty
SUBSYSTEM=mtty
MDEV_STATE=registered
SEQNUM=5818
USEC_INITIALIZED=6441387508



The one that is most relevant here is the one with the path /devices/virtual/mtty/mtty since that is the one that is actually the parent device for mdev support: 
ls -l /sys/class/mdev_bus/
total 0
lrwxrwxrwx. 1 root root 0 Jul 15 09:13 0000:00:02.0 -> ../../devices/pci0000:00/0000:00:02.0
lrwxrwxrwx. 1 root root 0 Jul 15 10:37 mtty -> ../../devices/virtual/mtty/mtty


But since this device is a one-of-a-kind device and not a standard device type (note the "SUBSYSTEM=mtty"), we would have to introduce support into libvirt specifically for this device type.
Comment 5 Jonathon Jongsma 2022-08-10 16:20:49 UTC 
upstream commit ab29ddfdf8774eaffbe4d6e83fd47ccb6b1c0308
Comment 6 smitterl 2022-10-20 16:05:48 UTC 
I confirmed the mediated devices can be created in RHEL 9.2. Zhiyi, Yalan, should we close this BZ with UPSTREAM, what do you think?

Version: 
libvirt-8.7.0-1.el9.s390x
kernel-5.14.0-176.el9.s390x


Details:
========
# modprobe mtty
# virsh nodedev-list
...
mtty_mtty
..
# cat nodedev*
<device>
	<parent>mtty_mtty</parent>
	<capability type="mdev">
	    <type id="mtty-1"/>
	    <uuid>d36d7d0f-cf3d-4fef-bb9c-ed393954996b</uuid>
        </capability>
</device>
<device>
	<parent>mtty_mtty</parent>
	<capability type="mdev">
	    <type id="mtty-2"/>
	    <uuid>d36d7d0f-c23d-4fef-bb9c-ed393954996b</uuid>
        </capability>
</device>
# virsh nodedev-create nodedev-mtty-1.xml 
Node device mdev_d36d7d0f_cf3d_4fef_bb9c_ed393954996b_mtty created from nodedev-mtty-1.xml
# virsh nodedev-create nodedev-mtty-2.xml 
Node device mdev_d36d7d0f_c23d_4fef_bb9c_ed393954996b_mtty created from nodedev-mtty-2.xml
# virsh nodedev-list
..
mdev_d36d7d0f_c23d_4fef_bb9c_ed393954996b_mtty
mdev_d36d7d0f_cf3d_4fef_bb9c_ed393954996b_mtty
..
# virsh nodedev-dumpxml mdev_d36d7d0f_c23d_4fef_bb9c_ed393954996b_mtty 
<device>
  <name>mdev_d36d7d0f_c23d_4fef_bb9c_ed393954996b_mtty</name>
  <path>/sys/devices/virtual/mtty/mtty/d36d7d0f-c23d-4fef-bb9c-ed393954996b</path>
  <parent>mtty_mtty</parent>
  <driver>
    <name>mtty</name>
  </driver>
  <capability type='mdev'>
    <type id='mtty-2'/>
    <uuid>d36d7d0f-c23d-4fef-bb9c-ed393954996b</uuid>
    <parent_addr>mtty</parent_addr>
    <iommuGroup number='1'/>
  </capability>
</device>

# virsh nodedev-dumpxml mdev_d36d7d0f_cf3d_4fef_bb9c_ed393954996b_mtty 
<device>
  <name>mdev_d36d7d0f_cf3d_4fef_bb9c_ed393954996b_mtty</name>
  <path>/sys/devices/virtual/mtty/mtty/d36d7d0f-cf3d-4fef-bb9c-ed393954996b</path>
  <parent>mtty_mtty</parent>
  <driver>
    <name>mtty</name>
  </driver>
  <capability type='mdev'>
    <type id='mtty-1'/>
    <uuid>d36d7d0f-cf3d-4fef-bb9c-ed393954996b</uuid>
    <parent_addr>mtty</parent_addr>
    <iommuGroup number='0'/>
  </capability>
</device>
Comment 8 yalzhang@redhat.com 2022-10-24 02:38:46 UTC 
Hi Sebas, I have tried on x86_64 with the command on comment 6, it works well. So I think we can close it as "current release" and add a keyword "TestOnly" since there is no changes in libvirt. 
What do you think?

Summary:
1. After kernel-modules-internal installed and mtty loaded, we can create mdev device for test;
2. nodedev-list/info/define/undefine/start/create/dumpxml/event works well;
3. nodedev-detach/reset/reattach can not work since it is not a pci device;
4. nodedev-autostart can work as expected if we can load mtty module at system boot time；

Scenario 1: create a mtty device:
install the package kernel-modules-internal
# rpm -q kernel-modules-internal
kernel-modules-internal-5.14.0-177.el9.x86_64

# modprobe mtty
# virsh nodedev-list  | grep mtty
mtty_mtty

# cat dev2.xml
<device>
	<parent>mtty_mtty</parent>
	<capability type="mdev">
	    <type id="mtty-2"/>
	    <uuid>d36d7d0f-c23d-4fef-bb9c-ed393954996b</uuid>
        </capability>
</device>

# virsh nodedev-create dev2.xml 
Node device mdev_d36d7d0f_c23d_4fef_bb9c_ed393954996b_mtty created from dev2.xml

# virsh nodedev-list --cap mdev
mdev_d36d7d0f_c23d_4fef_bb9c_ed393954996b_mtty

# virsh nodedev-dumpxml mdev_d36d7d0f_c23d_4fef_bb9c_ed393954996b_mtty
<device>
  <name>mdev_d36d7d0f_c23d_4fef_bb9c_ed393954996b_mtty</name>
  <path>/sys/devices/virtual/mtty/mtty/d36d7d0f-c23d-4fef-bb9c-ed393954996b</path>
  <parent>mtty_mtty</parent>
  <driver>
    <name>mtty</name>
  </driver>
  <capability type='mdev'>
    <type id='mtty-2'/>
    <uuid>d36d7d0f-c23d-4fef-bb9c-ed393954996b</uuid>
    <parent_addr>mtty</parent_addr>
    <iommuGroup number='135'/>
  </capability>
</device>

# virsh nodedev-info mdev_d36d7d0f_c23d_4fef_bb9c_ed393954996b_mtty
Name:           mdev_d36d7d0f_c23d_4fef_bb9c_ed393954996b_mtty
Parent:         mtty_mtty
Active:         yes
Persistent:     no
Autostart:      no

# virsh nodedev-autostart mdev_d36d7d0f_c23d_4fef_bb9c_ed393954996b_mtty
error: failed to mark device mdev_d36d7d0f_c23d_4fef_bb9c_ed393954996b_mtty as autostarted
error: Requested operation is not valid: cannot set autostart for transient device

Scenario 2: 
define a mdev device and set it as autostart:
# cat dev1.xml 
<device>
	<parent>mtty_mtty</parent>
	<capability type="mdev">
	    <type id="mtty-1"/>
	    <uuid>d36d7d0f-cf3d-4fef-bb9c-ed393954996b</uuid>
        </capability>
</device>

# virsh nodedev-define dev1.xml
Node device 'mdev_d36d7d0f_cf3d_4fef_bb9c_ed393954996b_mtty' defined from 'dev1.xml'

# virsh nodedev-start mdev_d36d7d0f_cf3d_4fef_bb9c_ed393954996b_mtty
Device mdev_d36d7d0f_cf3d_4fef_bb9c_ed393954996b_mtty started

# virsh nodedev-info mdev_d36d7d0f_cf3d_4fef_bb9c_ed393954996b_mtty
Name:           mdev_d36d7d0f_cf3d_4fef_bb9c_ed393954996b_mtty
Parent:         mtty_mtty
Active:         yes
Persistent:     yes
Autostart:      no

# virsh nodedev-autostart mdev_d36d7d0f_cf3d_4fef_bb9c_ed393954996b_mtty
Device mdev_d36d7d0f_cf3d_4fef_bb9c_ed393954996b_mtty marked as autostarted

# virsh nodedev-info mdev_d36d7d0f_cf3d_4fef_bb9c_ed393954996b_mtty
Name:           mdev_d36d7d0f_cf3d_4fef_bb9c_ed393954996b_mtty
Parent:         mtty_mtty
Active:         yes
Persistent:     yes
Autostart:      yes

reboot the host and check
# virsh nodedev-list  --cap mdev
# (no outputs)
# modprobe mtty
# virsh nodedev-list --tree
computer
  |
  +- mtty_mtty
  |   |
  |   +- mdev_d36d7d0f_cf3d_4fef_bb9c_ed393954996b_mtty
  |
Comment 9 smitterl 2022-10-24 07:42:49 UTC 
(In reply to yalzhang from comment #8)
> Hi Sebas, I have tried on x86_64 with the command on comment 6, it works
> well. So I think we can close it as "current release" and add a keyword
> "TestOnly" since there is no changes in libvirt. 
> What do you think?
> 
> Summary:
> 1. After kernel-modules-internal installed and mtty loaded, we can create
> mdev device for test;
> 2. nodedev-list/info/define/undefine/start/create/dumpxml/event works well;
> 3. nodedev-detach/reset/reattach can not work since it is not a pci device;
> 4. nodedev-autostart can work as expected if we can load mtty module at
> system boot time；
> 
> Scenario 1: create a mtty device:
> install the package kernel-modules-internal
> # rpm -q kernel-modules-internal
> kernel-modules-internal-5.14.0-177.el9.x86_64
> 
> # modprobe mtty
> # virsh nodedev-list  | grep mtty
> mtty_mtty
> 
> # cat dev2.xml
> <device>
> 	<parent>mtty_mtty</parent>
> 	<capability type="mdev">
> 	    <type id="mtty-2"/>
> 	    <uuid>d36d7d0f-c23d-4fef-bb9c-ed393954996b</uuid>
>         </capability>
> </device>
> 
> # virsh nodedev-create dev2.xml 
> Node device mdev_d36d7d0f_c23d_4fef_bb9c_ed393954996b_mtty created from
> dev2.xml
> 
> # virsh nodedev-list --cap mdev
> mdev_d36d7d0f_c23d_4fef_bb9c_ed393954996b_mtty
> 
> # virsh nodedev-dumpxml mdev_d36d7d0f_c23d_4fef_bb9c_ed393954996b_mtty
> <device>
>   <name>mdev_d36d7d0f_c23d_4fef_bb9c_ed393954996b_mtty</name>
>  
> <path>/sys/devices/virtual/mtty/mtty/d36d7d0f-c23d-4fef-bb9c-ed393954996b</
> path>
>   <parent>mtty_mtty</parent>
>   <driver>
>     <name>mtty</name>
>   </driver>
>   <capability type='mdev'>
>     <type id='mtty-2'/>
>     <uuid>d36d7d0f-c23d-4fef-bb9c-ed393954996b</uuid>
>     <parent_addr>mtty</parent_addr>
>     <iommuGroup number='135'/>
>   </capability>
> </device>
> 
> # virsh nodedev-info mdev_d36d7d0f_c23d_4fef_bb9c_ed393954996b_mtty
> Name:           mdev_d36d7d0f_c23d_4fef_bb9c_ed393954996b_mtty
> Parent:         mtty_mtty
> Active:         yes
> Persistent:     no
> Autostart:      no
> 
> # virsh nodedev-autostart mdev_d36d7d0f_c23d_4fef_bb9c_ed393954996b_mtty
> error: failed to mark device mdev_d36d7d0f_c23d_4fef_bb9c_ed393954996b_mtty
> as autostarted
> error: Requested operation is not valid: cannot set autostart for transient
> device
> 
> Scenario 2: 
> define a mdev device and set it as autostart:
> # cat dev1.xml 
> <device>
> 	<parent>mtty_mtty</parent>
> 	<capability type="mdev">
> 	    <type id="mtty-1"/>
> 	    <uuid>d36d7d0f-cf3d-4fef-bb9c-ed393954996b</uuid>
>         </capability>
> </device>
> 
> # virsh nodedev-define dev1.xml
> Node device 'mdev_d36d7d0f_cf3d_4fef_bb9c_ed393954996b_mtty' defined from
> 'dev1.xml'
> 
> # virsh nodedev-start mdev_d36d7d0f_cf3d_4fef_bb9c_ed393954996b_mtty
> Device mdev_d36d7d0f_cf3d_4fef_bb9c_ed393954996b_mtty started
> 
> # virsh nodedev-info mdev_d36d7d0f_cf3d_4fef_bb9c_ed393954996b_mtty
> Name:           mdev_d36d7d0f_cf3d_4fef_bb9c_ed393954996b_mtty
> Parent:         mtty_mtty
> Active:         yes
> Persistent:     yes
> Autostart:      no
> 
> # virsh nodedev-autostart mdev_d36d7d0f_cf3d_4fef_bb9c_ed393954996b_mtty
> Device mdev_d36d7d0f_cf3d_4fef_bb9c_ed393954996b_mtty marked as autostarted
> 
> # virsh nodedev-info mdev_d36d7d0f_cf3d_4fef_bb9c_ed393954996b_mtty
> Name:           mdev_d36d7d0f_cf3d_4fef_bb9c_ed393954996b_mtty
> Parent:         mtty_mtty
> Active:         yes
> Persistent:     yes
> Autostart:      yes
> 
> reboot the host and check
> # virsh nodedev-list  --cap mdev
> # (no outputs)
> # modprobe mtty
> # virsh nodedev-list --tree
> computer
>   |
>   +- mtty_mtty
>   |   |
>   |   +- mdev_d36d7d0f_cf3d_4fef_bb9c_ed393954996b_mtty
>   |

Sure, whatever you consider the best option. Thank you.