Considering bug 2143158 was moved to POST for 9.3.0, can a backport be done here for 8.9.0?

Assigned to Jonathon directly since he noted the upstream commit in the cloned from bug.

Tested with below steps:
1. make sure s390utils related packages, mdevctl and libvirt are installed:
# rpm -qa s390utils*
s390utils-cpuplugd-2.25.0-2.el8.s390x
s390utils-hmcdrvfs-2.25.0-2.el8.s390x
s390utils-2.25.0-2.el8.s390x
s390utils-chreipl-fcp-mpath-2.25.0-2.el8.s390x
s390utils-base-2.25.0-2.el8.s390x
s390utils-ziomon-2.25.0-2.el8.s390x
s390utils-iucvterm-2.25.0-2.el8.s390x
s390utils-cmsfs-2.25.0-2.el8.s390x
s390utils-osasnmpd-2.25.0-2.el8.s390x
s390utils-cmsfs-fuse-2.25.0-2.el8.s390x
s390utils-zdsfs-2.25.0-2.el8.s390x
s390utils-cpacfstatsd-2.25.0-2.el8.s390x
s390utils-core-2.25.0-2.el8.s390x
s390utils-mon_statd-2.25.0-2.el8.s390x
s390utils-devel-2.25.0-2.el8.s390x

# rpm -q mdevctl
mdevctl-1.1.0-2.el8.s390x


# rpm -q libvirt qemu-kvm
libvirt-8.0.0-21.module+el8.9.0+19166+e262ca96.s390x
qemu-kvm-6.2.0-35.module+el8.9.0+19166+e262ca96.s390x

2. Found mdevctl callout script at places:
# ls /etc/mdevctl.d/scripts.d/callouts/ap-check.sh
/etc/mdevctl.d/scripts.d/callouts/ap-check.sh

# cat /etc/mdevctl.d/scripts.d/callouts/ap-check.sh
[ -e /usr/lib/mdevctl/scripts.d/callouts/ap-check ] && /usr/lib/mdevctl/scripts.d/callouts/ap-check "$@"

# ls /usr/lib/mdevctl/scripts.d/callouts/ap-check
/usr/lib/mdevctl/scripts.d/callouts/ap-check

3. Restart virtnodedevd service
# systemctl restart virtnodedevd

4. virsh nodedev-create vfio_ap_nodedev.xml
# cat vfio_ap_nodedev.xml 
<device>
	<!-- same parent device for all -->
	<parent>ap_matrix</parent>
	<capability type="mdev">
		<type id="vfio_ap-passthrough"/>
		<uuid>d36d7d0f-cf3d-4fef-bb9c-ed393954996b</uuid>
		<attr name="assign_adapter" value="0x02"/>
		<attr name="assign_domain" value="0x002b"/>
	</capability>
</device>

# virsh nodedev-create vfio_ap_nodedev.xml
Node device mdev_d36d7d0f_cf3d_4fef_bb9c_ed393954996b_matrix created from vfio_ap_nodedev.xml

5. virsh nodedev-dumpxml mdev_d36d7d0f_cf3d_4fef_bb9c_ed393954996b_matrix, and could not see attribute "assign_adapter" and "assign_domain"
# virsh nodedev-dumpxml mdev_d36d7d0f_cf3d_4fef_bb9c_ed393954996b_matrix
<device>
  <name>mdev_d36d7d0f_cf3d_4fef_bb9c_ed393954996b_matrix</name>
  <path>/sys/devices/vfio_ap/matrix/d36d7d0f-cf3d-4fef-bb9c-ed393954996b</path>
  <parent>ap_matrix</parent>
  <driver>
    <name>vfio_mdev</name>
  </driver>
  <capability type='mdev'>
    <type id='vfio_ap-passthrough'/>
    <uuid>d36d7d0f-cf3d-4fef-bb9c-ed393954996b</uuid>
    <iommuGroup number='4'/>
  </capability>
</device>


6. Check the mdevctl output:
# mdevctl list -vu d36d7d0f-cf3d-4fef-bb9c-ed393954996b --dumpjson
{
  "mdev_type": "vfio_ap-passthrough",
  "start": "manual",
  "attrs": [
    {
      "assign_adapter": "2"
    },
    {
      "assign_domain": "43"
    }
  ]
}

# mdevctl list -vu d36d7d0f-cf3d-4fef-bb9c-ed393954996b
d36d7d0f-cf3d-4fef-bb9c-ed393954996b matrix vfio_ap-passthrough manual
  Attrs:
    @{0}: {"assign_adapter":"2"}
    @{1}: {"assign_domain":"43"}


7. virsh nodedev-dumpxml mdev_d36d7d0f_cf3d_4fef_bb9c_ed393954996b_matrix again. And still could not see attribute "assign_adapter" and "assign_domain"
# virsh nodedev-dumpxml mdev_d36d7d0f_cf3d_4fef_bb9c_ed393954996b_matrix
<device>
  <name>mdev_d36d7d0f_cf3d_4fef_bb9c_ed393954996b_matrix</name>
  <path>/sys/devices/vfio_ap/matrix/d36d7d0f-cf3d-4fef-bb9c-ed393954996b</path>
  <parent>ap_matrix</parent>
  <driver>
    <name>vfio_mdev</name>
  </driver>
  <capability type='mdev'>
    <type id='vfio_ap-passthrough'/>
    <uuid>d36d7d0f-cf3d-4fef-bb9c-ed393954996b</uuid>
    <iommuGroup number='4'/>
  </capability>
</device>


8. Then I restart virtnodedevd service again
# # systemctl restart virtnodedevd

9. Check virsh nodedev-dumpxml output, then found attribute "assign_adapter" and "assign_domain".
# virsh nodedev-dumpxml mdev_d36d7d0f_cf3d_4fef_bb9c_ed393954996b_matrix
<device>
  <name>mdev_d36d7d0f_cf3d_4fef_bb9c_ed393954996b_matrix</name>
  <path>/sys/devices/vfio_ap/matrix/d36d7d0f-cf3d-4fef-bb9c-ed393954996b</path>
  <parent>ap_matrix</parent>
  <driver>
    <name>vfio_mdev</name>
  </driver>
  <capability type='mdev'>
    <type id='vfio_ap-passthrough'/>
    <uuid>d36d7d0f-cf3d-4fef-bb9c-ed393954996b</uuid>
    <iommuGroup number='4'/>
    <attr name='assign_adapter' value='2'/>
    <attr name='assign_domain' value='43'/>
  </capability>
</device>

Hi Jonathon, I think this issue is same with https://bugzilla.redhat.com/show_bug.cgi?id=2143158#c19, and the fix is posted as comments https://bugzilla.redhat.com/show_bug.cgi?id=2143158#c22, so I think the new fix code also need to be a part of the backport.

Aha, it seems that somehow I had missed that patch. Let me look into it.

It would seem to me that we should use the FailedQA process in order to move this bug back to assigned, allow Jonathon to apply the updated change downstream, and then go through the re-verification.  Leaving a needinfo on qa for that decision...

NB: FailedQA always fails to "reset" the DTM/ITM and invariably that leads to missed deadline bot messaging. So we should update the values as we update this bug...