Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1910848

Summary:	SLOF: backport "virtio: Enable IOMMU" from upstream to 8.4
Product:	Red Hat Enterprise Linux 8	Reporter:	Yihuang Yu <yihyu>
Component:	SLOF	Assignee:	Virtualization Maintenance <virt-maint>
Status:	CLOSED WONTFIX	QA Contact:	xianwang <xianwang>
Severity:	medium	Docs Contact:	Jiri Herrmann <jherrman>
Priority:	medium
Version:	8.4	CC:	bfu, dgibson, jherrman, jinzhao, juzhang, lvivier, mdeng, ngu, qzhang, virt-maint, xianwang, xuma, zhenyzha
Target Milestone:	rc	Keywords:	Triaged
Target Release:	---	Flags:	xianwang: needinfo- pm-rhel: mirror+
Hardware:	ppc64le
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	Known Issue
Doc Text:	.Virtual machines with `iommu_platform=on` fail to start on IBM POWER RHEL 8 currently does not support the `iommu_platform=on` parameter for virtual machines (VMs) on IBM POWER system. As a consequence, starting a VM with this parameter on IBM POWER hardware results in the VM becoming unresponsive during the boot process.	Story Points:	---
Clone Of:		Environment:
Last Closed:	2021-01-11 06:16:23 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Yihuang Yu 2020-12-25 08:05:46 UTC

Description of problem:
The property of virtio devices named "iommu_platform=on" is important and it may be "on" by default in RHEL.8.4, the current SLOF version in RHEL.8.4 is "SLOF-20191022-3.git899d9883.module+el8.3.0+6423+e4cb6418.noarch", it lacks a commit(virtio: Enable IOMMU), so when qemu is started with iommu_platform=on, guest drivers refuse to accept it.

The missed commit in upstream:
commit 7477a174abc03d3e6f508412f69ea0bbd63b93ac
Author: Alexey Kardashevskiy <aik>
Date:   Wed Dec 4 13:06:27 2019 +1100

    virtio: Enable IOMMU
    
    When QEMU is started with iommu_platform=on, the guest driver must accept
    it or the device will fail.
    
    This enables IOMMU support for virtio-net, -scsi, -block, -serial, -9p
    devices. -serial and -9p are only compile tested though.
    
    For virtio-net we map all RX buffers once and TX when xmit() is called
    and unmap older pages when we are about to reuse the VQ descriptor.
    As all other devices are synchronous, we unmap IOMMU pages right after
    completion of a transaction.
    
    This depends on QEMU's:
    https://patchwork.ozlabs.org/patch/1194067/
    
    Signed-off-by: Alexey Kardashevskiy <aik>
    Tested-by: Michael Roth <mdroth.ibm.com>
    ---
    Changes:
    v5:
    * fixed queue size calculation
    * reworked virtio_free_desc() to only unmap what is has mapped
    
    v4:
    * ditched vqs->id in virtio_queue_init_vq
    
    v2:
    * added Mike's fs/dma-instance-function.fs
    * total rework


Version-Release number of selected component (if applicable):
Host kernel: 4.18.0-266.el8.ppc64le
qemu version: qemu-kvm-4.2.0-39.module+el8.4.0+9248+2cae4f71.ppc64le
SLOF version: SLOF-20191022-3.git899d9883.module+el8.3.0+6423+e4cb6418.noarch

How reproducible:
always

Steps to Reproduce:
1. Launch a guest with iommu_platform=on
MALLOC_PERTURB_=1  /usr/libexec/qemu-kvm \
    -S  \
    -name 'avocado-vt-vm1'  \
    -sandbox on  \
    -machine pseries  \
    -nodefaults \
    -device VGA,bus=pci.0,addr=0x2 \
    -m 60416  \
    -smp 64,maxcpus=64,cores=32,threads=1,sockets=2  \
    -cpu 'host' \
    -chardev socket,nowait,server,id=qmp_id_qmpmonitor1,path=/tmp/avocado_o3wb62rf/monitor-qmpmonitor1-20201225-021306-VFqrSUOm  \
    -mon chardev=qmp_id_qmpmonitor1,mode=control \
    -chardev socket,nowait,server,id=qmp_id_catch_monitor,path=/tmp/avocado_o3wb62rf/monitor-catch_monitor-20201225-021306-VFqrSUOm  \
    -mon chardev=qmp_id_catch_monitor,mode=control \
    -chardev socket,nowait,server,id=chardev_serial0,path=/tmp/avocado_o3wb62rf/serial-serial0-20201225-021306-VFqrSUOm \
    -device spapr-vty,id=serial0,reg=0x30000000,chardev=chardev_serial0 \
    -device qemu-xhci,id=usb1,bus=pci.0,addr=0x3 \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \
    -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pci.0,addr=0x4,disable-legacy=on,disable-modern=off,iommu_platform=on \
    -blockdev node-name=file_image1,driver=file,auto-read-only=on,discard=unmap,aio=threads,filename=/home/kar/vt_test_images/rhel840-ppc64le-virtio-scsi.qcow2,cache.direct=on,cache.no-flush=off \
    -blockdev node-name=drive_image1,driver=qcow2,read-only=off,cache.direct=on,cache.no-flush=off,file=file_image1 \
    -device scsi-hd,id=image1,drive=drive_image1,write-cache=on \
    -device virtio-net-pci,mac=9a:d7:7c:9c:d3:58,id=idCftkwz,netdev=idaoRFmk,bus=pci.0,addr=0x5,disable-legacy=on,disable-modern=off,iommu_platform=on  \
    -netdev tap,id=idaoRFmk,vhost=on,vhostfd=20,fd=16  \
    -vnc :0  \
    -rtc base=utc,clock=host  \
    -boot menu=off,order=cdn,once=c,strict=off \
    -enable-kvm

2. Capture SLOF outputs from the guest serial console

3.

Actual results:
2020-12-25 02:13:14: SLOF^[[0m^[[?25l **********************************************************************
2020-12-25 02:13:14: ^[[1mQEMU Starting
2020-12-25 02:13:14: ^[[0m Build Date = Apr 28 2020 01:44:26
2020-12-25 02:13:14:  FW Version = mockbuild@ release 20191022
2020-12-25 02:13:14:  Press "s" to enter Open Firmware.
2020-12-25 02:13:14:
2020-12-25 02:13:14: ^[[0m^[[?25h
2020-12-25 02:13:15: C0000C0100C0120C0140C0200C0240C0260C02E0C0300C0320C0340
2020-12-25 02:13:15: C0360
2020-12-25 02:13:15: C0370C0380
2020-12-25 02:13:15: C0371C0373
2020-12-25 02:13:16: C0374C03F0C0400C0480C04C0C04D0
2020-12-25 02:13:16: C0500Populating /vdevice methods
2020-12-25 02:13:16: Populating /vdevice/vty@30000000
2020-12-25 02:13:16: Populating /vdevice/nvram@71000000
2020-12-25 02:13:16: C05A0
2020-12-25 02:13:16: Populating /pci@800000020000000
2020-12-25 02:13:16:                      00 1000 (D) : 1234 1111    qemu vga
2020-12-25 02:13:16:                      00 1800 (D) : 1b36 000d    serial bus [ usb-xhci ]
2020-12-25 02:13:17:                      00 2000 (D) : 1af4 1048    virtio [ scsi ]
2020-12-25 02:13:17: Populating /pci@800000020000000/scsi@4
2020-12-25 02:13:17: virtioscsi_init: failed        <--------------------------------        Device initialization failed
2020-12-25 02:13:17:                      00 2800 (D) : 1af4 1041    virtio [ net ]
2020-12-25 02:13:17: virtio-net-open failed       <--------------------------------        Device initialization failed
2020-12-25 02:13:17: C0600C06C0C0700
2020-12-25 02:13:17: C0800
2020-12-25 02:13:17: C0880
2020-12-25 02:13:18: No NVRAM common partition, re-initializing...
2020-12-25 02:13:18: C0890C08A0
2020-12-25 02:13:18: C08A8Installing QEMU fb
2020-12-25 02:13:18:
2020-12-25 02:13:18:
2020-12-25 02:13:18:
2020-12-25 02:13:18: C08B0Scanning USB
2020-12-25 02:13:18:   XHCI: Initializing
2020-12-25 02:13:18: C08C0C08D0No console specified using hvterm
2020-12-25 02:13:18: C08E0C08E8C08FF
2020-12-25 02:13:19:
2020-12-25 02:13:19:
2020-12-25 02:13:19:
2020-12-25 02:13:19:   Welcome to Open Firmware
2020-12-25 02:13:19:
2020-12-25 02:13:19:   Copyright (c) 2004, 2017 IBM Corporation All rights reserved.
2020-12-25 02:13:19:   This program and the accompanying materials are made available
2020-12-25 02:13:19:   under the terms of the BSD License available at
2020-12-25 02:13:19:   http://www.opensource.org/licenses/bsd-license.php
2020-12-25 02:13:19:
2020-12-25 02:13:19:
2020-12-25 02:13:19: Trying to load:  from: disk ...
2020-12-25 02:13:19: E3405: No such device
2020-12-25 02:13:19:
2020-12-25 02:13:19: E3407: Load failed
2020-12-25 02:13:19:
2020-12-25 02:13:19:   Type 'boot' and press return to continue booting the system.
2020-12-25 02:13:19:   Type 'reset-all' and press return to reboot the system.
2020-12-25 02:13:19:
2020-12-25 02:13:19:
2020-12-25 02:13:19: Ready!
2020-12-25 02:13:19: 0 > 

Expected results:
Guest started successfully

Additional info:
The SLOF in RHELAV-8.4 includes the lacked commit, but I think RHEL.8.4 is also worth backporting this commit.

Comment 1 David Gibson 2021-01-04 00:07:59 UTC

Yihuang,

iommu_platform should not be on by default in RHEL-8.4, only in RHEL-AV-8.4 which has a newer SLOF.

Comment 2 Yihuang Yu 2021-01-04 01:17:37 UTC

(In reply to David Gibson from comment #1)
> Yihuang,
> 
> iommu_platform should not be on by default in RHEL-8.4, only in RHEL-AV-8.4
> which has a newer SLOF.

David, thank you for the explanation.
But we may need to consider the case where users manually enable it(like me), this bug will be a blocker.
So I still tend to fix it, otherwise we need a document to remind users.

Comment 3 David Gibson 2021-01-04 02:41:22 UTC

I think this is more complexity than it's worth: essentially iommu_platform=on is not supported with RHEL-8.4, only RHEL-AV-8.4.  Without a compelling reason to add that support, I don't think we want to bother with the backport.

Comment 4 Yihuang Yu 2021-01-04 12:26:20 UTC

Make sense, since RHEL-8.5 and RHELAV-8.5 will have newer SLOF versions, adding documentation without fixing it on RHEL-8.4 is also an acceptable option for me, even though this is a test blocker.

Comment 5 David Gibson 2021-01-05 01:57:08 UTC

Yihuang,

How is this blocking tests?  Shouldn't tests be using either a full RHEL or full RHEL-AV configuration, not a mixture?

Comment 6 Yihuang Yu 2021-01-06 11:51:00 UTC

(In reply to David Gibson from comment #5)
> Yihuang,
> 
> How is this blocking tests?  Shouldn't tests be using either a full RHEL or
> full RHEL-AV configuration, not a mixture?

Hello David,

Yes, we should keep the full RHEL/RHELAV configuration for use.
But in my opinion, this option is user-configurable, so it is best to declare in the documentation that RHEL-8.4 is not supported. Otherwise, when someone launching a guest with iommu_platform=on in RHEL-8.4, the guest will hang in the SLOF stage.

Comment 7 David Gibson 2021-01-07 00:05:01 UTC

I can see that we want to improve the documentation (though it would need to be documented at the libvirt level, not qemu).  I still don't see how that makes it a test blocker.

Comment 8 xianwang 2021-01-11 02:44:23 UTC

(In reply to David Gibson from comment #7)
> I can see that we want to improve the documentation (though it would need to
> be documented at the libvirt level, not qemu).  I still don't see how that
> makes it a test blocker.

Hi, David,
I have communicated with yihyu, we both agree with you that we declare this point that RHEL-8.4 does not support configuring "iommu_platform=on" in documentation, this should be a resolution for this bug, and this is not a test blocker, so, how do you think I open a new documentation bug for this?

Comment 9 David Gibson 2021-01-11 06:14:34 UTC

Unfortunately, I'm not sure who we need to talk to about a documentation update - my contact there doesn't seem to be around any more.

Comment 10 xianwang 2021-01-11 06:40:17 UTC

(In reply to David Gibson from comment #9)
> Unfortunately, I'm not sure who we need to talk to about a documentation
> update - my contact there doesn't seem to be around any more.

Oh, I am not very understand what you said, I think we could note this in RHEL-8.4.0 product official documentation.
Hi, Jiri,
Do you think I could report a documentation bug simply as: in RHEL8.4.0 on power platform, it failed to boot a guest with configuration "iommu_platform=on"? 

thanks