Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 2187605

Summary: [aarch64] Compile out pcie-to-pci bridge on arm
Product: Red Hat Enterprise Linux 9 Reporter: Eric Auger <eric.auger>
Component: qemu-kvmAssignee: Sebastian Ott <seott>
qemu-kvm sub component: PCI QA Contact: Yihuang Yu <yihyu>
Status: CLOSED WONTFIX Docs Contact:
Severity: medium    
Priority: medium CC: alexander.lougovski, cohuck, gshan, lijin, virt-maint
Version: 9.3Keywords: RFE, Triaged
Target Milestone: rcFlags: pm-rhel: mirror+
Target Release: ---   
Hardware: aarch64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-05-30 17:27:44 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Eric Auger 2023-04-18 06:47:58 UTC
The vsmmu and virtio-iommu are known to be buggy with the pcie-to-pci bridge. Fixing that will require important changes in the iommu group probing method in the iommu subsystem. Until this gets fixed, let's compile out the pcie-to-pci bridge emphasizing this is not supported and not to be tested.

Comment 2 Eric Auger 2023-05-09 12:44:05 UTC
Hi Sebastian,

Just looking at the attachment, you cannot simply disable CONFIG_PCIE_PORT because we need the root ports which are extensively used in our pcie topologies. I wonder why/if the pcie_pci_bridge.c does not deserve its own config. I would try to understand if there is no way to decouple the pcie_pci_bridge from the root ports. This would require some upstream activity though. Try to gather more info on the rationale while reading docs/pcie* in the qemu directory.

Comment 3 Sebastian Ott 2023-05-09 14:24:00 UTC
Hej Eric,

..sry, it's the first time I've looked at the qemu build system. I don't think I've disabled CONFIG_PCIE_PORT but removed pcie_pci_bridge.c from the compilation unit when CONFIG_PCIE_PORT is set. If an extra config option that's dependent on CONFIG_PCIE_PORT is the way to go then I'll do that.

Thanks!

Comment 4 Eric Auger 2023-05-09 17:24:05 UTC
(In reply to Sebastian Ott from comment #3)
> Hej Eric,
> 
> ..sry, it's the first time I've looked at the qemu build system. I don't
> think I've disabled CONFIG_PCIE_PORT but removed pcie_pci_bridge.c from the
> compilation unit when CONFIG_PCIE_PORT is set. If an extra config option
> that's dependent on CONFIG_PCIE_PORT is the way to go then I'll do that.
> 
> Thanks!

Oh indeed I skimmed through your patch too quickly. In practice, downstream, we usually modify
./configs/devices/aarch64-softmmu/aarch64-rh-devices.mak to enable devices. This obviously assumes that your device can be selected using a proper config independently on the others. We try to avoid RHEL-only patches as the solution you currently suggests. I would suggest you try to isolate the pcie-to-pci bridge upstream, if this makes sense and if this is feasible. Once isolated we could let the option disabled by default on RHEL.

Comment 10 Sebastian Ott 2023-05-11 15:47:22 UTC
I've got the build running using:
./configure --with-devices-aarch64=aarch64-rh-devices --target-list=aarch64-softmmu

Unless someone objects I'll send https://bugzilla.redhat.com/attachment.cgi?id=1964039&action=diff upstream.

I've confirmed that with these hw/pci-bridge/pcie_pci_bridge.c is not compiled.

Is there some means to test this? Like a qemu commandline using "pcie-pci-bridge" ?

Comment 11 Eric Auger 2023-05-11 17:00:10 UTC
(In reply to Sebastian Ott from comment #10)
> I've got the build running using:
> ./configure --with-devices-aarch64=aarch64-rh-devices
> --target-list=aarch64-softmmu
> 
> Unless someone objects I'll send
> https://bugzilla.redhat.com/attachment.cgi?id=1964039&action=diff upstream.
> 
> I've confirmed that with these hw/pci-bridge/pcie_pci_bridge.c is not
> compiled.
> 
> Is there some means to test this? Like a qemu commandline using
> "pcie-pci-bridge" ?

example:

-device pcie-pci-bridge,bus=pcie.0,addr=0x4.0x0,id=pci.1 \
-device pcie-root-port,bus=pcie.0,chassis=3,id=pcie.3,addr=0x3.0x0 \
-device virtio-blk-pci,bus=pcie.3,scsi=off,drive=drv0,id=virtio-disk0,bootindex=1

but you mostly need to check the right devices are compiled here I think. 

You shall be able to use qmp query-pci to check the pci topology but the qemu cmd would have failed before in your case.

Eric

Comment 12 Sebastian Ott 2023-05-24 16:08:56 UTC
The upstream patch was merged upstream as:
6a36a4ced8 hw/pci-bridge: make building pcie-to-pci bridge configurable

I created a request to merge the upstream and downstream patch:
https://gitlab.com/redhat/rhel/src/qemu-kvm/qemu-kvm/-/merge_requests/282

Comment 13 Sebastian Ott 2023-05-25 15:11:59 UTC
(In reply to Sebastian Ott from comment #12)
> I created a request to merge the upstream and downstream patch:
> https://gitlab.com/redhat/rhel/src/qemu-kvm/qemu-kvm/-/merge_requests/282

wrong repo - new MR: https://gitlab.com/redhat/centos-stream/src/qemu-kvm/-/merge_requests/168

Comment 14 Eric Auger 2023-05-30 17:27:44 UTC
We eventually decided that the removal of the pcie-to-pci bridge device cannot happen in the middle of a major release. So we postpone this action and close the BZ as WONTFIX. Thanks to Sebastian's work upstream, it will be straightforward to compile the device out if we decide to do so.