RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1400785 - qemu: Remove pxi-expander-bridge (PXB) device for Power
Summary: qemu: Remove pxi-expander-bridge (PXB) device for Power
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: qemu-kvm-rhev
Version: 7.4
Hardware: ppc64le
OS: Unspecified
unspecified
medium
Target Milestone: rc
: ---
Assignee: David Gibson
QA Contact: Qunfang Zhang
URL:
Whiteboard:
Depends On:
Blocks: 1436051
TreeView+ depends on / blocked
 
Reported: 2016-12-02 05:04 UTC by David Gibson
Modified: 2017-08-02 03:35 UTC (History)
8 users (show)

Fixed In Version: qemu-kvm-rhev-2.9.0-1.el7
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1436051 (view as bug list)
Environment:
Last Closed: 2017-08-01 23:39:45 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2017:2392 0 normal SHIPPED_LIVE Important: qemu-kvm-rhev security, bug fix, and enhancement update 2017-08-01 20:04:36 UTC

Description David Gibson 2016-12-02 05:04:58 UTC 
Description of problem:

We have some qemu devices that further consideration shows aren't really useful on Power:

1) ehci-hcd

EHCI is not well tested on Power, and has hit a number of bugs.  There's nothing it can do that can't be done with an XHCI device, which is the recommended USB controller for Power guests.

2) pxb

PCI Expander Bridge.  I'm trying to confirm, but this appears to be a hack to work around x86's poor support for multiple PCI domains / host bridges.  On Power we should prefer creating multiple independent host bridges.


We should consider removing these devices from qemu for RHEL 7.4.
Comment 1 David Gibson 2016-12-05 04:57:34 UTC 
Marcel,

I'm trying to understand exactly what the pxb device is for.  My impression is that it's essentially a hack to deal with the fact that x86 doesn't really handle truly independent PCI host bridges nicely. Does that sound right?

I think we don't want it on Power, but I'm trying to make sure.

[Incidentally pxi_expander_bridge.c #includes hw/i386/pc.h, but doesn't seem to need it (no compile errors if I take it away).]
Comment 2 Marcel Apfelbaum 2016-12-05 11:13:20 UTC 
(In reply to David Gibson from comment #1)
> Marcel,

Hi David,

> 
> I'm trying to understand exactly what the pxb device is for.  My impression
> is that it's essentially a hack to deal with the fact that x86 doesn't
> really handle truly independent PCI host bridges nicely. Does that sound
> right?
>

You are right, anyway, the real reason is to be able
to "assign" a PCI device to the correct Guest NUMA node.

If the guest RAM comes from different host NUMA nodes and you have
a host assigned device , you need to associate it somehow to
the corresponding guest NUMA node.

The pxb/pxb-pcie device exposes an extra root bus that can be associated
with a specific guest NUMA node using the ACPI tables. 
 
> I think we don't want it on Power, but I'm trying to make sure.
> 

Do you have a solution for the problem above?

> [Incidentally pxi_expander_bridge.c #includes hw/i386/pc.h, but doesn't seem
> to need it (no compile errors if I take it away).]

Thanks,
Marcel
Comment 3 Andrea Bolognani 2016-12-05 13:53:58 UTC 
(In reply to Marcel Apfelbaum from comment #2)
> > I'm trying to understand exactly what the pxb device is for.  My impression
> > is that it's essentially a hack to deal with the fact that x86 doesn't
> > really handle truly independent PCI host bridges nicely. Does that sound
> > right?
> 
> You are right, anyway, the real reason is to be able
> to "assign" a PCI device to the correct Guest NUMA node.
> 
> If the guest RAM comes from different host NUMA nodes and you have
> a host assigned device , you need to associate it somehow to
> the corresponding guest NUMA node.
> 
> The pxb/pxb-pcie device exposes an extra root bus that can be associated
> with a specific guest NUMA node using the ACPI tables. 
>  
> > I think we don't want it on Power, but I'm trying to make sure.
> 
> Do you have a solution for the problem above?

You should be able to achieve the same end result on ppc64
guests by creating additional PHBs (spapr-pci-host-bridge):
Bug 1280542 is about adding support for this to libvirt.
Comment 4 David Gibson 2016-12-06 23:50:53 UTC 
Right.  i.e. we have a solution on the qemu side, but not yet the libvirt side.

Well.. we do upstream, in RHEL7.3, I believe you can add PHBs, but you can't set their NUMA node.  That's fixed upstream so it should be in RHEL 7.4 as well.
Comment 5 David Gibson 2017-01-06 04:30:42 UTC 
So we can track them separately, move the EHCI related stuff to bug 1410674.  Refocus this bug purely on removing the pxb.
Comment 6 David Gibson 2017-01-06 04:34:49 UTC 
The PXB is currently included unconditionally upstream, so it will need to be made conditional there, rather than only requiring a downstream config change.
Comment 7 David Gibson 2017-01-06 07:10:08 UTC 
I've posted an RFC patch for this upstream:

https://lists.gnu.org/archive/html/qemu-devel/2017-01/msg00715.html
Comment 8 David Gibson 2017-02-09 00:13:07 UTC 
This is now merged upstream, so we should get it in the rebase to 2.9.

Qunfang, can we get a QA ack for this: this should reduce the overall testing load, since all the PXB related tests can be dropped.
Comment 10 Qunfang Zhang 2017-05-04 09:14:44 UTC 
Verified that PXB device existed on qemu-kvm-rhev-2.6.0-28.el7_3.9.ppc64le.rpm, and removed on qemu-kvm-rhev-2.9.0-2.el7.ppc64le.

For old qemu-kvm-rhev-2.6.0-28.el7_3.9.ppc64le: 

# /usr/libexec/qemu-kvm -device ?
Controller/Bridge/Hub devices:
name "pci-bridge", bus PCI, desc "Standard PCI Bridge"
name "pci-bridge-seat", bus PCI, desc "Standard PCI Bridge (multiseat)"
name "pxb", bus PCI, desc "PCI Expander Bridge"
name "pxb-pcie", bus PCI, desc "PCI Express Expander Bridge"
name "spapr-pci-host-bridge", bus System
name "spapr-pci-vfio-host-bridge", bus System
name "usb-host", bus usb-bus
name "usb-hub", bus usb-bus

For qemu-kvm-rhev-2.9.0-2.el7.ppc64le:

# /usr/libexec/qemu-kvm -device ?
Controller/Bridge/Hub devices:
name "pci-bridge", bus PCI, desc "Standard PCI Bridge"
name "pci-bridge-seat", bus PCI, desc "Standard PCI Bridge (multiseat)"
name "spapr-pci-host-bridge", bus System
name "spapr-pci-vfio-host-bridge", bus System
name "usb-host", bus usb-bus
name "usb-hub", bus usb-bus

Setting to VERIFIED.
Comment 12 errata-xmlrpc 2017-08-01 23:39:45 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392
Comment 13 errata-xmlrpc 2017-08-02 01:17:23 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392
Comment 14 errata-xmlrpc 2017-08-02 02:09:23 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392
Comment 15 errata-xmlrpc 2017-08-02 02:50:09 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392
Comment 16 errata-xmlrpc 2017-08-02 03:14:52 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392
Comment 17 errata-xmlrpc 2017-08-02 03:35:00 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392
Note You need to log in before you can comment on or make changes to this bug.