Created attachment 1622368 [details]
reproducer program

Is this reproducible with the "modesetting" driver? If you uninstall xorg-x11-drv-intel
(which may also remove xorg-x11-drivers, which is harmless) and restart X you should
fall back to the generic modesetting driver.

The modesetting driver does have some limitations relative to the intel driver, so this
may not be an acceptable workaround, but it would help to isolate where the guilty code
change is to compare the two.

It is not reproducible if I use "nomodeset" on the grub line.  However, the limitations we get with that make it an unacceptable workaround.  I have not tried uninstalling the driver as you suggest.  Will try that next.

(In reply to Benjamin Winter from comment #9)
> It is not reproducible if I use "nomodeset" on the grub line.  However, the
> limitations we get with that make it an unacceptable workaround.  I have not
> tried uninstalling the driver as you suggest.  Will try that next.

Ping, any update on this?

It appears to be reproducible when I uninstall the xorg-x11-drv-intel driver.  Similar long execution time.

Another data point - I used an HP Z420 system with an NVidia Quadro K600 graphics card using the xorg-x11-drv-nouveau driver.
RHEL 7.6 - 6m22.365s
nomodeset - 1m40.183s
yum erase xorg-x11-drv-nouveau - 6m15.549s
nomodeset - 1m40.133s

So... I have questions.

The two sosreports attached to this bug are actually comparing two entirely different drivers, not just two builds of the intel driver. In both cases Xorg.0.log says that the server version is 1.17.2-22.el7, which corresponds to 7.3. The "new" configuration fails to load the intel driver at all:

[    14.689] (II) Loading /usr/lib64/xorg/modules/drivers/intel_drv.so
[    14.696] (II) Module intel: vendor="X.Org Foundation"
[    14.696] 	compiled for 1.20.0, module version = 2.99.917
[    14.697] 	Module class: X.Org Video Driver
[    14.697] 	ABI class: X.Org Video Driver, version 24.0
[    14.697] (EE) module ABI major version (24) doesn't match the server's version (19)
[    14.697] (II) UnloadModule: "intel"
[    14.697] (II) Unloading intel
[    14.697] (EE) Failed to load module "intel" (module requirement mismatch, 0)

And from that point it proceeds with the generic "modesetting" driver. Now, I don't quite understand how you got to that point, because the driver and server packages have rpm-level version interlocks that should have prevented you from installing a driver that's incompatible with the installed server. I suspect what happened was:

- Test with 7.6, notice performance was bad
- Backrev to 7.3, find performance acceptable
- Copy the intel_drv.so from the 7.6 package into an otherwise 7.3 system
- 7.3's xserver refuses to load a 7.6 driver, falls back to modesetting

Now, in 7.4, we changed the default driver for _most_ Intel graphics chips to be modesetting instead of intel, as noted in the 7.4 release notes:

https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/7.4_release_notes/new_features_desktop

We did this for a number of reasons but primarily because Intel have largely failed to make releases for their driver, which means we've been on our own for testing and validation of arbitrary git snapshots. (You can see this reflected in the xorg-x11-drv-intel package version number. 2.99.917 was the last nominal release, in December 2014, and everything has been random snapshots since.) Since the modesetting driver uses roughly identical OpenGL-based acceleration paths for _all_ supported GPUs - be they Intel or AMD or NVIDIA or whatever - it made sense to have just one set of bugs. The problem then is that the intel driver is finely tuned for Intel's hardware, so performance suffers. Suffering by a factor of 7x like you're seeing here is admittedly quite a lot worse than expected.

The good news is that this change is only a default setting, it can be overridden by explicitly asking for the intel driver in /etc/X11/xorg.conf. The following snippet should be all you need to revert to using the intel driver on 7.4 and later:

Section "Device"
    Identifier "Integrated GPU"
    Driver "intel"
EndSection

Please give that a try and see if it restores performance to an acceptable level.

Yes, this works and gives me the proper performance again.  Thanks!

This is what I did:
1) Install a fresh copy of RHEL 7.6
2) Install fltk since the reproducer program uses libfltk
3) Run the reproducer program - it took 12 minutes to run
4) Create an /etc/X11/xorg.conf.d/test.conf file with the device section specifying the intel driver. Reboot
5) Run the reproducer program - it took 1 min 40 secs to run

So we clearly need to continue using the intel driver for performance.  The slowness of the modesetting driver is unacceptable.  What is the support path for the intel driver in the future?
How do I make a similar change for RHEL 8 since RHEL 8 uses Wayland instead of Xorg?

> So we clearly need to continue using the intel driver for performance.
> The slowness of the modesetting driver is unacceptable.  What is the
> support path for the intel driver in the future?

The intel driver is supported, but not the default, for both RHEL 7 and 8. I suspect that part of the performance problem here is that Intel GPUs underwent a major architecture shift between the Cantiga you're using here, where the memory controller and GPU live on the northbridge, and Sandybridge, where they live on the same die as the CPU. OpenGL performance got _way_ better between those two points as a result, and the acceleration path used in both modesetting and the Xwayland server is OpenGL-based.

> How do I make a similar change for RHEL 8 since RHEL 8 uses Wayland instead of Xorg?

The wayland session is the default in RHEL8, but the Xorg session is still available. So the change would be to use the same xorg.conf file as in RHEL7, and also install gnome-session-xsession and select it from the gdm login screen (little gear icon either next to the login button or in the bottom right corner, I think, and I'm sure there's a gsettings(1) invocation you can make to apply that choice automatically at install time.).

Now, Cantiga isn't technically supported in RHEL8 (the minimum was Haswell IIRC), so the question is somewhat academic: any machine you'd be running RHEL8 would post-date the above architecture changes. But it's entirely likely that you'd encounter similar but maybe lesser performance problems even on a supported GPU.

I'll try to get the reproducer program running on a newer machine using the GL-based acceleration code to see which path through the X server it's taking, there's probably some optimizations we can make to speed things up.

Red Hat Enterprise Linux 7 shipped it's final minor release on September 29th, 2020. 7.9 was the last minor releases scheduled for RHEL 7.
From intial triage it does not appear the remaining Bugzillas meet the inclusion criteria for Maintenance Phase 2 and will now be closed. 

From the RHEL life cycle page:
https://access.redhat.com/support/policy/updates/errata#Maintenance_Support_2_Phase
"During Maintenance Support 2 Phase for Red Hat Enterprise Linux version 7,Red Hat defined Critical and Important impact Security Advisories (RHSAs) and selected (at Red Hat discretion) Urgent Priority Bug Fix Advisories (RHBAs) may be released as they become available."

If this BZ was closed in error and meets the above criteria please re-open it flag for 7.9.z, provide suitable business and technical justifications, and follow the process for Accelerated Fixes:
https://source.redhat.com/groups/public/pnt-cxno/pnt_customer_experience_and_operations_wiki/support_delivery_accelerated_fix_release_handbook  

Feature Requests can re-opened and moved to RHEL 8 if the desired functionality is not already present in the product. 

Please reach out to the applicable Product Experience Engineer[0] if you have any questions or concerns.  

[0] https://bugzilla.redhat.com/page.cgi?id=agile_component_mapping.html&product=Red+Hat+Enterprise+Linux+7