Bug 2075132

Raising the Customer Escalation flag, per RME EN-50824 and the business impact provided by the customer,

Business impact: 
We have 2000+ Red Hat desktops in our private environment.  Several mission critical applications are unable to launch for our environment, making it impossible for our users to do their jobs.

We have custom X applications that rely on being able to present remotely to the desktop over the X ports (60xx).  We received a bug fix for GDM that actually fixes the ability to *turn on* the port, but now have a bug in the Xorg server that breaks the desktop completely (Xorg will not load) when IPv6 is disabled (which it is in our environment.)  We use nVidia video cards on all physical desktops and therefore cannot use Wayland.  We currently cannot proceed in our 8.5 deployment because of the mission critical application block.

2 bugs have been filed on this:  2029202 against GDM which has been fixed, which then led to bug 2075132 against libXTrans, which has not been fixed.  Until we get a build that allows TCP ports to listen appropriately on the Xorg server, we can't proceed with our 8.x rollout.  We need a build that actually works properly with IPv6 disabled.  Bug 2075132 has been marked as medium priority and medium severity, which seems completely off since it simply does not work at all with IPv6 disabled.

So the problematic code is here in the xserver:

```
void•
CreateWellKnownSockets(void)•
{•
 ...
    else { /* -displayfd and no explicit display number */•
        Bool found = 0;•
        for (i = 0; i < 65536 - X_TCP_PORT; i++) {•
            if (TryCreateSocket(i, &partial) && !partial) {•
                found = 1;•
                break;•
            }•
            else•
                CloseWellKnownConnections();•
        }•
...
    }•
...
}•
```

The `TryCreateSocket()` function not only returns whether or not it is able to create any of its sockets, it also returns if it is able to create all of its sockets, through the `partial` out variable.

This is an important distinction, because you could imagine a situation where e.g. a rogue X server is listening on a tcp port, but not on the local socket in /tmp. In that situation, it's important for the `-displayfd` code to disregard that potential display number, so that e.g., DISPLAY=:1 and DISPLAY=localhost:1 don't go to different X servers, defying user expectations.

But, unfortunately, `TryCreateSocket` also returns `partial=TRUE` in the case where it can bind to `AF_INET` but not `AF_INET6`.

I think the xtrans code needs to set the `TRANS_DISABLED` flag on the transport if it fails with `EAFNOSUPPORT` so it will get ignored when doing the `partial` handling.

Created attachment 1877647 [details]
proposed fix (updated)

This patch does what I proposed in the comment 3 with one minor change. I used the NOLISTEN flag instead of the DISABLED flag, because the DISABLED flag doesn't restrict it from the listener count when checking for how to set partial.

This attachment obsoletes attachment 1877644 [details] which was missing the actual patch in the patch.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: xorg-x11-server and xorg-x11-server-Xwayland security and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:7583

The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days