Bug 1289714 - find a solution for XWayland games trying to set display resolution
find a solution for XWayland games trying to set display resolution
Status: NEW
Product: Fedora
Classification: Fedora
Component: xorg-x11-server (Show other bugs)
rawhide
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: X/OpenGL Maintenance List
Fedora Extras Quality Assurance
: FutureFeature
: 1286217 (view as bug list)
Depends On:
Blocks: WaylandRelated
  Show dependency treegraph
 
Reported: 2015-12-08 13:59 EST by Kamil Páral
Modified: 2017-12-08 15:03 EST (History)
9 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
xwayland patch to enable some common resolutions (1.96 KB, patch)
2017-11-20 10:26 EST, Robert Mader
no flags Details | Diff
Translate the mouse position when fake resolution is set (1.01 KB, patch)
2017-11-20 15:18 EST, Robert Mader
no flags Details | Diff
Translate the mouse position when fake resolution is set (1.72 KB, patch)
2017-11-24 08:39 EST, Robert Mader
no flags Details | Diff
Hack to make mutter fullscreen all games (unclean solution) (546 bytes, patch)
2017-11-24 08:59 EST, Robert Mader
no flags Details | Diff
mutter patch to expose some common resolutions (WIP) (1007 bytes, patch)
2017-12-07 11:34 EST, Robert Mader
no flags Details | Diff
xwayland patch to accept fake modesetting (446 bytes, patch)
2017-12-07 11:38 EST, Robert Mader
no flags Details | Diff
implement wp_viewporter (WIP) (27.94 KB, patch)
2017-12-08 13:24 EST, Robert Mader
no flags Details | Diff

  None (edit)
Description Kamil Páral 2015-12-08 13:59:28 EST
This is the year or Linux gaming. We now have thousands of games available for Linux thanks to Steam, and Steam machines running SteamOS (Linux) are sold worldwide. All of those games out there are X11-based. Some of them will get updated for Wayland, but most never will. If we want to push Wayland as the default display technology, we need to make sure those games work.

One of the core issues is display resolution. Game developers and players are used to games changing their display resolution (especially 3D games) in order to achieve satisfactory performance. That is no longer possible in Wayland, and there are good reasons for that, and solutions (scaling the ouput). However, we also need a solution for XWayland games, which is the majority.

Under XWayland, there seems to be only a single resolution available when querying display through xrandr - the current resolution - and no custom resolution can be set.

According to my short testing, there are several ways how games currently cope with that (all of this covers fullscreen behavior only, I haven't seen any issues when running in a windowed mode):

A) Run in a letterbox fixed to a single resolution - the game in enclosed in black bars around the screen (corresponding to the game's default resolution), no change can be made
Examples: OpenArena

B) Run in a letterbox, but allow different resolutions to be set, adjusting letterbox - different resolutions change the rendering size and the size of the black bars around it
Examples: Extreme Tux Racer, Battle for Wesnoth

C) Run in a desktop resolution - you can either see desktop resolution as the only available resolution, or you can see other resolutions available, but nothing happens when you try to set it, you stay fixed to the current one. This is basically the same case as A), only the game default resolution seems to be your desktop resolution. Internal game logic probably differentiates A) and C).
Examples: 0 A.D., Xonotic, Supertuxkart, Neverball

D) Run scaled in a desktop resolution - only single resolution is available, but the game output is scaled as opposed to letterboxing
Examples: Half-Life

E) Use internal scaling instead of relying on system resolutions - this is what some modern games do. Instead of changing the monitor resolution, keep it intact, just adjust the internal rendering resolution and then either scale up the output or display a smaller game area.
Examples: Pillars of Eternity, Gnomoria

F) Crash - when configuring the game in the better case, on startup in the worst case
Examples: Minecraft (according to bug 1287864, not tested)


Only E) is the ideal solution for users of Wayland desktop. A) and B) make the game screen small or tiny, C) and D) don't allow to tune the game performance to available hardware.

There is a simple yet tiresome workaround for B) and C) - switch desktop resolution manually to a desired size, run the game, switch resolution manually back afterwards. I don't know about any workarounds for A), D) and F).


Please try to find a solution for this. If we can't make the crushing majority of games work, Wayland will not convert a large portion of our user base (anyone who sometimes plays a game), they will stay with X11.

For example, could we fake the resolution list for XWayland apps and if they try to set a custom fullscreen resolution, could we tell them it succeeded, but scale their output instead? (similarly to what we intend to do with Wayland-based games)
Comment 1 Kamil Páral 2015-12-08 14:01:17 EST
*** Bug 1286217 has been marked as a duplicate of this bug. ***
Comment 2 Olivier Fourdan 2015-12-16 09:09:03 EST
(In reply to Kamil Páral from comment #0)
> F) Crash - when configuring the game in the better case, on startup in the
> worst case
> Examples: Minecraft (according to bug 1287864, not tested)

Bug 1287864 is a bug in Minecraft itself, not something we can fix.
Comment 3 Olivier Fourdan 2016-04-07 03:52:55 EDT
(In reply to Olivier Fourdan from comment #2)
> Bug 1287864 is a bug in Minecraft itself, not something we can fix.

Actually, a workarround for this was just added to Xwayland, included in xserver 1.18.3 so Minecraft should start in Wayland now (upstream).
Comment 4 Kamil Páral 2016-09-12 09:35:15 EDT
This is still an issue in F25 (e.g. with openarena I can't change its resolution at all).
Comment 5 Olivier Fourdan 2016-09-12 10:41:14 EDT
Well, for games using Xwayland this feature was denied:

https://fedoraproject.org/wiki/Wayland_features#XRandR_control_of_Wayland_outputs
Comment 6 Sean Young 2016-09-12 16:50:47 EDT
That comment said it should be implemented in mutter, not Xwayland.
Comment 7 Kamil Páral 2016-09-13 07:07:42 EDT
(In reply to Olivier Fourdan from comment #5)
> Well, for games using Xwayland this feature was denied:

Yes, and I understand why. However, the purpose of this ticket was whether we can find a different solution to this (because X11-based games will be with us for a long time still). For example:

(In reply to Kamil Páral from comment #0)
> For example, could we fake the resolution list for XWayland apps and if they
> try to set a custom fullscreen resolution, could we tell them it succeeded,
> but scale their output instead? (similarly to what we intend to do with
> Wayland-based games)

Is that doable (or something similar)? Is there a will and time to implement it?
Comment 8 Robert Mader 2017-11-02 10:02:57 EDT
I just like to add that a scaling solution as mentioned above would also heavily benefit wine games, probably a big factor.
Comment 9 Robert Mader 2017-11-03 07:32:47 EDT
Ok I started tinkering on this for fun and it looks surprisingly simple.
I changed xwayland in the xserver so it allows 1024x768 as resolution and mutter plays along quite nicely.
When changing the resolution in Warcraft 3, it properly resizes to that.
When minimizing the game and selecting again (this is not 100% reproducibly atm and probably a bug in mutter), the output even gets scaled up!

The input handling is limited to the official size of the window (1024x768 instead of 1366x768) thought, so I can't move the mouse to the far right anymore.

I'll try if I can make that behavior explicit in mutter and fix the input.

For those interested, here's some screenshots:
https://cloud.silentundo.org/s/7HnuNToHnvRBfcj
Comment 10 Robert Mader 2017-11-13 12:57:37 EST
Little update:

If I understand everything correctly, mutter on X11 (or Xorg itself?) currently remembers randr settings for each application and sets them on focus change (e.g. taping out and back again).

We can maybe mirror that behaviour for xwayland, but instead of setting the resolution just apply a resize to the interface the application renders to and some sort of mouse coordinate wrapper.
I'll tinker on that a bit but I'm really slow as it's still hard for me how everything interacts and how many different ways there are to set the resolution.


The most clean solution would of course be to have one xwayland instance per application, but that will probably not happen anytime soon.
Comment 11 Robert Mader 2017-11-16 09:54:38 EST
Ok I was totally wrong, the app always sets the resolution itself.

So the idea would be as follows: 
- make xwayland accept randr mode setting (not really setting it but remembering and reporting it on future request).
- make mutter check the set resolution when it sets an x-window to fullscreen on wayland. If the resolution is not the native one, take only the output of the reported resolution and scale it up to native size
- create a shim layer for mouse input that reports the current mouse position accordingly
Comment 12 Fedora End Of Life 2017-11-16 13:37:06 EST
This message is a reminder that Fedora 25 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 25. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '25'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 25 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged  change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.
Comment 13 Robert Mader 2017-11-20 10:26 EST
Created attachment 1355915 [details]
xwayland patch to enable some common resolutions

This is a small patch for xwayland to enable some common resolutions make xwayland accept/save the settings.
The rest probably has to get handled by the compositor (I'm focusing on mutter)
Comment 14 Robert Mader 2017-11-20 10:53:42 EST
With the patch above applied, most applications somewhat work.

When fullscreened by mutter, all 3d-games I tested so far get scaled up well. What remains to be done here:
- when a fake resolution is set, mutter often doesn't set the fullscreen flag for the corresponding windows (or unsets them)
- mouse input is somewhat broken, needs some translation layer
- black-bar mode for resolutions not fitting the screen (should be relatively easy)


The 2d-games I've tested don't get scaled up right now. But that might have to do with the fact that all of them use legacy fullscreening, therefore mutter doesn't know how big they're supposed to be. If mutter knew, it should be easy.
Comment 15 Olivier Fourdan 2017-11-20 11:29:38 EST
I don't think making up modes in Xwayland is the right approach, it should retrieve those from the wl_output::mode given by the Wayland compositor (gnome-shell in this case).

Right now, gnome-shell doesn't list any but the current one, so adding this in gnome-shell would be a start.

But then Xwayland as of now ignores all modes but the current one (!), so you'd need to fix Xwayland to take thoise into account and list them in the available modes in XRandR.

  https://cgit.freedesktop.org/xorg/xserver/tree/hw/xwayland/xwayland-output.c#n102

But even with that in place, Xwayland would need a way to notify the Wayland compositor that one of its X11 client with a surface fullscreen has requested a different mode. However, We do not want Xwayland to control the output mode though (see https://fedoraproject.org/wiki/Wayland_features#XRandR_control_of_Wayland_outputs) as any XRandR client (including the command xrandr itself) would be able to change the modes in Wayland.

PS: I think you'd get probably more feedback and traction by discussing this upstream on xorg-devel and wayland devel mailing lists rather than in a bug downstream.
Comment 16 Robert Mader 2017-11-20 15:11:40 EST
@Olivier Fourdan: thanks for the feedback, the reason I'm tracking it here is mainly for the fakt that it's not yet clear which components need to get changed (only xwayland or also mutter/the window manager).
But you're right.

For the other part: what I'm currently trying is exactly not to set the screen mode, as that is not supposed to happen under wayland. So xwayland should not tell the compositor to do that but only fake it.
The idea is to just scale up the output. That in turn makes it independent from the real modes the screen supports.
This feature overall is mostly about legacy support (old games which don't detect the actual screen resolution or don't know anything bigger than say 1024x768).

So we could add all supported screen resolutions, but I think it's important to make sure the most common ones are in the list (640x480, 800x600, 1024x768), as those basically are the ones which will get used.
Think of games like Diablo 2, Age of Empires 2, Counter Strike and the like.
Comment 17 Robert Mader 2017-11-20 15:18 EST
Created attachment 1356127 [details]
Translate the mouse position when fake resolution is set

This patch translates the mouse position accordingly to the fake resolution. It assumes that, when a fake resolution is set, the window is fullscreen (therefore scaled up to the full screen size).
That would be the usual case.
Comment 18 Robert Mader 2017-11-20 15:53:06 EST
Alright, with the two patches above everything on the xwayland side should be done, the rest probably has to get figured out on the compositor side (properly detecting when a window should get fullscreened, resetting the fake resolution if the application does not itself on quit etc.).

With these patches, I can already play most 3d-games with lower resolutions, like starcraft 2, source-engine based shooters (only the menu, shooters don't yet work in gnome/wayland, but that's another story) or the talos principle.

As soon as I have some mutter patches ready, I'll create a copr for those interested to test.
Comment 19 Robert Mader 2017-11-24 08:39 EST
Created attachment 1358665 [details]
Translate the mouse position when fake resolution is set
Comment 20 Robert Mader 2017-11-24 08:59 EST
Created attachment 1358678 [details]
Hack to make mutter fullscreen all games (unclean solution)
Comment 21 Olivier Fourdan 2017-11-29 09:29:38 EST
FWIW, I have just posted a patch for Xwayland to list all the wl_output::mode(s) advertised by the compositor as available XRandR modes, so that you wouldn't need to fake modes in Xwayland.

  https://patchwork.freedesktop.org/series/34628/

That works with weston which lists multiple available modes, but gnome-shell/mutter only lists the current mode.
Comment 22 Robert Mader 2017-12-07 11:34 EST
Created attachment 1364382 [details]
mutter patch to expose some common resolutions (WIP)

With Oliviers patch (https://patchwork.freedesktop.org/series/34628/), xwayland exposes modes send by mutter. 
This makes mutter expose three hard-coded common modes. Actually it should expose all modes it would list in system-setting AND these modes (they are the most important modes for old games).
Comment 23 Robert Mader 2017-12-07 11:38 EST
Created attachment 1364383 [details]
xwayland patch to accept fake modesetting

This makes xwayland fake/accept modes (WIP). Together with Oliviers patch and 1364382 (mutter patch to expose some common resolutions (WIP)), it makes it possible to launch games which don't work without modesetting, such as Diablo 2 over Wine.
Comment 24 Robert Mader 2017-12-07 11:45:44 EST
Little status update of the todo list:
- make mutter expose all solutions it would in system-settings, in addition to some very import ones which are hard-coded so far
- implement wl_viewport. There's some old work here: https://mail.gnome.org/archives/commits-list/2014-November/msg00700.html
- make mutter add viewports to xwayland windows which don't have the screen resolution and want to become fullscreen
- done?
Comment 25 Robert Mader 2017-12-08 13:24 EST
Created attachment 1364992 [details]
implement wp_viewporter (WIP)

This is based on old patches by Jasper St. Pierre, see
https://mail.gnome.org/archives/commits-list/2014-November/msg00799.html
and 
https://mail.gnome.org/archives/commits-list/2014-November/msg00800.html

This makes it work with gnome 3.26.2 and stabilizes it as wp_viewporter instead of wl_scaler.

It's not tested yet, but it builds :)
Comment 26 Robert Mader 2017-12-08 15:03:59 EST
External bug-tracker for wp_viewport: https://bugzilla.gnome.org/show_bug.cgi?id=791405

Note You need to log in before you can comment on or make changes to this bug.