Bug 1874282 - dbus-x11 could start per-connect session bus when user does ssh X11 DISPLAY forwarding
Summary: dbus-x11 could start per-connect session bus when user does ssh X11 DISPLAY f...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: dbus
Version: 8.2
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: rc
: 8.0
Assignee: David King
QA Contact: Petr Schindler
URL:
Whiteboard:
: 1686892 (view as bug list)
Depends On:
Blocks: 1916124 1916125
TreeView+ depends on / blocked
 
Reported: 2020-08-31 21:55 UTC by Andrew Mike
Modified: 2022-04-13 17:34 UTC (History)
28 users (show)

Fixed In Version: dbus-1.12.8-12.el8
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1916124 1916125 (view as bug list)
Environment:
Last Closed: 2021-05-18 15:05:03 UTC
Type: Bug
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github flatpak xdg-desktop-portal-gtk issues 72 0 None closed Unable to init server: Could not connect: Connection refused 2021-02-16 16:28:05 UTC
Red Hat Knowledge Base (Solution) 5477871 0 None None None 2020-10-16 05:20:57 UTC
Red Hat Product Errata RHBA-2021:1670 0 None None None 2021-05-18 15:05:17 UTC

Internal Links: 1940348

Description Andrew Mike 2020-08-31 21:55:12 UTC
Description of problem: When certain GNOME applications start over X11-forwarded SSH, the startup of the application is delayed by 20-30 seconds.


Version-Release number of selected component (if applicable):
xdg-desktop-portal-gtk-1.4.0-1.el8.x86_64

How reproducible: 100%


Steps to Reproduce:
1. Log into to a system via SSH with the -Y option.
2. Launch a GNOME application like gedit or nautilus.

Actual results: Application starts after 20-30 seconds of delay, with the following error observed in journalctl:

Aug 31 17:50:13 localhost.localdomain xdg-desktop-por[4151]: Failed to create app chooser proxy: Error calling StartServiceByName for org.freedesktop.impl.portal.desktop.gtk: Timeout was reached
Aug 31 17:50:13 localhost.localdomain xdg-desktop-por[4151]: No skeleton to export
Aug 31 17:50:28 localhost.localdomain systemd[3597]: xdg-desktop-portal.service: Start operation timed out. Terminating.
Aug 31 17:50:28 localhost.localdomain systemd[3597]: xdg-desktop-portal.service: Failed with result 'timeout'.
Aug 31 17:50:28 localhost.localdomain systemd[3597]: Failed to start Portal service.

Expected results: Application starts instantly without error.


Additional info:
- The error seems to be related to this upstream bug: https://github.com/flatpak/xdg-desktop-portal-gtk/issues/72
- Running "dbus-update-activation-environment --systemd DBUS_SESSION_BUS_ADDRESS DISPLAY XAUTHORITY" fixes the issue, as explained in the upstream bug.
- Firefox seems unaffected; it might only apply to GNOME applications, or applications aware of xdg-desktop-portal.

Comment 1 Ray Strode [halfline] 2020-09-01 19:13:00 UTC
So to fix this, I think we need hooks from ssh.

Something like /etc/ssh/sshrc but for distro use (/usr/share/ssh/sshrc.d/* ?).

Ideally, we'd be able to:

1. set GDK_BACKEND=x11 environment variable
2. run dbus-launch to start a session bus for just this ssh connection

Note we would need some way to know the user is doing X forwarding, since we don't want to do any of the above if the user isn't forwarding the display.

But, I think that should be enough to sufficiently disconnect the app from the user session that things work.

Of course, we could also potentially use the same mechanism to do the `dbus-update-activation-environment` call mentioned in comment 0 to put the new DISPLAY in the user bus and systemd user manager environment, but I don't think that's quite a nice of a solution.

Jakub, is this something we could potentially get in 8.4 ? Basically, we just want to be able run a script when the user connects that let's us tweak the environment the shell is run in, and run some commands.

Comment 3 Ray Strode [halfline] 2020-09-01 19:36:23 UTC
hmm, thinking about this more, we might be able to get away with profile.d snippets. something like:

/etc/profile.d/set-up-ssh-forwarding-session.sh

[ -z "$SSH_CONNECTION" ] && return
[ -z "$DISPLAY" ] && return
[ "${DISPLAY:0:1}" = ":" ] && return
[ "$SHLVL" -ne 1 ] && return

export GDK_BACKEND=x11
eval `dbus-launch --sh-syntax --exit-with-session`

Comment 4 Jakub Jelen 2020-09-02 08:09:31 UTC
If I remember well, I was filling some bug about handling of X11 forwarding when we planned RHEL8. But I do not remember the outcome, if there was any.

What you suggest in the profile.d scripts makes sense for me. I believe it should not be shipped by ssh, but by some gnome application/desktop environment (it is probably up to you to chose where it will be more appropriate) so headless servers do not need to go through these hops. If it will be shipped by something that does not depend on dbus-launch, checking if the executable is in place before calling exec might make sense too.

It would deserve some more testing before we would push this. As far as I remember, there is bunch of applications running in client/server mode (gnome-terminal), which might be solved by the above, but might need some other hops to work as expected.

Comment 6 Ray Strode [halfline] 2020-09-02 14:30:45 UTC
*** Bug 1686892 has been marked as a duplicate of this bug. ***

Comment 7 Dan Astoorian 2020-09-02 14:45:37 UTC
The dbus-update-activation-environment invocation updates the variables in the DBUS instance, but note that if the user is already logged into the graphical console in addition to the SSH session, updating those variables may interfere with the existing console session.  (E.g., after running dbus-update-activation-environment from the X11-forwarded SSH session, applications launched on the console desktop via DBUS may be started in the X11-forwarded session rather than on the console.)

Comment 8 Ray Strode [halfline] 2020-09-02 15:03:44 UTC
That's true, which is why i've suggested an alternative in comment 1.  comment 1 was unintentionally private before, so I've opened it up now.

Comment 9 Ray Strode [halfline] 2020-09-02 15:05:10 UTC
(though, in general running multiple simultaneous "sessions" like this for the same user is iffy, and not well supported)

Comment 14 Debarshi Ray 2020-09-18 10:30:49 UTC
(In reply to Ray Strode [halfline] from comment #3)
> hmm, thinking about this more, we might be able to get away with profile.d
> snippets. something like:
> 
> /etc/profile.d/set-up-ssh-forwarding-session.sh
> 
> [ -z "$SSH_CONNECTION" ] && return
> [ -z "$DISPLAY" ] && return
> [ "${DISPLAY:0:1}" = ":" ] && return
> [ "$SHLVL" -ne 1 ] && return

Should be:
 [ "$SHLVL" -ne 2 ] && return

> export GDK_BACKEND=x11
> eval `dbus-launch --sh-syntax --exit-with-session`

Comment 15 Ray Strode [halfline] 2020-09-18 20:43:35 UTC
(In reply to Debarshi Ray from comment #14)

> > [ "$SHLVL" -ne 1 ] && return
> 
> Should be:
>  [ "$SHLVL" -ne 2 ] && return
Not sure I'm following.

We only want to run the script if it's an unnested shell.  If the user runs "su" or something, we don't want to run the script again.

So SHLVL has to be "1" if we want to run the script.  If it's not equal to 1 the script should return without doing anything.

am I missing something?

Comment 16 Debarshi Ray 2020-09-22 10:48:24 UTC
(In reply to Ray Strode [halfline] from comment #15)
> (In reply to Debarshi Ray from comment #14)
> 
> > > [ "$SHLVL" -ne 1 ] && return
> > 
> > Should be:
> >  [ "$SHLVL" -ne 2 ] && return
> Not sure I'm following.

I might be guilty of not properly testing the deployed code.

I merely played around with the snippet in an interactive terminal and threw it at a customer, who mentioned that it had to be '2' for the snippet to take effect. I then echo:ed SHLVL from a throwaway shell script that I ran from an interactive shell. It came out as '2'.

Sorry for the noise. :/

Comment 17 David King 2020-09-22 11:35:27 UTC
Broadly, I do not mind having this snippet provided in dbus-x11, but I have a few reservations.

For RHEL9, switching to dbus-broker is, presumably, desirable, as that would be the logical outcome of an import from recent versions (F31, and onwards, I think) Fedora, where the default was changed there. If that is the case (and if it has not been discussed or considered, now is probably the time to do so), where does that leave dbus-x11, and should it continue to be installed as described in comment #12?

This change uses dbus-launch instead of the more modern dbus-run-session, which in its man page specifically describes a use case of login shells over SSH. Should dbus-run-session be used instead?

I am not convinced that a new dbus session bus per SSH session is the best way to set this up. Are we planning on using the user bus, as opposed to the session bus, for RHEL9? If that is the plan, then making this addition (or behaviour change, if you prefer) to RHEL8 would seem a little harder to undo during the RHEL9 transition.

I am not against a change like this, but I would prefer we push users towards exporting a complete, functional session over the network (such as through Vino or gnome-remote-desktop_, rather than claiming that doing the with X11-forwarding over SSH is well-supported.

Comment 37 Petr Schindler 2021-01-11 20:16:47 UTC
Our test suite passes for all architectures. Applications over ssh start as expected. I couldn't able to reproduce this bug with dbus-1.12.8-12.el8.x86_64.rpm

Comment 48 Joe Wright 2021-02-26 18:37:48 UTC
Responses inline:
#####################################################################
>   - Can you ensure that /etc/profile.d/ssh-x-forwarding.sh is on the system?
yes, it is.

>   - What shell are you using?
/bin/bash

>   - What is the value of GDK_BACKEND before you explicitly set it to x11?

It seems to be unset when using FastX, but set when using "ssh -X ..." . gnome-terminal fails over ssh when GDK_BACKEND=x11, . FastX works with  GDK_BACKEND=x11

>   - Does it work for any other users on the same system?

Same behavior for other users.
#####################################################################

Attaching requested output files as well

Comment 52 Dan Astoorian 2021-04-14 22:34:49 UTC
This change in dbus-1.12.8-12 appears to be responsible for Bug 1940067.

Comment 53 schanzle 2021-04-16 13:57:10 UTC
(In reply to Dan Astoorian from comment #52)
> This change in dbus-1.12.8-12 appears to be responsible for Bug 1940067.

Agreed.  This change breaks ssh for common use cases (including rsync) for users with "ForwardX11 yes" in their ~/.ssh/config.  This is clearly a regression.  Should it be promptly reverted?  Does it meet customer expectations of "Enterprise Linux"?  As I understand it, testing and refinement should be done in Fedora and/or CentOS Stream, not RHEL8.

Comment 54 Paulo Andrade 2021-04-22 16:50:19 UTC
  Why the ssh-x-forwarding.sh has --exit-with-session, what consumes input and
will race with the shell, and ssh-x-forwarding.csh has --exit-with-x11.

  Maybe it should be better to tell users to run:

$ dbus-launch gnome-terminal

and so on on ssh -X connections; then it would be required to fix why it
hangs at exit.

  Also, this is an issue for upstream, but --exit-with-session likely should
use another approach to detect a SIGHUP in stdin file descriptor. As is now
it will race with the shell for input, and users will see only a few of the
characters they type actually go to the terminal.

Comment 55 Paulo Andrade 2021-04-22 17:09:32 UTC
  Untested, but maybe could do something like this, with a new option
specific for ss-x-forwarding.{c,}sh. Instead of the babysit function,
that races for tty input with the shell, have something like this:

  ino_fd = inotify_init();
  inotify_add_watch(ino_fd, "/dev/stdin", IN_CLOSE_NOWRITE);
  << proceed as inotify(7) example on a loop, waiting for the close event >>

do proper error checking, exiting at any failure, then just wait for the close
event, or, more simple, and somewhat like what it is already doing, spawn a
new shell, wait for it to exit, then just close stdin itself, so that the
parent shell will also exit.

Comment 56 Ray Strode [halfline] 2021-04-22 18:30:55 UTC
Okay, I think it's now obvious relying on dbus-launch to kill the dbus-daemon automatically when the session exits isn't going to work.

The ssh connection will keep the X connection alive until it has no more clients, and dbus-launch is a client itself.

This means dbus-launch relies on detecting hang ups on the terminal to know when to exit, but in the case of single command ssh invocations, there's on terminal.
 
My first thought was add [ -z "$SSH_TTY" ] && return to the top of the profile.d script, but that would obviously cause a regression for the original fix in the single command case.

And the "eating keystrokes" problem Paulo has found makes it clear it's not even a good idea in the case where there is a tty.
Probably rather than relying on the tty, we should rely on the logind session, which is what the dbus-daemon should really be scoped to, since it actually defines the scope of the session.

I think if we're going to fix this with just shell script changes, we could do something like:

╎❯ cat /etc/profile.d/ssh-x-forwarding.sh
[ -z "$SSH_CONNECTION" ] && return
[ -z "$DISPLAY" ] && return
[ "${DISPLAY:0:1}" = ":" ] && return
[ "$SHLVL" -ne 1 ] && return

export GDK_BACKEND=x11

setsid -f /usr/libexec/dbus-1/dbus-start-ssh-session-bus

and then,

╎❯ /usr/libexec/dbus-1/dbus-start-ssh-session-bus

#!/bin/bash
exec >& /dev/null
eval `dbus-launch --sh-syntax`
while inotifywait /run/systemd/sessions/$XDG_SESSION_ID; do
        session_state=$(loginctl show-session $XDG_SESSION_ID | sed -n 's/^State=\(.*\)/\1/p')
        [ "$session_state" = "active" ] && continue
        kill -TERM $DBUS_SESSION_BUS_PID
        break
done

But it might be better to fix dbus-launch --exit-with-session to look at a logind session first and foremost if it's getting run in one.

Comment 57 Ray Strode [halfline] 2021-04-22 18:56:05 UTC
actually this might be a little more fault tolerant:

#!/bin/bash
exec >& /dev/null
eval `dbus-launch --sh-syntax`

while
    session_state=$(loginctl show-session $XDG_SESSION_ID | sed -n 's/^State=\(.*\)/\1/p')
    [ "$session_state" = "active" ]
do
        inotifywait /run/systemd/sessions/$XDG_SESSION_ID
done

kill -TERM $DBUS_SESSION_BUS_PID

(since it's unlikely, though possible, that the logind session could exit as soon as it starts before this script runs)

Comment 59 errata-xmlrpc 2021-05-18 15:05:03 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (dbus bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:1670

Comment 60 schanzle 2021-05-18 19:43:27 UTC
(In reply to Ray Strode [halfline] from comment #57)
...
> while
>     session_state=$(loginctl show-session $XDG_SESSION_ID | sed -n
> 's/^State=\(.*\)/\1/p')
>     [ "$session_state" = "active" ]
> do
>         inotifywait /run/systemd/sessions/$XDG_SESSION_ID
> done


Could the 'while' expression/list be simplified to:

while loginctl show-session $XDG_SESSION_ID | grep -q ^State=active; do
...

Relying on the exit value of grep and eliminating the session_state variable (hopefully 'unset' later in the final code).  Sorry for the late suggestion.  Perhaps for future release.

Comment 61 Simon Matter 2021-05-27 06:52:43 UTC
I'm not sure how all the above has been fixed according to https://access.redhat.com/errata/RHBA-2021:1670 when it says it's fixed in dbus-1.12.8-12.el8.

I've tested dbus-1.12.8-12.el8 and now also the newer dbus-1.12.8-12.el8_4.2 and it doesn't seem to fix the above because it still has the faulty snippets in /etc/profile.d/ssh-x-forwarding.[c]sh.

Am I missing something?

Comment 62 Ray Strode [halfline] 2021-06-01 20:16:01 UTC
Simon, I believe the remaining issues are being tracked by bug 1940067


Note You need to log in before you can comment on or make changes to this bug.