Bug 1283581 - Not possible to log in after upgrade to Fedora 23
Not possible to log in after upgrade to Fedora 23
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: lxdm (Show other bugs)
23
Unspecified Unspecified
unspecified Severity high
: ---
: ---
Assigned To: Christoph Wickert
Fedora Extras Quality Assurance
:
Depends On:
Blocks: 1268900
  Show dependency treegraph
 
Reported: 2015-11-19 05:58 EST by Tore Anderson
Modified: 2015-12-09 22:51 EST (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-12-06 01:58:28 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Tore Anderson 2015-11-19 05:58:02 EST
Description of problem:

After upgrading from Fedora 22 to 23, I can no longer log in to the graphical user interface. When I input a correct user name and password on the LXDM login screen, everything on the screen (except for the background image) briefly disappears, but after ~1 second the big "Login:" text, the username text input box, and the session/language selection bar near the bottom re-appears.

I'm normally using LXDE, but the same thing happens regardless of which X session I select, which makes me believe that this isn't a problem with the desktop environments / X sessions, but with LXDM itself. This assumption is further strengthened by the fact that I can work around the problem by simply downgrading LXDM (see below).

I am certain that LXDM is able to validate the password correctly. If I intentionally enter an incorrect password, the big "Login:" text and the session/language selection bar does not briefly vanish, so the observed behaviour is differs between using a correct password and an incorrect one.

The user in question can log in normally on the text console without any problems.

Version-Release number of selected component (if applicable):

lxdm-0.5.1-7-D20151007gite8f38708.fc23.x86_64

How reproducible:

100%

Steps to Reproduce:
1. Attempt to log in as a regular user on the LXDM login screen

Actual results:

Login fails, and the login screen re-appears within a second or so.

Expected results:

Login is successful.

Additional info:

SELinux is disabled, so this is not a duplicate of bug #1274169.

In LXDM's journal I can see it complains about not being able to open /usr/lib64/security/pam_gnome_keyring.so, which is not present on the system. Uncommenting the two relevant lines from /etc/pam.d/lxdm causes those errors to no longer appear, but other than that does not change anything. When attempting to log in with pam_gnome_keyring.so disabled, the following two lines appear in the LXDM journal:

pam_unix(lxdm:auth): authentication failure; logname= uid=0 euid=0 tty= ruser= rhost= user=tore
pam_sss(lxdm:auth): authentication success; logname= uid=0 euid=0 tty= ruser= rhost= user=tore

That pam_unix.so doesn't find the user is expected, since it's stored in an external LDAP tree, not in /etc/passwd. The user's home directory is a remote NFS mount.

Downgrading to lxdm-0.4.1-10.fc22.x86_64 solves the problem and allows me to log in to the system again. (I do not have to downgrade any other packages in order to make this happen.)
Comment 1 Mamoru TASAKA 2015-11-19 22:22:07 EST
It is hard to debug this unless I can find the detailed procedure to reproduce this issue.

> /usr/lib64/security/pam_gnome_keyring.so, which is not present
It is optional, and pam setting does not change on lxdm between 0.4.1 and 0.5.1 and 0.5.2.

Would you write in detail how to reproduce this issue? (i.e. would you have some way to reproduce this issue when you setup your system newly? If you can reproduce this with some procedure from the fresh F-23 install, it is highly appreciated.) 
Especially:

* I don't have NFS-used system. Can you find out if this is related to your NFS setup? Can you reproduce this even if you newly create another account without using NFS and login with that new user?
* Would you attach (as a log) the result of "$ rpm -qa | sort" ?

Or
* Would you have some analysis about to which stage LXDE session proceeded?
Comment 2 Tore Anderson 2015-11-20 01:54:22 EST
I will try over the weekend to see if I can make a procedure on how to reproduce from a fresh F-23 install.

It does seem related to NFS. If I unmount /home/tore and instead create an empty (local) directory owned by the "tore" user, I can log in successfully. The very interesting thing is that if I log back out, remount the NFS on /home/tore (from the console), I can successfully log back in. Strange, huh? If however I restart lxdm, it is again impossible to log in. So it seems that as long as I've logged in successfully once in the lifetime of the lxdm binary (using a local home directory), I can log in using an NFS home directory too.

I don't know how to debug to which stage the LXDE session proceeded. There is no manual page for the lxdm binary, nor does "lxdm --help" or "lxdm -h" give any output that suggests how to enable more debugging. If you have any pointers on how to do that, I would appreciate that.
Comment 3 Mamoru TASAKA 2015-11-20 02:32:29 EST
If you can have some way with which I can reproduce this issue, I can debug this issue.

Maybe if you can tell me some detailed procedure how you created your acount with NFS, especially
- your NFS configuration
- where you create your account
- and some other information you think which may be related

By the way, it is possible that you can log in with lxdm-0.5.1-6.D20150806git17ac3772.fc23 . Would you try this?
Comment 4 Tore Anderson 2015-11-20 03:34:00 EST
NFS configuration (from /etc/exports) looks like this (IP address anonymised):

[2001:db8::1234]:/srv/nfs/ms/home/tore	/home/tore	nfs	defaults,x-systemd.automount	0	0

The account is created in the company LDAP tree. Pretty standard stuff.

I tested lxdm-0.5.1-6.D20150806git17ac3772.fc23.x86_64.rpm, it works fine.

I also tested lxdm-0.5.2-1.fc23.x86_64.rpm, this version does not work either. So the bug must have been introduced somewhere between 17ac3772 and e8f38708.
Comment 5 Mamoru TASAKA 2015-11-20 04:02:13 EST
(In reply to Tore Anderson from comment #4)

> I tested lxdm-0.5.1-6.D20150806git17ac3772.fc23.x86_64.rpm, it works fine.
> 
> I also tested lxdm-0.5.2-1.fc23.x86_64.rpm, this version does not work
> either. So the bug must have been introduced somewhere between 17ac3772 and
> e8f38708.

Okay, actually this is only one commit:
http://git.lxde.org/gitweb/?p=lxde/lxdm.git;a=commitdiff;h=e8f387089e241360bdc6955d3e479450722dcea3

This is to fix bug 1268900 , but I may have to revert this.
I guess the upstream
http://sourceforge.net/p/lxde/bugs/786/
is the same phenomenon.
Comment 6 Mamoru TASAKA 2015-11-20 04:13:51 EST
So perhaps lxdm (or some other LXDE component) cannot write / read /var/run/lxdm or ~/.Xauthority (although currently I don't know the exact component / cause of this)
Comment 7 Tore Anderson 2015-11-20 04:19:01 EST
The description of upstream bug 786 does exactly describe my situation, indeed.

There is one thing that comes to mind about NFS home directories, namely that the "root" user does not have write access to it:

~$ LANG=C sudo touch foo
touch: cannot touch 'foo': Permission denied
~$ LANG=C touch foo
~$ 

I do not know if this is relevant, and I do not quite see why it would be from the commit diff you linked to either, but if LXDM attempts to write some file in my home directory (like ~/.Xauthority) before changing privileges, then that is going to fail.
Comment 8 Tomas Hoger 2015-11-20 05:10:19 EST
If I were to guess, I'd suspect problems with getting .Xauthority file updated.  Is it possible that lxdm is updating the file on local file system just to have it hidden by NFS mounted directory and hence inaccessible to your DE?  Or maybe it's not created at all, as you mention that creating local directory helps.

You can try uncommenting the following in lxdm.conf:

## set this if you don't want to put xauth file at ~/.Xauthority
# xauth_path=/tmp

and see if that makes a difference.  Of course, /tmp is a poor choice, as (based on a quick look) the code does not protect against symlink attacks.  So /var/run/lxdm/ would be safer if it can be used, i.e. if the code creating Xauthority file is run with sufficient privileges (I've not checked).

(In reply to Tore Anderson from comment #2)
> The very interesting thing is that if I log back out, remount the NFS on
> /home/tore (from the console), I can successfully log back in. Strange, huh?
> If however I restart lxdm, it is again impossible to log in. So it seems
> that as long as I've logged in successfully once in the lifetime of the lxdm
> binary (using a local home directory), I can log in using an NFS home
> directory too.

I'd guess xhost localuser authorization is what causes this.  In Fedora, xhost +si:localuser:`id -un` is run by xinit and lxdm does not restart X server between sessions (see bug 1269917), so the localuser authorization stays even after session logout, which has its own set of security concerns.
Comment 9 Tomas Hoger 2015-11-20 05:13:45 EST
(In reply to Tore Anderson from comment #7)
> I do not know if this is relevant, and I do not quite see why it would be
> from the commit diff you linked to either, but if LXDM attempts to write
> some file in my home directory (like ~/.Xauthority) before changing
> privileges, then that is going to fail.

The code changed by the commit uses auth file in /var/run/lxdm/lxdm-:%d.auth, but it's auth file used by lxdm, not by applications in your X session.  The default user Xauthority file is ~/.Xauthority, and you can use xauth_path to override.
Comment 10 Tore Anderson 2015-11-20 05:45:01 EST
Thanks Tomas. I did some more tests based on the information you added.

First, I made a directory "/rootonly" to which only root has write access:

drwxr-xr-x 2 root root 4096 nov.  20 11:30 /rootonly

Then I set "xauth_path=/rootonly" in lxdm.conf, upgraded to lxdm-0.5.1-7.D20151007gite8f38708.fc23.x86_64, and restarted LXDM. The NFS home directory was already mounted at this point.

I attempted to log in, which succeeded. The following file was created:

-rw------- 1 tore tore 71 nov.  20 11:30 /rootonly/.Xauth7097

From this we can surmise that the code that wrote this .Xauth7097 file was running as root (otherwise it would have been unable to create any file inside /rootonly/).

As I mentioned earlier, this is doomed to fail when the directory in question is on a NFS (except if the NFS is exported with the non-default option no_root_squash, cf. exports(5)). This is because the root user has no special "can freely write everywhere" privilege on NFS mounts. Thus, in order for LXDE to support creating Xauthority files inside users' home directories, it must drop prileges to the user about to log in before it writes the file.

I must admit I don't fully understand why commit e8f3870 breaks this, as it seems be about files inside /var/run/lxdm (which is not on NFS and has never been), but then again, I'm not familiar with the surrounding code.
Comment 11 Tomas Hoger 2015-11-20 06:54:25 EST
(In reply to Tore Anderson from comment #10)
> As I mentioned earlier, this is doomed to fail when the directory in
> question is on a NFS (except if the NFS is exported with the non-default
> option no_root_squash, cf. exports(5)). This is because the root user has no
> special "can freely write everywhere" privilege on NFS mounts.

Right, and it's likely the reason why lxdm provides xauth_path configuration option to allow storing Xauthority file outside of the NFS mounted home.

> Thus, in order for LXDE to support creating Xauthority files inside users'
> home directories, it must drop prileges to the user about to log in before
> it writes the file.

Or be configured to not store in home, as that's what it already can do?

> I must admit I don't fully understand why commit e8f3870 breaks this, as it
> seems be about files inside /var/run/lxdm (which is not on NFS and has never
> been), but then again, I'm not familiar with the surrounding code.

As noted in comment 9, the file mentioned in that commit is what's used by X server.  Before and after the commit, client default is still ~/.Xauthority.  The difference is that before that commit, X server is not started with -auth argument, and therefore accepts connection from any user able to connect to it, even without having valid authentication cookie.  Therefore, failure to create ~/.Xauthority is not fatal.  Apps in your X session can't provide X server with valid cookie, but it happily accepts their connections anyway.  After the commit, X server is started with -auth argument and hence refuses connections without valid cookie.
Comment 12 Mamoru TASAKA 2015-11-20 09:08:29 EST
Maybe
http://koji.fedoraproject.org/scratch/mtasaka/task_11921228/
can make some progress?
Comment 13 Tore Anderson 2015-11-20 10:10:39 EST
Since I've left work for today I won't be able to test before Monday. I'll let you know, and until then have a nice weekend!
Comment 14 Mamoru TASAKA 2015-11-20 21:58:17 EST
Upstream modified the code differently. Please try this:

http://koji.fedoraproject.org/koji/buildinfo?buildID=700587
Comment 15 Tore Anderson 2015-11-23 01:06:55 EST
(In reply to Mamoru TASAKA from comment #14)
> Upstream modified the code differently. Please try this:
> 
> http://koji.fedoraproject.org/koji/buildinfo?buildID=700587

It works fine, I can log in without any problems now with all config files left to their defaults. $XAUTHORITY gets set to /var/run/lxdm/.Xauth<uid>.
Comment 16 Mamoru TASAKA 2015-11-23 01:30:40 EST
Okay, thank you.
Comment 17 Fedora Update System 2015-11-23 01:33:38 EST
lxdm-0.5.2-2.D20151121gitd5b7ae6b.fc23 has been submitted as an update to Fedora 23. https://bodhi.fedoraproject.org/updates/FEDORA-2015-867aba2b3d
Comment 18 Fedora Update System 2015-11-23 21:23:24 EST
lxdm-0.5.2-2.D20151121gitd5b7ae6b.fc23 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report.
If you want to test the update, you can install it with
$ su -c 'dnf --enablerepo=updates-testing update lxdm'
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-867aba2b3d
Comment 19 Mamoru TASAKA 2015-11-23 22:52:11 EST
Perhaps 0.5.3 will be released today or tomorrow, I will repush it then.
Comment 20 Fedora Update System 2015-11-24 09:08:01 EST
lxdm-0.5.3-1.fc23 has been submitted as an update to Fedora 23. https://bodhi.fedoraproject.org/updates/FEDORA-2015-e3ec4cbf8f
Comment 21 Fedora Update System 2015-11-24 21:53:26 EST
lxdm-0.5.3-1.fc23 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report.
If you want to test the update, you can install it with
$ su -c 'dnf --enablerepo=updates-testing update lxdm'
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-e3ec4cbf8f
Comment 22 Tore Anderson 2015-11-27 02:51:04 EST
I've noticed a new issue that seems related to how this original issue was fixed, namely that "xauth list" reaches a timeout. This command is run by the SSH client whenever I log on to another machine, so it causes a rather annoying 20 second delay every time:

$ time /usr/bin/xauth -v list :0
/usr/bin/xauth:  timeout in locking authority file /var/run/lxdm/.Xauth7097

real	0m20.004s
user	0m0.001s
sys	0m0.001s

If I give my own user write access to the containing directory it works fine, but that's obviously not a proper fix:

$ sudo chmod 777 /var/run/lxdm/
$ time /usr/bin/xauth -v list :0
Using authority file /var/run/lxdm/.Xauth7097
echo.ms.redpill-linpro.com/unix:0  MIT-MAGIC-COOKIE-1  <snip>

real	0m0.003s
user	0m0.001s
sys	0m0.001s

Tore
Comment 23 Mamoru TASAKA 2015-11-27 10:17:05 EST
Please file a new ticket for different issues. Also, I would appreciate it if you would report your issue to the upstream bug tracker, as it seems that you have some your customized setting and it may be difficult to reproduce.
Comment 24 Tore Anderson 2015-11-28 08:13:59 EST
Ok, I submitted a new bug here: https://sourceforge.net/p/lxde/bugs/789/

For the record, though: I am not using any customized settings at all, everything is left at whatever defaults comes with the lxdm-0.5.3-1.fc23.x86_64 package:

tore@envy:~$ sudo rpm -V lxdm ; echo $?
0
tore@envy:~$
Comment 25 Fedora Update System 2015-11-29 03:11:40 EST
lxdm-0.4.1-11.fc22 has been submitted as an update to Fedora 22. https://bodhi.fedoraproject.org/updates/FEDORA-2015-596e37febc
Comment 26 Fedora Update System 2015-11-29 17:20:38 EST
lxdm-0.4.1-11.fc22 has been pushed to the Fedora 22 testing repository. If problems still persist, please make note of it in this bug report.
If you want to test the update, you can install it with
$ su -c 'dnf --enablerepo=updates-testing update lxdm'
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-596e37febc
Comment 27 Fedora Update System 2015-12-05 20:23:56 EST
lxdm-0.5.3-1.fc23 has been pushed to the Fedora 23 stable repository. If problems still persist, please make note of it in this bug report.
Comment 28 Mamoru TASAKA 2015-12-06 01:58:28 EST
Closing.
Comment 29 Fedora Update System 2015-12-09 22:51:37 EST
lxdm-0.4.1-11.fc22 has been pushed to the Fedora 22 stable repository. If problems still persist, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.