Bug 1724423 - NIS based system experiences 25 second authentication delay after upgrade
Summary: NIS based system experiences 25 second authentication delay after upgrade
Keywords:
Status: CLOSED EOL
Alias: None
Product: Fedora
Classification: Fedora
Component: nss_nis
Version: 30
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Matej Mužila
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-06-27 04:32 UTC by Krish
Modified: 2022-01-21 15:38 UTC (History)
15 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2020-05-26 18:40:02 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Krish 2019-06-27 04:32:52 UTC
Description of problem:

System running Fedora 29, using NIS was upgraded to Fedora 30 using DNF. Subsequently, a 25 sec delay in login times were noticed for regular users. Journalctl output for sshd indicated that pam_systemd module is being invoked after password has been accepted from the user. Subsequently pam_systemd times out and pam_unix succeeds. Fedora 29 and previous releases never invoked a pam_systemd password check. systemd-resolved is disabled on this system.

Version-Release number of selected component (if applicable):


How reproducible:

Consistently reproducible. Fedora 29 system using NIS was upgraded to Fedora 30 repeatedly, and each time the 25 second delay and the pam_systemd error was noted.

Steps to Reproduce:
1. Install Fedora 29 on a system. Setup NIS with ypbind, nsswitch and yp.conf pointing to the correct service. Attempt login and observe journalctl -u sshd (No pam_systemd error)
2. Upgrade using dnf system-upgrade to release version 30
3. Attempt login as same user. 25 second delay noted. Journalctl -u sshd shows pam_systemd error

Actual results:
Journalctl output for sshd on Fedora 30-

Jun 27 00:06:14 dbase.lhup.edu sshd[5565]: Accepted password for kpillai from 10.1.2.2 port 38432 ssh2
Jun 27 00:06:39 dbase.lhup.edu sshd[5565]: pam_systemd(sshd:session): Failed to create session: Connection timed out
Jun 27 00:06:39 dbase.lhup.edu sshd[5565]: pam_unix(sshd:session): session opened for user kpillai by (uid=0)


Expected results:
Jun 27 23:40:16 dbase.lhup.edu sshd[8590]: Accepted password for kpillai from 10.1.2.2  port 65412 ssh2
Jun 27 23:40:16 dbase.lhup.edu sshd[8590]: pam_unix(sshd:session): session opened for user kpillai by (uid=0)


Additional info:
Please note 25 second delay as a result of pam_systemd session failure for Fedora 30 - No observable delay for the Fedora 29 case. The system was running fine for at least 12 years on various versions of Fedora and was upgraded recently to Fedora 30 which triggered this degradation

Comment 1 Tomas Mraz 2019-06-27 06:24:41 UTC
Please note that this is a session phase of PAM calls which does not have anything to do with passwords.

Comment 2 Jakub Jelen 2019-06-27 07:37:05 UTC
The log says that there is something wrong with system pam module. Hopefully, the systemd developers will be able to help you with debugging this further. There is nothing I can do about misbehaving pam modules in openssh.

Comment 3 Krish 2019-06-27 10:29:48 UTC
Thanks...pam_systemd should not be invoked at all. If this was a systemd based authentication environment it would have worked, but it is NIS based. Can someone from system-maint take a look at this please? Our lab servers have been running Fedora forever, serving hundreds of students, and this really messes things up. Should I leave the bug as it is, or should I close and reopen this?

Comment 4 Jakub Jelen 2019-06-27 10:42:41 UTC
pam_systemd is invoked as part of the session step of pam stack for quite a long time so this is certainly not something new in Fedora 30. I would recommend starting with adding a debug argument to the pam_systemd module and gathering some more logs.

As a dirty workaround, you could try to remove the pam_systemd from the PAM stack for sshd, but it might break other things (especially because the pam_systemd is in some of the includes that are used by other tools).

You do not need to open a new bug. I think this is fine. I would wait for systemd maintainers to check what new the pam module does or what could have got broken in your configuration.

Comment 5 Tomas Mraz 2019-06-27 10:58:54 UTC
pam_systemd is not there for any authentication. pam_systemd purpose is to create and set up the systemd session for the user login. That applies to all users regardless of where they come from whether they are coming from local files, NIS, LDAP, AD, whatever.

Comment 6 Krish 2019-06-27 11:31:55 UTC
(In reply to Tomas Mraz from comment #5)
> pam_systemd is not there for any authentication. pam_systemd purpose is to
> create and set up the systemd session for the user login. That applies to
> all users regardless of where they come from whether they are coming from
> local files, NIS, LDAP, AD, whatever.

The pam_systemd errors (and messages) do not show up in Fedora 29 and earlier versions for sshd (please see Description of the problem journalctl output) . So something has obviously changed in pam_systemd usage by ssh from 29 to 30.

Comment 7 Krish 2019-06-29 19:44:33 UTC
This is related since it's the same error and might help with the debugging process. The Fedora 30 above is running in multi-user.target, so I'm not sure if a desktop login is possible given the delay.  But the exact same issue (the 25 seconds timeout on pam_systemd) happens for Ubuntu 18.04 LTS as well, running in graphical target. Ubuntu 16.04 LTS works fine, but after an upgrade to 18.04 LTS, gdm hangs and returns you to login screen, and times out because of the 25 second delay with pam_systemd error, totally unusable. Looks like this error was reported in ubuntu a year ago and was never fixed.
https://askubuntu.com/questions/1031022/using-nis-client-in-ubuntu-18-04-crashes-both-gnome-and-unity

Comment 8 Ben Cotton 2020-04-30 20:13:34 UTC
This message is a reminder that Fedora 30 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 30 on 2020-05-26.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '30'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 30 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 9 Ben Cotton 2020-05-26 18:40:02 UTC
Fedora 30 changed to end-of-life (EOL) status on 2020-05-26. Fedora 30 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.

Comment 10 Plumber Bot 2022-01-21 15:38:58 UTC
Dropping the stale needinfo. If our input is still needed, please set the needinfo again.


Note You need to log in before you can comment on or make changes to this bug.