Bug 1657295

Summary:	remote user can not ssh to a winbind joined system
Product:	Red Hat Enterprise Linux 8	Reporter:	Patrik Kis <pkis>
Component:	authselect	Assignee:	Pavel Březina <pbrezina>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Steeve Goveas <sgoveas>
Severity:	high	Docs Contact:	Filip Hanzelka <fhanzelk>
Priority:	high
Version:	8.1	CC:	fhanzelk, fweimer, igkioka, jhrozek, lmanasko, ndehadra, pbrezina, sbose
Target Milestone:	rc	Keywords:	Regression, TestBlocker
Target Release:	8.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Known Issue
Doc Text:	.Changing `/etc/nsswitch.conf` requires a manual system reboot Any change to the `/etc/nsswitch.conf` file, for example running the `authselect select profile_id` command, requires a system reboot so that all relevant processes use the updated version of the `/etc/nsswitch.conf` file. If a system reboot is not possible, restart the service that joins your system to Active Directory, which is the `System Security Services Daemon` (SSSD) or `winbind`.	Story Points:	---
Clone Of:		Environment:
Last Closed:	2019-06-11 10:32:21 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1682278
Bug Blocks:

Description Patrik Kis 2018-12-07 15:47:07 UTC

Description of problem:
With the new authselect ssh login does not work for remote users when the system was joined to AD with winbind.
The issue is caused by this change:
"session     required    pam_systemd.so"
what used to be:
"-session    optional     pam_systemd.so"

If the problem is in the systemd pam module forward the bug to systemd please.

Version-Release number of selected component (if applicable):
authselect-1.0-8.el8
systemd-239-8.el8

How reproducible:
always

Steps to Reproduce:
1. join the system to AD with realmd. Eg.
# realm -v join --user=admin --client-software=winbind --membership-software=samba  <AD>
...

2. Try to login
# ssh <user>@<AD>@localhost
...
Last login: Fri Dec  7 10:37:35 2018 from 127.0.0.1
Connection to 127.0.0.1 closed.


Actual results: Logs:


 sshd[22113]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=127.0.0.1  user=user
 sshd[22113]: pam_winbind(sshd:auth): getting password (0x00000390)
 sshd[22113]: pam_winbind(sshd:auth): pam_get_item returned a password
 sshd[22113]: pam_winbind(sshd:auth): user 'AD\user' granted access
 sshd[22113]: pam_winbind(sshd:account): user 'AD\user' granted access
 sshd[22113]: Accepted password for user from 127.0.0.1 port 32838 ssh2
 sshd[22113]: pam_systemd(sshd:session): pam-systemd initializing
 sshd[22113]: pam_systemd(sshd:session): Asking logind to create session: uid=2001103 pid=22113 service=sshd type=tty class=user desktop= seat= vtnr=0 tty= display= remote=yes remote_user= remote_host=127.0.0.1
 sshd[22113]: pam_systemd(sshd:session): Session limits: memory_max=n/a tasks_max=n/a cpu_weight=n/a io_weight=n/a
 sshd[22113]: pam_systemd(sshd:session): Failed to create session: No such file or directory
 sshd[22113]: pam_unix(sshd:session): session opened for user AD\user by (uid=0)
 sshd[22113]: error: PAM: pam_open_session(): System error
 sshd[22116]: Received disconnect from 127.0.0.1 port 32838:11: disconnected by user
 sshd[22116]: Disconnected from user user 127.0.0.1 port 32838
 sshd[22113]: pam_winbind(sshd:setcred): user 'AD\user' OK

Comment 1 Sumit Bose 2018-12-11 12:11:18 UTC

I think the underlying reason for this is that logind does not re-read /etc/nsswitch.conf and hence is not aware of nss-winbind and the new users served by winbind.

I can make the test pass with:

diff --git a/Regression/bz1484072-realmd-join-add-deprecated-options-to-samba-file/runtest.sh b/Regression/bz1484072-realmd-join-add-deprecated-options-to-samba-file/runtest.sh
index ca83832..01a5875 100755
--- a/Regression/bz1484072-realmd-join-add-deprecated-options-to-samba-file/runtest.sh
+++ b/Regression/bz1484072-realmd-join-add-deprecated-options-to-samba-file/runtest.sh
@@ -133,6 +133,7 @@ rlJournalStart
         # On slower machines the sssd and mainly winbind daemon may not be ready immediately
         echo "Winbind is slooow. Give it some time. Sleeping for 20 seconds."
         sleep 30
+        rlRun "systemctl restart systemd-logind"
         rlRun "realmd_check_join $realmd_USER $realmd_USER_PASS $realmd_REALM_DOMAIN_AD"
         # Check the home folder location (it should be user@domain all in lowercase)
         rlRun "getent passwd $realmd_USER@$realmd_REALM_DOMAIN_AD &>id.log"


Not sure what is the best way forward here. authselect can signal/restart logind whenever /etc/nsswitch.conf is touched or just print a message and let the callers do this?

Comment 2 Sumit Bose 2018-12-11 12:16:55 UTC

Ah, I forgot to say, this is not a logind issue, it will affect all long running processes which were started before authselect changed /etc/nsswitch.conf and using authconfig would lead to the same issue as well.

From nsswitch.conf man page:

"""
 Within each process that uses nsswitch.conf, the entire file is read only once.  If the file is later changed, the process will continue using the old configuration.
"""

Comment 13 Nikhil Dehadrai 2019-01-10 10:39:52 UTC

Seeing similar issue in ipa-hbac-func test suite, where 'su' command fails


Example::

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   user access to master for ftp, on master, add rule
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

:: [ 15:43:47 ] :: [  BEGIN   ] :: Kinit as admin user :: actually running 'kinitAs admin Secret123'
Password for admin: 
Default principal: admin
:: [ 15:43:48 ] :: [   LOG    ] :: kinit as admin with password Secret123 was successful.
:: [ 15:43:48 ] :: [   PASS   ] :: Kinit as admin user (Expected 0, got 0)
:: [ 15:43:48 ] :: [  BEGIN   ] :: Running 'mkdir -p /home/user1'
:: [ 15:43:48 ] :: [   PASS   ] :: Command 'mkdir -p /home/user1' (Expected 0, got 0)
:: [ 15:43:48 ] :: [  BEGIN   ] :: Running 'chmod 700 /home/user1'
:: [ 15:43:48 ] :: [   PASS   ] :: Command 'chmod 700 /home/user1' (Expected 0, got 0)
:: [ 15:43:48 ] :: [  BEGIN   ] :: Running 'create_ipauser user1 user1 user1 '
:: [ 15:43:48 ] :: [   LOG    ] :: create ipa user: [user1], firstname: [user1], lastname: [user1]  password: [testpw123]
:: [ 15:43:50 ] :: [   LOG    ] :: create ipa user: [user1], password: [testpw123]
:: [ 15:43:50 ] :: [  BEGIN   ] :: add test user account :: actually running 'echo dummy123 |           ipa user-add user1                         --first 'user1'                         --last  'user1'                         --password '
------------------
Added user "user1"
------------------
  User login: user1
  First name: user1
  Last name: user1
  Full name: user1 user1
  Display name: user1 user1
  Initials: uu
  Home directory: /home/user1
  GECOS: user1 user1
  Login shell: /bin/sh
  Principal name: user1
  Principal alias: user1
  User password expiration: 20190110101351Z
  Email address: user1
  UID: 1752800006
  GID: 1752800006
  Password: True
  Member of groups: ipausers
  Kerberos keys available: True
:: [ 15:43:52 ] :: [   PASS   ] :: add test user account (Expected 0, got 0)
:: [ 15:43:54 ] :: [  BEGIN   ] :: Running 'FirstKinitAs user1 dummy123 testpw123'
Using default cache: 0
Using principal: user1
Password for user1: 
Password expired.  You must change it now.
Enter new password: 
Enter it again: 
Authenticated to Kerberos v5
Default principal: user1
:: [ 15:43:55 ] :: [   LOG    ] :: kinit as user1 with new password testpw123 was successful.
:: [ 15:43:55 ] :: [   PASS   ] :: Command 'FirstKinitAs user1 dummy123 testpw123' (Expected 0, got 0)
user1
:: [ 15:43:55 ] :: [   PASS   ] :: Command 'create_ipauser user1 user1 user1 ' (Expected 0, got 0)
:: [ 15:43:55 ] :: [  BEGIN   ] :: Running 'chown user1.user1 /home/user1'
:: [ 15:43:55 ] :: [   PASS   ] :: Command 'chown user1.user1 /home/user1' (Expected 0, got 0)
:: [ 15:43:55 ] :: [   LOG    ] :: Adding Dummy Code-Debugging
:: [ 15:43:56 ] :: [  BEGIN   ] :: Restarting systemd login service :: actually running 'systemctl restart systemd-logind'
:: [ 15:43:56 ] :: [   PASS   ] :: Restarting systemd login service (Expected 0, got 0)
:: [ 15:43:56 ] :: [  BEGIN   ] :: Running 'sleep 120'
:: [ 15:45:56 ] :: [   PASS   ] :: Command 'sleep 120' (Expected 0, got 0)
:: [ 15:45:56 ] :: [   LOG    ] :: Dummy Code-Debugging end
:: [ 15:45:56 ] :: [  BEGIN   ] :: Running 'su - user1 -c 'pwd''
su: cannot open session: System error
:: [ 15:45:56 ] :: [   FAIL   ] :: Command 'su - user1 -c 'pwd'' (Expected 0, got 1)



Logs:

[root@vm-idm-020 ~]# cat /var/log/secure | grep 'su'
Jan 10 15:40:38 vm-idm-020 su[9611]: pam_unix(su-l:session): session opened for user user1 by (uid=0)
Jan 10 15:40:38 vm-idm-020 su[9611]: pam_unix(su-l:session): session closed for user user1
Jan 10 15:40:39 vm-idm-020 sshd[9677]: pam_sss(sshd:auth): authentication success; logname= uid=0 euid=0 tty=ssh ruser= rhost=10.65.206.154 user=user1
Jan 10 15:40:51 vm-idm-020 su[9837]: pam_unix(su-l:session): session opened for user user1 by (uid=0)
Jan 10 15:40:51 vm-idm-020 su[9837]: pam_unix(su-l:session): session closed for user user1
Jan 10 15:41:08 vm-idm-020 su[9991]: pam_unix(su-l:session): session opened for user user2 by (uid=0)
Jan 10 15:41:08 vm-idm-020 su[9991]: pam_unix(su-l:session): session closed for user user2
Jan 10 15:41:25 vm-idm-020 su[10149]: pam_unix(su-l:session): session opened for user user3 by (uid=0)
Jan 10 15:41:25 vm-idm-020 su[10149]: pam_unix(su-l:session): session closed for user user3
Jan 10 15:41:31 vm-idm-020 sshd[10247]: pam_sss(sshd:auth): authentication success; logname= uid=0 euid=0 tty=ssh ruser= rhost=10.65.206.154 user=user1
Jan 10 15:41:32 vm-idm-020 sshd[10303]: pam_sss(sshd:auth): authentication success; logname= uid=0 euid=0 tty=ssh ruser= rhost=10.65.206.154 user=user3
Jan 10 15:45:56 vm-idm-020 su[11289]: pam_systemd(su-l:session): Failed to create session: Start job for unit user failed with 'failed'
Jan 10 15:45:56 vm-idm-020 su[11289]: pam_unix(su-l:session): session opened for user user1 by (uid=0)
Jan 10 15:48:04 vm-idm-020 su[11518]: pam_systemd(su-l:session): Failed to create session: Start job for unit user failed with 'failed'
Jan 10 15:48:04 vm-idm-020 su[11518]: pam_unix(su-l:session): session opened for user user2 by (uid=0)

Comment 14 Pavel Březina 2019-01-11 10:08:40 UTC

This seems to be the same a bug reported against IPA: https://bugzilla.redhat.com/show_bug.cgi?id=1664974

Comment 15 Pavel Březina 2019-03-05 11:37:23 UTC

The commit that brought this issue up has been reverted [1]. But it should still be documented that changing nsswitch.conf requires reboot.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1643928

Comment 19 Pavel Březina 2019-06-11 10:32:21 UTC

Changes to nsswitch.conf requires reboot of the machine/daemons for the daemons to pick it up. Automatic reload of nsswitch table will be implemented in glibc [1] and authselect can not do anything about it besides documenting it. Documentation has been provided and this particular issue has been worked around in RHEL8.0. Thus closing as CURRENTRELEASE.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=132608