Note: This bug is displayed in read-only format because
the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
.Accessing authorized keys no longer fails when switching users
Previously, group information cache in OpenSSH was not cleaned when changing the user for retrieving authorized keys using the `AuthorizedKeysCommand*` configuration options. Consequently, an attempt to access authorized keys failed for the new user due to incorrect group information. This bug has been fixed, and authorized keys can now be successfully accessed when the user is changed.
Description of problem:
When:
- using AuthorizedKeysCommand to some program (e.g. AuthorizedKeysCommand=/usr/bin/true)
and
- the user logging in (e.g. user1/user1) is member of a top group (e.g. users)
and
- /home directory is not accessible for users not in the top group (chgrp users /home ; chmod 750 /home)
Then Public key authentication using ~/.ssh/authorized_keys file fails with Permission Denied.
Version-Release number of selected component (if applicable):
All RHEL7 openssh versions: openssh-6.4p1-8.el7 -> openssh-7.4p1-16.el7.x86_64
How reproducible:
Always
Steps to Reproduce:
1. Change permissions for /home
# chgrp users /home
# chmod 750 /home
2. Create a user, member of "users" as supplementary group
# useradd -G users user1
# mkdir /home/user1/.ssh
# touch /home/user1/.ssh/authorized_keys
# chown -R user1:user1 /home/user1/.ssh
# chmod -R go-rwx /home/user1/.ssh
3. Execute sshd to verify the setup
# /usr/sbin/sshd -p 8022 -d
From a remote host: $ ssh -p 8022 -o PasswordAuthentication=no user1@server
-> Fails because we didn't have a pub key, but it's functional, see trace below
debug1: trying public key file /home/user2/.ssh/authorized_keys
debug1: fd 4 clearing O_NONBLOCK
debug1: restore_uid: 0/0
4. Execute sshd with dummy AuthorizedKeysCommand
# /usr/sbin/sshd -p 8022 -d -o AuthorizedKeysCommand=/usr/bin/true -o AuthorizedKeysCommandUser=nobody
From a remote host: $ ssh -p 8022 -o PasswordAuthentication=no user1@server
-> Fails with Permission denied, see trace below
debug1: trying public key file /home/user1/.ssh/authorized_keys
debug1: Could not open authorized keys '/home/user1/.ssh/authorized_keys': Permission denied
debug1: restore_uid: 0/0
Additional info:
Looking at the strace, we can see that in 4., the group being restored is "nobody" (99), whereas both "user1" and "users" should be restored. This causes the issue since /home is readable only for members of "users".
Strace for 3.:
2659 16:19:08.330622 setgroups(2, [1000, 100]) = 0 <0.000026>
2659 16:19:08.330714 getgroups(0, NULL) = 2 <0.000021>
2659 16:19:08.330786 getgroups(2, [100, 1000]) = 2 <0.000021>
2659 16:19:08.330896 setgroups(2, [100, 1000]) = 0 <0.000023>
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2659 16:19:08.330980 setresgid(-1, 1000, -1) = 0 <0.000023>
2659 16:19:08.331056 setresuid(-1, 1000, -1) = 0 <0.000023>
2659 16:19:08.331130 open("/home/user1/.ssh/authorized_keys", O_RDONLY|O_NONBLOCK) = 4</home/user1/.ssh/authorized_keys> <0.000036>
Strace for 4. (Permission denied):
2643 16:18:45.891249 setresuid(-1, 0, -1) = 0 <0.000004>
2643 16:18:45.891263 setresgid(-1, 0, -1) = 0 <0.000004>
2643 16:18:45.891277 setgroups(0, NULL) = 0 <0.000005>
2643 16:18:45.891296 geteuid() = 0 <0.000004>
2643 16:18:45.891309 getegid() = 0 <0.000003>
2643 16:18:45.891321 getgroups(0, NULL) = 0 <0.000004>
2643 16:18:45.891334 setgroups(1, [99]) = 0 <0.000005>
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2643 16:18:45.891348 setresgid(-1, 1000, -1) = 0 <0.000007>
2643 16:18:45.891379 setresuid(-1, 1000, -1) = 0 <0.000004>
2643 16:18:45.891394 open("/home/user1/.ssh/authorized_keys", O_RDONLY|O_NONBLOCK) = -1 EACCES (Permission denied) <0.000006>
The root cause is that user_groups variable is computed only once (user_groupslen == -1):
uidswap.c:
97 /* set and save the user's groups */
98 if (user_groupslen == -1) {
99 if (initgroups(pw->pw_name, pw->pw_gid) < 0)
100 fatal("initgroups: %s: %.100s", pw->pw_name,
101 strerror(errno));
102
103 user_groupslen = getgroups(0, NULL);
104 if (user_groupslen < 0)
105 fatal("getgroups: %.100s", strerror(errno));
106 if (user_groupslen > 0) {
107 user_groups = xreallocarray(user_groups,
108 user_groupslen, sizeof(gid_t));
109 if (getgroups(user_groupslen, user_groups) < 0)
110 fatal("getgroups: %.100s", strerror(errno));
111 } else { /* user_groupslen == 0 */
112 free(user_groups);
113 }
114 }
In the scenario, the first time temporarily_use_uid() is called is when switching the parent sshd process to "nobody" after forking the AuthorizedKeysCommand program:
pam_ssh_agent_auth-0.10.3/pam_user_key_allowed2.c:
219 debug("Running AuthorizedKeysCommand: \"%s\" as \"%s\" with argument: \"%s\"",
220 authorized_keys_command, pw->pw_name, username);
...
228 switch ((pid = fork())) {
...
283 temporarily_use_uid(pw);
284
285 close(p[1]);
...
EDIT: First part is ok, Second isn't.
The first call being made is here, before forking in fact:
auth2-pubkey.c:
398 static pid_t
399 subprocess(const char *tag, struct passwd *pw, const char *command,
400 int ac, char **av, FILE **child)
401 {
...
412 debug3("%s: %s command \"%s\" running as %s", __func__,
413 tag, command, pw->pw_name);
414
415 /* Verify the path exists and is safe-ish to execute */
416 if (*av[0] != '/') {
417 error("%s path is not absolute", tag);
418 return 0;
419 }
420 temporarily_use_uid(pw);
Created attachment 1445835[details]
proposed patch
Thank you for the good analysis. My initial bet was that some pw structure is being overwritten somewhere.
I reproduced the issue with RHEL7 virtual machine, but from the above steps, I was not able to reproduce the issue in current Fedora 28, even though I do not see any significant difference in the code there. I will investigate this further before reporting upstream.
This is something we should fix. The attached patch is resolving the issue for me. The idea is that we will reuse the stored groups only if we will be swapping for the same uid).
Ok, I reproduced the same issue also in Fedora 28 but for some reason, there is not the permission denied (probably some permissions elsewhere), but the strace provides the proof that the same issue is in upstream:
debug1: temporarily_use_uid: 1005/1009 (e=0/0)
[pid 3869] setgroups(1, [99]) = 0
[pid 3869] setresgid(-1, 1009, -1) = 0
debug1: trying public key file /home/user1/.ssh/authorized_keys
debug1: fd 10 clearing O_NONBLOCK
The upstream bug is attached.
Workaround is to relax the permissions ("rx" for others on /home"), or relax for group "nobody" only, using the following ACL:
# setfacl -m group:nobody:rx /home
> when changing the user.
should be more clear that it is only about the AuthorizedKeysCommand. Something like
> when changing the user for retrieving authorized keys using AuthorizedKeysCommand* configuration options.
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
https://access.redhat.com/errata/RHSA-2019:2143
Description of problem: When: - using AuthorizedKeysCommand to some program (e.g. AuthorizedKeysCommand=/usr/bin/true) and - the user logging in (e.g. user1/user1) is member of a top group (e.g. users) and - /home directory is not accessible for users not in the top group (chgrp users /home ; chmod 750 /home) Then Public key authentication using ~/.ssh/authorized_keys file fails with Permission Denied. Version-Release number of selected component (if applicable): All RHEL7 openssh versions: openssh-6.4p1-8.el7 -> openssh-7.4p1-16.el7.x86_64 How reproducible: Always Steps to Reproduce: 1. Change permissions for /home # chgrp users /home # chmod 750 /home 2. Create a user, member of "users" as supplementary group # useradd -G users user1 # mkdir /home/user1/.ssh # touch /home/user1/.ssh/authorized_keys # chown -R user1:user1 /home/user1/.ssh # chmod -R go-rwx /home/user1/.ssh 3. Execute sshd to verify the setup # /usr/sbin/sshd -p 8022 -d From a remote host: $ ssh -p 8022 -o PasswordAuthentication=no user1@server -> Fails because we didn't have a pub key, but it's functional, see trace below debug1: trying public key file /home/user2/.ssh/authorized_keys debug1: fd 4 clearing O_NONBLOCK debug1: restore_uid: 0/0 4. Execute sshd with dummy AuthorizedKeysCommand # /usr/sbin/sshd -p 8022 -d -o AuthorizedKeysCommand=/usr/bin/true -o AuthorizedKeysCommandUser=nobody From a remote host: $ ssh -p 8022 -o PasswordAuthentication=no user1@server -> Fails with Permission denied, see trace below debug1: trying public key file /home/user1/.ssh/authorized_keys debug1: Could not open authorized keys '/home/user1/.ssh/authorized_keys': Permission denied debug1: restore_uid: 0/0 Additional info: Looking at the strace, we can see that in 4., the group being restored is "nobody" (99), whereas both "user1" and "users" should be restored. This causes the issue since /home is readable only for members of "users". Strace for 3.: 2659 16:19:08.330622 setgroups(2, [1000, 100]) = 0 <0.000026> 2659 16:19:08.330714 getgroups(0, NULL) = 2 <0.000021> 2659 16:19:08.330786 getgroups(2, [100, 1000]) = 2 <0.000021> 2659 16:19:08.330896 setgroups(2, [100, 1000]) = 0 <0.000023> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2659 16:19:08.330980 setresgid(-1, 1000, -1) = 0 <0.000023> 2659 16:19:08.331056 setresuid(-1, 1000, -1) = 0 <0.000023> 2659 16:19:08.331130 open("/home/user1/.ssh/authorized_keys", O_RDONLY|O_NONBLOCK) = 4</home/user1/.ssh/authorized_keys> <0.000036> Strace for 4. (Permission denied): 2643 16:18:45.891249 setresuid(-1, 0, -1) = 0 <0.000004> 2643 16:18:45.891263 setresgid(-1, 0, -1) = 0 <0.000004> 2643 16:18:45.891277 setgroups(0, NULL) = 0 <0.000005> 2643 16:18:45.891296 geteuid() = 0 <0.000004> 2643 16:18:45.891309 getegid() = 0 <0.000003> 2643 16:18:45.891321 getgroups(0, NULL) = 0 <0.000004> 2643 16:18:45.891334 setgroups(1, [99]) = 0 <0.000005> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2643 16:18:45.891348 setresgid(-1, 1000, -1) = 0 <0.000007> 2643 16:18:45.891379 setresuid(-1, 1000, -1) = 0 <0.000004> 2643 16:18:45.891394 open("/home/user1/.ssh/authorized_keys", O_RDONLY|O_NONBLOCK) = -1 EACCES (Permission denied) <0.000006>