Hide Forgot
Description of problem: When: - using AuthorizedKeysCommand to some program (e.g. AuthorizedKeysCommand=/usr/bin/true) and - the user logging in (e.g. user1/user1) is member of a top group (e.g. users) and - /home directory is not accessible for users not in the top group (chgrp users /home ; chmod 750 /home) Then Public key authentication using ~/.ssh/authorized_keys file fails with Permission Denied. Version-Release number of selected component (if applicable): All RHEL7 openssh versions: openssh-6.4p1-8.el7 -> openssh-7.4p1-16.el7.x86_64 How reproducible: Always Steps to Reproduce: 1. Change permissions for /home # chgrp users /home # chmod 750 /home 2. Create a user, member of "users" as supplementary group # useradd -G users user1 # mkdir /home/user1/.ssh # touch /home/user1/.ssh/authorized_keys # chown -R user1:user1 /home/user1/.ssh # chmod -R go-rwx /home/user1/.ssh 3. Execute sshd to verify the setup # /usr/sbin/sshd -p 8022 -d From a remote host: $ ssh -p 8022 -o PasswordAuthentication=no user1@server -> Fails because we didn't have a pub key, but it's functional, see trace below debug1: trying public key file /home/user2/.ssh/authorized_keys debug1: fd 4 clearing O_NONBLOCK debug1: restore_uid: 0/0 4. Execute sshd with dummy AuthorizedKeysCommand # /usr/sbin/sshd -p 8022 -d -o AuthorizedKeysCommand=/usr/bin/true -o AuthorizedKeysCommandUser=nobody From a remote host: $ ssh -p 8022 -o PasswordAuthentication=no user1@server -> Fails with Permission denied, see trace below debug1: trying public key file /home/user1/.ssh/authorized_keys debug1: Could not open authorized keys '/home/user1/.ssh/authorized_keys': Permission denied debug1: restore_uid: 0/0 Additional info: Looking at the strace, we can see that in 4., the group being restored is "nobody" (99), whereas both "user1" and "users" should be restored. This causes the issue since /home is readable only for members of "users". Strace for 3.: 2659 16:19:08.330622 setgroups(2, [1000, 100]) = 0 <0.000026> 2659 16:19:08.330714 getgroups(0, NULL) = 2 <0.000021> 2659 16:19:08.330786 getgroups(2, [100, 1000]) = 2 <0.000021> 2659 16:19:08.330896 setgroups(2, [100, 1000]) = 0 <0.000023> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2659 16:19:08.330980 setresgid(-1, 1000, -1) = 0 <0.000023> 2659 16:19:08.331056 setresuid(-1, 1000, -1) = 0 <0.000023> 2659 16:19:08.331130 open("/home/user1/.ssh/authorized_keys", O_RDONLY|O_NONBLOCK) = 4</home/user1/.ssh/authorized_keys> <0.000036> Strace for 4. (Permission denied): 2643 16:18:45.891249 setresuid(-1, 0, -1) = 0 <0.000004> 2643 16:18:45.891263 setresgid(-1, 0, -1) = 0 <0.000004> 2643 16:18:45.891277 setgroups(0, NULL) = 0 <0.000005> 2643 16:18:45.891296 geteuid() = 0 <0.000004> 2643 16:18:45.891309 getegid() = 0 <0.000003> 2643 16:18:45.891321 getgroups(0, NULL) = 0 <0.000004> 2643 16:18:45.891334 setgroups(1, [99]) = 0 <0.000005> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2643 16:18:45.891348 setresgid(-1, 1000, -1) = 0 <0.000007> 2643 16:18:45.891379 setresuid(-1, 1000, -1) = 0 <0.000004> 2643 16:18:45.891394 open("/home/user1/.ssh/authorized_keys", O_RDONLY|O_NONBLOCK) = -1 EACCES (Permission denied) <0.000006>
The root cause is that user_groups variable is computed only once (user_groupslen == -1): uidswap.c: 97 /* set and save the user's groups */ 98 if (user_groupslen == -1) { 99 if (initgroups(pw->pw_name, pw->pw_gid) < 0) 100 fatal("initgroups: %s: %.100s", pw->pw_name, 101 strerror(errno)); 102 103 user_groupslen = getgroups(0, NULL); 104 if (user_groupslen < 0) 105 fatal("getgroups: %.100s", strerror(errno)); 106 if (user_groupslen > 0) { 107 user_groups = xreallocarray(user_groups, 108 user_groupslen, sizeof(gid_t)); 109 if (getgroups(user_groupslen, user_groups) < 0) 110 fatal("getgroups: %.100s", strerror(errno)); 111 } else { /* user_groupslen == 0 */ 112 free(user_groups); 113 } 114 } In the scenario, the first time temporarily_use_uid() is called is when switching the parent sshd process to "nobody" after forking the AuthorizedKeysCommand program: pam_ssh_agent_auth-0.10.3/pam_user_key_allowed2.c: 219 debug("Running AuthorizedKeysCommand: \"%s\" as \"%s\" with argument: \"%s\"", 220 authorized_keys_command, pw->pw_name, username); ... 228 switch ((pid = fork())) { ... 283 temporarily_use_uid(pw); 284 285 close(p[1]); ...
EDIT: First part is ok, Second isn't. The first call being made is here, before forking in fact: auth2-pubkey.c: 398 static pid_t 399 subprocess(const char *tag, struct passwd *pw, const char *command, 400 int ac, char **av, FILE **child) 401 { ... 412 debug3("%s: %s command \"%s\" running as %s", __func__, 413 tag, command, pw->pw_name); 414 415 /* Verify the path exists and is safe-ish to execute */ 416 if (*av[0] != '/') { 417 error("%s path is not absolute", tag); 418 return 0; 419 } 420 temporarily_use_uid(pw);
Issue can be reproduced on fedora27 (openssh-7.6p1-5.fc27.x86_64)
Created attachment 1445835 [details] proposed patch Thank you for the good analysis. My initial bet was that some pw structure is being overwritten somewhere. I reproduced the issue with RHEL7 virtual machine, but from the above steps, I was not able to reproduce the issue in current Fedora 28, even though I do not see any significant difference in the code there. I will investigate this further before reporting upstream. This is something we should fix. The attached patch is resolving the issue for me. The idea is that we will reuse the stored groups only if we will be swapping for the same uid).
Ok, I reproduced the same issue also in Fedora 28 but for some reason, there is not the permission denied (probably some permissions elsewhere), but the strace provides the proof that the same issue is in upstream: debug1: temporarily_use_uid: 1005/1009 (e=0/0) [pid 3869] setgroups(1, [99]) = 0 [pid 3869] setresgid(-1, 1009, -1) = 0 debug1: trying public key file /home/user1/.ssh/authorized_keys debug1: fd 10 clearing O_NONBLOCK The upstream bug is attached.
Workaround is to relax the permissions ("rx" for others on /home"), or relax for group "nobody" only, using the following ACL: # setfacl -m group:nobody:rx /home
For the record, upstream already committed the fix: https://github.com/openssh/openssh-portable/commit/26f96ca
> when changing the user. should be more clear that it is only about the AuthorizedKeysCommand. Something like > when changing the user for retrieving authorized keys using AuthorizedKeysCommand* configuration options.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2019:2143