Bug 1328243 - OpenSSH only looks for .k5login in user directory
Summary: OpenSSH only looks for .k5login in user directory
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: openssh
Version: 7.2
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: rc
: ---
Assignee: Jakub Jelen
QA Contact: Stanislav Zidek
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-04-18 20:08 UTC by vjhouse
Modified: 2019-12-16 05:40 UTC (History)
3 users (show)

Fixed In Version: openssh-6.6.1p1-27.el7
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-11-03 20:19:58 UTC
Target Upstream Version:


Attachments (Terms of Use)
read k5login_directory from krb5.conf and use appropriate path for k5login search (2.18 KB, patch)
2016-05-06 15:30 UTC, Jakub Jelen
no flags Details | Diff


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2016:2588 normal SHIPPED_LIVE Moderate: openssh security, bug fix, and enhancement update 2016-11-03 12:09:45 UTC

Description vjhouse 2016-04-18 20:08:16 UTC
Description of problem:
OpenSSH was patched in the RHEL/CentOS 7 so that it looks for the .k5login file every time GSSAPI/Kerberos is used in the users home directory.  Reference patch "openssh-6.6p1-kuserok.patch".  This is assuming that the users home directory is accessible or exists at the time of the login.  This is not allowing the use of krb5.conf to change to an alternate path for the .k5login if needed.

One case where this is an issue is if you use autofs with kereberos authentication.  OpenSSH will try to locate the file in the user directory which will trigger the automounter, however locating the .k5login is before the kerberos credentials are passed to the remote server which causes autofs to fail to mount and results in a chdir error.  With other versions of OpenSSH there is no issue as it will not try to enter the directory too early.  What should occur is that OpenSSH checks with the krb5.conf file for the location of the .k5login directory if search_directory is set to another location.  This would function like authorized_keys and OpenSSH with an alternate path, alternatively being able to set the path in the OpenSSH config would work as well.

One should never assume that a directory exists on the server and there should always be a way to an alternate directory.



Version-Release number of selected component (if applicable):
openssh.x86_64  6.6.1p1-25.el7_2 


How reproducible:
Everytime with using kerberos and autofs with autofs set to use kerberos credentials for auth.


Steps to Reproduce:
1.  Set up 2 SSH servers, 1 samba file server.  All RHEL/CentOS 7.2.
2.  Set up autofs on the 2 SSH servers to mount the user home directory from the samba server with kerberos credentials
3.  Enable gssapi authentication on the SSH servers.
4.  Attach servers to an AD environment Samba/Winbind, use authconfig.
5.  Verify that user can obtain kerberos credentials on both SSH servers.
6.  Log into first SSH server as an AD user and then SSH into next SSH server.  Should allow user to use their kerberos ticket to authenticate to other server and not require a second login.  However SSH will fail as it is trying to mount with autofs before the kerberos information is passed since .k5login will allow users to authenticate as another if set, hence no kerberos for autofs to mount when looking for the .k5login.  strace of sshd will confirm that OpenSSH is looking for the .k5login only in the users home directory every time.
7.  This works as expected if using a version of OpenSSH not supplied by official RHEL/CentOS repos.

Actual results:
Autofs will fail to open the directory for OpenSSH as it just errored out looking for the .k5login file before it received kerberos credentials.

Expected results:
OpenSSH when using Kerberos and Autofs should not check for the .k5login just in the users home directory.  Instead it should check where the admin points the server to so that Autofs does not fail.

Additional info:
If you do a trace and enable the .k5users you will see this also is always looking in the users home directory before the kerberos credentials are passed.  However OpenSSH allows for .k5users check to be disabled.  A similar check for .k5login should be afforded to users, would be a simple patch to the patch that caused the issue.

Comment 2 vjhouse 2016-04-18 20:16:23 UTC
Also reference patch: openssh-6.6p1-force_krb.patch

Comment 3 Jakub Jelen 2016-04-19 07:16:14 UTC
    Thank you for taking the time to enter a bug report with us. We appreciate the feedback and look to use reports such as this to guide our efforts at improving our products. That being said, this bug tracking system is not a mechanism for requesting support, and we are not able to  guarantee the timeliness or suitability of a resolution.

    If this issue is critical or in any way time sensitive, please raise a ticket through your regular Red Hat support channels to make certain  it receives the proper attention and prioritization to assure a timely resolution. 

    For information on how to contact the Red Hat production support team, please visit:

    https://www.redhat.com/en/services/support

Comment 4 vjhouse 2016-04-22 16:33:34 UTC
This is not technical support.  This is a direct result of a patch that was done that created a bug with how OpenSSH is supposed to and does function as normally.  RHEL/CentOS OpenSSH is making a call to look for .k5login that the regular OpenSSH does not do.  Read the patches they did this on purpose and they did not fully test and made a mistake which == bug!

Look at the below code pulled directly from force_krb.patch

- if (krb5_kuserok(krb_context, princ, name)) { 
+ /* krb5_kuserok() returns 1 if .k5login DNE and this is self-login. 
+ * We have to make sure to check .k5users in that case. */ 
+ k5login_exists = ssh_gssapi_k5login_exists(); 
+ /* NOTE: .k5login and .k5users must opened as root, not the user, 
+ * because if they are on a krb5-protected filesystem, user credentials 
+ * to access these files aren't available yet. */ 

+ssh_gssapi_k5login_exists() 
+{ 
+ char file[MAXPATHLEN]; 
+ struct passwd *pw = the_authctxt->pw; 
+ 
+ snprintf(file, sizeof(file), "%s/.k5login", pw->pw_dir); 
+ return access(file, F_OK) == 0; 
+}

The correct method is not to always assume that the user directory is mounted at the time of the access.  This is causing a failure with autofs.  The correct way of coding this is to give the admin an option to set the path of the .k5login or abide by the krb5_pam.so library which the offending code is not communicating with.

I know how to fix the bug and it is simply as repatching each time and pulling out this check that should never be "forced".  This was a sloppy patch and never should have been rolled out.  If I make my own rpm from the base OpenSSH source this issue goes away as well.

Comment 8 Jakub Jelen 2016-05-06 15:30:35 UTC
Created attachment 1154695 [details]
read k5login_directory from krb5.conf and use appropriate path for k5login search

Finally I managed to read through all the related bugs and kerberos stuff used around openssh.

As far as I understand, this problem is direct consequence of the patch applied from the bug #1153011 (am I right Simo?), solving problems of krb5_aname_to_localname() by calling krb5_kuserok(), when the file .k5login is not in home directory, but it does not check if the file directory is already mounted (it is not).

It looks like chicken and egg problem at this time:
 * We don't want to call krb5_kuserok() because it would allow .k5login (can be disabled in krb5.conf by different path)
 * We don't know if there is .k5login, before krb5_kuserok() is called (and home directory mounted).

Probably the best option would be just implement reading  k5login_directory  from  krb5.conf  in sshd (we already use  profile_get_string  for  ccache_name) and checking the configured path instead of the hardcoded one. This should solve the problem in this case, isn't it?

I wrote simple patch and verified basic functionality of gssapi in our tests (but I don't have such setup as above around to test if it really solves the problem). Would it be possible for you to test with this patch?

Comment 9 Simo Sorce 2016-05-06 20:00:59 UTC
I am not sure I'd say the problem is direct consequence of bug #1153011 but they are defeinitely all related.

Reading your patch makes me think you are on the right path to solve this specific problem, as it will allow administrators to change the k5login directory so that their GSSAPI logins do not deadlock on the mount point.

I guess it would be nice to have some documentation that points out how to deal with this whole problem, but I am not sure where it would endup, should it be in RHEL's ssh related documentation ?

Comment 10 vjhouse 2016-05-06 21:03:09 UTC
I looked through the patch itself and it should fix the problem I am having, however when I try and patch and build my own RPM with rpmbuild I am having some issues with some of the requirements of RHEL's OpenSSH as the packages required are not in the regular repos.

groff
libedit-devel
fipscheck-devel

yum says the packages do not exist, need to figure out how to get them into RHEL 7

didn't have these issues with installing the latest OpenSSH source for testing

Any hints to getting this patch installed would be great, this is why I didn't want to code my own patch to begin with as I may not understand what else is going on.  I will keep trying on my own, but if a precompiled rpm is available with the patch or there is another way to patch that I am not doing that would be great as well.  I can just clone VMs till something works out.  I have seen some of these packages elsewhere, but I would trust a download from somewhere controlled by redhat or centos more.

Anyways thank you so much for taking the time already!  This problem has been vexxing me for a year, I had to back off and use a workaround that has its own limitations, was hoping someone else would feel the effect of this elsewhere.  I had placed a support ticket with redhat on this as well and I am waiting for an engineer that has an idea about all of this to look at the problem on that side, but your patch is the move I would have made if I can just get it all to compile correctly and verify that it does work.

Comment 11 vjhouse 2016-05-06 21:06:01 UTC
Could be I have a bad SPEC file as well for openssh, I am using the latest one and just modified it to include the new patch.  Will look at an older one and make sure it has the same issues.

Comment 12 vjhouse 2016-05-06 23:03:36 UTC
figured it out didn't have optional enabled on this system...

Comment 13 vjhouse 2016-05-06 23:22:39 UTC
Very excited, it seems to work for my case.  I just need to do an strace and confirm that sshd is attempting to open my alternate directory that is in the krb5.conf file.

So far GREAT JOB!

Wish I had started this last May.

Comment 14 Jakub Jelen 2016-05-09 06:56:32 UTC
(In reply to Simo Sorce from comment #9)
> Reading your patch makes me think you are on the right path to solve this
> specific problem, as it will allow administrators to change the k5login
> directory so that their GSSAPI logins do not deadlock on the mount point.
> 
> I guess it would be nice to have some documentation that points out how to
> deal with this whole problem, but I am not sure where it would endup, should
> it be in RHEL's ssh related documentation ?

We should certainly add a note into manual page for  sshd, that the server is checking  k5login_directory  option from  krb5.conf  and behaves accordingly. Currently, there is only mentioned

     ~/.k5login

with "redirection" to ksu(1). There might be just note, but more complex guide should be in some kerberos integration guide, mentioning specific problems with samba, autofs and so.

(In reply to vjhouse from comment #13)
> Very excited, it seems to work for my case.  I just need to do an strace and
> confirm that sshd is attempting to open my alternate directory that is in
> the krb5.conf file.

Thank you for testing and glad it worked for you.
Anyway, there are debug logs informing what file is being read, but strace is always right.

Comment 15 Jakub Jelen 2016-05-09 08:05:49 UTC
Proposed extension of manual page for sshd (added the last sentence):

     ~/.k5login
     ~/.k5users
             These files enforce GSSAPI/Kerberos authentication access control.  Further details are described in ksu(1).  The location of the k5login file depends on the configuration option k5login_directory in the krb5.conf(5).

Comment 16 vjhouse 2016-05-09 15:54:12 UTC
strace worked out, however there is a possible issue that I saw in the strace

first instance of the .k5login access:

access("/var/k5login/testuser", F_OK) = -1 ENOENT (No such file or directory)

Path is correct, next two have an extra slash:

access("/var/k5login//testuser", F_OK) = -1 ENOENT (No such file or directory)


krb5.conf setting:
k5login_directory = /var/k5login/

Taking the last "/" out of the k5login_directory path fixes the last 2 calls to access the .k5login.  I will test with a .k5login file set, maybe the extra slash won't hurt.


Like the update to the docs, what was there was really vague especially if working with openssh from other systems.


On the positive with this issue, I learned more about openssh and kerberos this way than I probably would have without the issue.  I think this is the best thing about being able to read the raw source of the patches and the code.

Comment 17 Jakub Jelen 2016-05-10 09:32:53 UTC
Yes, there was a problem with the trailing slash that I already fixed in my repository, but the attached patch was wrong. Thank you for pointing that out. There should be 

	k5login_directory[strlen(k5login_directory)-1] != '/' ? "/" : "",

instead of 

	k5login_directory[strlen(k5login_directory)-1] != "/" ? "" : "/",

As far as I understand that, the first access() should come from ssh and the others already from kerberos library, isn't it? But the additional slash should not generally matter.

Comment 23 errata-xmlrpc 2016-11-03 20:19:58 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2016-2588.html


Note You need to log in before you can comment on or make changes to this bug.