Bug 1328243

Also reference patch: openssh-6.6p1-force_krb.patch

    Thank you for taking the time to enter a bug report with us. We appreciate the feedback and look to use reports such as this to guide our efforts at improving our products. That being said, this bug tracking system is not a mechanism for requesting support, and we are not able to  guarantee the timeliness or suitability of a resolution.

    If this issue is critical or in any way time sensitive, please raise a ticket through your regular Red Hat support channels to make certain  it receives the proper attention and prioritization to assure a timely resolution. 

    For information on how to contact the Red Hat production support team, please visit:

    https://www.redhat.com/en/services/support

This is not technical support.  This is a direct result of a patch that was done that created a bug with how OpenSSH is supposed to and does function as normally.  RHEL/CentOS OpenSSH is making a call to look for .k5login that the regular OpenSSH does not do.  Read the patches they did this on purpose and they did not fully test and made a mistake which == bug!

Look at the below code pulled directly from force_krb.patch

- if (krb5_kuserok(krb_context, princ, name)) { 
+ /* krb5_kuserok() returns 1 if .k5login DNE and this is self-login. 
+ * We have to make sure to check .k5users in that case. */ 
+ k5login_exists = ssh_gssapi_k5login_exists(); 
+ /* NOTE: .k5login and .k5users must opened as root, not the user, 
+ * because if they are on a krb5-protected filesystem, user credentials 
+ * to access these files aren't available yet. */ 

+ssh_gssapi_k5login_exists() 
+{ 
+ char file[MAXPATHLEN]; 
+ struct passwd *pw = the_authctxt->pw; 
+ 
+ snprintf(file, sizeof(file), "%s/.k5login", pw->pw_dir); 
+ return access(file, F_OK) == 0; 
+}

The correct method is not to always assume that the user directory is mounted at the time of the access.  This is causing a failure with autofs.  The correct way of coding this is to give the admin an option to set the path of the .k5login or abide by the krb5_pam.so library which the offending code is not communicating with.

I know how to fix the bug and it is simply as repatching each time and pulling out this check that should never be "forced".  This was a sloppy patch and never should have been rolled out.  If I make my own rpm from the base OpenSSH source this issue goes away as well.

Created attachment 1154695 [details]
read k5login_directory from krb5.conf and use appropriate path for k5login search

Finally I managed to read through all the related bugs and kerberos stuff used around openssh.

As far as I understand, this problem is direct consequence of the patch applied from the bug #1153011 (am I right Simo?), solving problems of krb5_aname_to_localname() by calling krb5_kuserok(), when the file .k5login is not in home directory, but it does not check if the file directory is already mounted (it is not).

It looks like chicken and egg problem at this time:
 * We don't want to call krb5_kuserok() because it would allow .k5login (can be disabled in krb5.conf by different path)
 * We don't know if there is .k5login, before krb5_kuserok() is called (and home directory mounted).

Probably the best option would be just implement reading  k5login_directory  from  krb5.conf  in sshd (we already use  profile_get_string  for  ccache_name) and checking the configured path instead of the hardcoded one. This should solve the problem in this case, isn't it?

I wrote simple patch and verified basic functionality of gssapi in our tests (but I don't have such setup as above around to test if it really solves the problem). Would it be possible for you to test with this patch?

I am not sure I'd say the problem is direct consequence of bug #1153011 but they are defeinitely all related.

Reading your patch makes me think you are on the right path to solve this specific problem, as it will allow administrators to change the k5login directory so that their GSSAPI logins do not deadlock on the mount point.

I guess it would be nice to have some documentation that points out how to deal with this whole problem, but I am not sure where it would endup, should it be in RHEL's ssh related documentation ?

I looked through the patch itself and it should fix the problem I am having, however when I try and patch and build my own RPM with rpmbuild I am having some issues with some of the requirements of RHEL's OpenSSH as the packages required are not in the regular repos.

groff
libedit-devel
fipscheck-devel

yum says the packages do not exist, need to figure out how to get them into RHEL 7

didn't have these issues with installing the latest OpenSSH source for testing

Any hints to getting this patch installed would be great, this is why I didn't want to code my own patch to begin with as I may not understand what else is going on.  I will keep trying on my own, but if a precompiled rpm is available with the patch or there is another way to patch that I am not doing that would be great as well.  I can just clone VMs till something works out.  I have seen some of these packages elsewhere, but I would trust a download from somewhere controlled by redhat or centos more.

Anyways thank you so much for taking the time already!  This problem has been vexxing me for a year, I had to back off and use a workaround that has its own limitations, was hoping someone else would feel the effect of this elsewhere.  I had placed a support ticket with redhat on this as well and I am waiting for an engineer that has an idea about all of this to look at the problem on that side, but your patch is the move I would have made if I can just get it all to compile correctly and verify that it does work.

Could be I have a bad SPEC file as well for openssh, I am using the latest one and just modified it to include the new patch.  Will look at an older one and make sure it has the same issues.

figured it out didn't have optional enabled on this system...

Very excited, it seems to work for my case.  I just need to do an strace and confirm that sshd is attempting to open my alternate directory that is in the krb5.conf file.

So far GREAT JOB!

Wish I had started this last May.

(In reply to Simo Sorce from comment #9)
> Reading your patch makes me think you are on the right path to solve this
> specific problem, as it will allow administrators to change the k5login
> directory so that their GSSAPI logins do not deadlock on the mount point.
> 
> I guess it would be nice to have some documentation that points out how to
> deal with this whole problem, but I am not sure where it would endup, should
> it be in RHEL's ssh related documentation ?

We should certainly add a note into manual page for  sshd, that the server is checking  k5login_directory  option from  krb5.conf  and behaves accordingly. Currently, there is only mentioned

     ~/.k5login

with "redirection" to ksu(1). There might be just note, but more complex guide should be in some kerberos integration guide, mentioning specific problems with samba, autofs and so.

(In reply to vjhouse from comment #13)
> Very excited, it seems to work for my case.  I just need to do an strace and
> confirm that sshd is attempting to open my alternate directory that is in
> the krb5.conf file.

Thank you for testing and glad it worked for you.
Anyway, there are debug logs informing what file is being read, but strace is always right.

Proposed extension of manual page for sshd (added the last sentence):

     ~/.k5login
     ~/.k5users
             These files enforce GSSAPI/Kerberos authentication access control.  Further details are described in ksu(1).  The location of the k5login file depends on the configuration option k5login_directory in the krb5.conf(5).

strace worked out, however there is a possible issue that I saw in the strace

first instance of the .k5login access:

access("/var/k5login/testuser", F_OK) = -1 ENOENT (No such file or directory)

Path is correct, next two have an extra slash:

access("/var/k5login//testuser", F_OK) = -1 ENOENT (No such file or directory)


krb5.conf setting:
k5login_directory = /var/k5login/

Taking the last "/" out of the k5login_directory path fixes the last 2 calls to access the .k5login.  I will test with a .k5login file set, maybe the extra slash won't hurt.


Like the update to the docs, what was there was really vague especially if working with openssh from other systems.


On the positive with this issue, I learned more about openssh and kerberos this way than I probably would have without the issue.  I think this is the best thing about being able to read the raw source of the patches and the code.

Yes, there was a problem with the trailing slash that I already fixed in my repository, but the attached patch was wrong. Thank you for pointing that out. There should be 

	k5login_directory[strlen(k5login_directory)-1] != '/' ? "/" : "",

instead of 

	k5login_directory[strlen(k5login_directory)-1] != "/" ? "" : "/",

As far as I understand that, the first access() should come from ssh and the others already from kerberos library, isn't it? But the additional slash should not generally matter.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2016-2588.html