RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 720688 - [RFE] return multiple server addresses to the Kerberos locator plugin
Summary: [RFE] return multiple server addresses to the Kerberos locator plugin
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: sssd
Version: 7.0
Hardware: x86_64
OS: Linux
medium
medium
Target Milestone: pre-dev-freeze
: ---
Assignee: Tomas Halman
QA Contact: Amith
URL:
Whiteboard:
Depends On:
Blocks: 736857 1203710 1205796 1644708 1647919 1695582
TreeView+ depends on / blocked
 
Reported: 2011-07-12 13:56 UTC by Jan Wenzel
Modified: 2023-03-24 13:26 UTC (History)
22 users (show)

Fixed In Version: sssd-1.16.4-11.el7
Doc Type: Enhancement
Doc Text:
Clone Of:
: 1695582 (view as bug list)
Environment:
Last Closed: 2019-08-06 13:02:00 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)
sssd.conf used on the machine (1.72 KB, text/plain)
2011-07-20 10:15 UTC, Jan Wenzel 		no flags Details
View All


Links
System ID Private Priority Status Summary Last Updated
FedoraHosted SSSD 941 0 None None None Never
Github SSSD sssd issues 1983 0 None closed return multiple server addresses to the Kerberos locator plugin 2020-09-16 15:50:44 UTC
Github SSSD sssd issues 2925 0 None closed Add a new option to disable the Kerberos locator plugin completely 2020-09-16 15:50:44 UTC
Red Hat Knowledge Base (Solution) 238013 0 None None None Never
Red Hat Product Errata RHSA-2019:2177 0 None None None 2019-08-06 13:02:43 UTC

Description Jan Wenzel 2011-07-12 13:56:27 UTC 
Description of problem:
I have a definition for kpasswd_server with a different port number than 464 in my /etc/krb5.conf (realm section).
When I call kpasswd for a principal in that realm, it ignores the port number (or the complete kpasswd_server stanza) and uses the default kpasswd port 464, which then fails with the message "Authentication error: Failed reading application request".
If i move the plugin away, it works as expected: communication goes through the correct port.


Version-Release number of selected component (if applicable):
sssd-1.2.1-28.el6_0.4.x86_64
krb5-workstation-1.8.2-3.el6_0.6.x86_64

How reproducible:
=== /etc/krb5.conf ===
[libdefaults]
 default_realm = EXAMPLE.COM
 dns_lookup_realm = false
 dns_lookup_kdc = false
 ticket_lifetime = 24h
 forwardable = true
 allow_weak_crypto = true

[realms]
 EXAMPLE.COM = {
  kdc = kdc:48607
  admin_server = kdc:38607
  kpasswd_server = kdc:28607
 }
=== /var/kerberos/krb5kdc/kdc.conf ===
 EXAMPLE.COM = {
  acl_file = /var/kerberos/krb5kdc/kadm5.acl
  key_stash_file = /var/kerberos/krb5kdc/.k5.EXAMPLE.COM
  dict_file = /usr/share/dict/words
  kadmind_port = 38607
  kpasswd_port = 28607
 }
=== /etc/sysconfig/krb5kdc ===
KRB5KDC_ARGS="-p 48607"
KRB5REALM=EXAMPLE.COM
======================
Steps to Reproduce:
1. kpasswd for user in realm fails
2. mv /usr/lib64/krb5/plugins/libkrb5 /usr/lib64/krb5/plugins/libkrb5.bak
3. kpasswd for user in realm succeeds
  
Actual results:
I would expect that hardcoded settings in /etc/krb5.conf override every default value.

Expected results:
The plugin seems to overwrite my manual configuration with default values.

Additional info:
We use different ports for every service component to start many kerberos environments on that kdc machines. It is absolutely required that these ports are used for password changes.
Comment 2 Stephen Gallagher 2011-07-12 14:03:53 UTC 
Set
krb5_kpasswd = kerberos.example.com:991

in sssd.conf. By default, the locator plugin will use the value of krb5_kdcip if krb5_kpasswd isn't specified separately.

See sssd-krb5(5) for more details.
Comment 3 Jan Wenzel 2011-07-12 14:10:28 UTC 
(In reply to comment #2)
> Set
> krb5_kpasswd = kerberos.example.com:991
> 
> in sssd.conf. By default, the locator plugin will use the value of krb5_kdcip
> if krb5_kpasswd isn't specified separately.
> 
> See sssd-krb5(5) for more details.

I already have
krb5_kpasswd = kdc-01:28607,kdc-02:28607
in my sssd.conf. It does not work.
Note: We are using the ldap backend for kerberos, so failover works.
Comment 4 Stephen Gallagher 2011-07-12 14:15:50 UTC 
Can you try upgrading to SSSD 1.5.x in RHEL 6.1? If I remember correctly, we had a bug at some point where the locator plugin wasn't returning the kpasswd values unless a password-change had been initiated by SSSD (as opposed to kpasswd).
Comment 5 Jan Wenzel 2011-07-15 09:48:00 UTC 
OK, password change with kpassed works with sssd-1.5.1-34.el6_1.1.x86_64 from RHEL 6.1
Comment 6 Jenny Severance 2011-07-15 14:41:32 UTC 
Based on comment 5 .. Closing bug Current Release
Comment 7 Jan Wenzel 2011-07-19 07:27:36 UTC 
Sorry but I have to reopen this bug:
The port argument is only used for the first server entry. If the first server is offline, it tries to connect to the second server with the default port. So this is still a bug with the current version.

Also, the log file says that krb5_kdcip is deprecated and should be replaced by krb5_server. If I do this, it uses the default kpwasswd even for the first server entry.
Comment 9 Jakub Hrozek 2011-07-19 16:43:57 UTC 
(In reply to comment #7)
> Sorry but I have to reopen this bug:
> The port argument is only used for the first server entry. If the first server
> is offline, it tries to connect to the second server with the default port. So
> this is still a bug with the current version.
> 

I'm sorry, but it's not really clear to me what the behavior is vs. what it should be.

Given your example above, if "kdc-01:28607" is down, kpasswd does not try to contact the second server (kdc-02) at all or does it "just" ignore the port number?

> Also, the log file says that krb5_kdcip is deprecated and should be replaced by
> krb5_server. If I do this, it uses the default kpwasswd even for the first
> server entry.

Sorry, but which server do you mean by "default kpasswd"? Are you saying that if sssd defines krb5_server and krb5_kpasswd, kpasswd only contacts the KDC and not the kpasswd server?

Would you mind attaching your sssd.conf? 

Also, it would be helpful to see the logs - can you please put "debug_level = 9" into the domain section of your /etc/sssd/sssd.conf, restarting sssd, re-running the testcase and then attaching /var/log/sssd/sssd_$DOMAINNAME.log

Of course, feel free to sanitize the logs by removing sensitive data if needed.

Thank you!
Comment 12 Jan Wenzel 2011-07-20 10:16:18 UTC 
(In reply to comment #9)
> (In reply to comment #7)
> > Sorry but I have to reopen this bug:
> > The port argument is only used for the first server entry. If the first server
> > is offline, it tries to connect to the second server with the default port. So
> > this is still a bug with the current version.
> > 
> 
> I'm sorry, but it's not really clear to me what the behavior is vs. what it
> should be.
> 
> Given your example above, if "kdc-01:28607" is down, kpasswd does not try to
> contact the second server (kdc-02) at all or does it "just" ignore the port
> number?

It "just" ignores the port number for the second server.

> > Also, the log file says that krb5_kdcip is deprecated and should be replaced by
> > krb5_server. If I do this, it uses the default kpwasswd even for the first
> > server entry.
> 
> Sorry, but which server do you mean by "default kpasswd"? Are you saying that
> if sssd defines krb5_server and krb5_kpasswd, kpasswd only contacts the KDC and
> not the kpasswd server?

I mean the default kpasswd port. kpasswd contacts the defined kpasswd server with the default kpasswd port instead of the configured port if i replace krb5_kdcip with krb5_server.

> 
> Would you mind attaching your sssd.conf? 
As you wish.

> Also, it would be helpful to see the logs - can you please put "debug_level =
> 9" into the domain section of your /etc/sssd/sssd.conf, restarting sssd,
> re-running the testcase and then attaching /var/log/sssd/sssd_$DOMAINNAME.log
> 
> Of course, feel free to sanitize the logs by removing sensitive data if needed.

I will do this if needed.

> Thank you!
Thank you, too!
Comment 13 Jakub Hrozek 2011-07-20 13:25:46 UTC 
(In reply to comment #12)
> (In reply to comment #9)
> > (In reply to comment #7)
> > > Sorry but I have to reopen this bug:
> > > The port argument is only used for the first server entry. If the first server
> > > is offline, it tries to connect to the second server with the default port. So
> > > this is still a bug with the current version.
> > > 
> > 
> > I'm sorry, but it's not really clear to me what the behavior is vs. what it
> > should be.
> > 
> > Given your example above, if "kdc-01:28607" is down, kpasswd does not try to
> > contact the second server (kdc-02) at all or does it "just" ignore the port
> > number?
> 
> It "just" ignores the port number for the second server.
> 

I am surprised it even tries to contact the second server, for two reasons:

First, the fix which always resolves the kpasswd server from SSSD is not present in sssd-1.5.1-34.el6_1.1, it went to the RHEL6.2 branch only as far as  I can tell.

Second, even if SSSD resolves the first kpasswd server, it has no way of telling it needs to fail over if the "kpasswd" utility is used. The "kpasswd" utility completely bypasses SSSD and the PAM stack and talks directly to the kpasswd server (unlike passwd).
Comment 14 Jan Wenzel 2011-07-20 14:59:59 UTC 
> I am surprised it even tries to contact the second server, for two reasons:
>
> First, the fix which always resolves the kpasswd server from SSSD is not
> present in sssd-1.5.1-34.el6_1.1, it went to the RHEL6.2 branch only as far as 
> I can tell.
> 
> Second, even if SSSD resolves the first kpasswd server, it has no way of
> telling it needs to fail over if the "kpasswd" utility is used. The "kpasswd"
> utility completely bypasses SSSD and the PAM stack and talks directly to the
> kpasswd server (unlike passwd).
The kpasswd utility uses the krb5_locator plugin as mentioned in the subject of this bug. If it completely bypasses sssd (if we move the plugin away) it works as expected. So this must be a sssd related bug.

I can prepare a debug level 10 log of this domain in the next week if this will help you.
Comment 15 Jakub Hrozek 2011-07-20 15:33:56 UTC 
(In reply to comment #14)
> > I am surprised it even tries to contact the second server, for two reasons:
> >
> > First, the fix which always resolves the kpasswd server from SSSD is not
> > present in sssd-1.5.1-34.el6_1.1, it went to the RHEL6.2 branch only as far as 
> > I can tell.
> > 
> > Second, even if SSSD resolves the first kpasswd server, it has no way of
> > telling it needs to fail over if the "kpasswd" utility is used. The "kpasswd"
> > utility completely bypasses SSSD and the PAM stack and talks directly to the
> > kpasswd server (unlike passwd).
> The kpasswd utility uses the krb5_locator plugin as mentioned in the subject of
> this bug. If it completely bypasses sssd (if we move the plugin away) it works
> as expected. So this must be a sssd related bug.
> 
> I can prepare a debug level 10 log of this domain in the next week if this will
> help you.

Right, kpasswd (or rather libkrb) uses the server it gets via krb5_locator plugin. That's not the problem.

The problem is that kpasswd has no way of telling SSSD that the server given to libkrb5 by the plugin is down because it does not talk to SSSD but rather to the kpasswd server directly.

If passwd was used instead of kpasswd, the change password request would go through the PAM stack and eventually reach pam_sss and SSSD. SSSD has means to detect that the server is down and is able to fail over.
Comment 16 Marko Karg 2011-10-27 14:29:04 UTC 
My customer would like to see this in 6.3 if anyhow possible - can we get pm ack on this one please?

Thanks

Marko
Comment 31 Jakub Hrozek 2016-11-23 14:05:29 UTC 
Thank you for filing this bug report.

Since fixing this bug requires work in the upstream SSSD project which is not scheduled for the immediate future, I added a conditional development nack, pending upstream availability.
Comment 33 Sumit Bose 2017-05-05 13:20:20 UTC 
Sorry, this is still work-in-progress. I have a patch for the plugin itself so that it can handle multiple addresses. But the SSSD side to make those addresses available to the plugin is still missing.
Comment 38 Fabiano Fidêncio 2018-06-14 18:17:05 UTC 
Fixed as part of the following series ...
 master:
  efae950
  9f68324
  c1fbc6b
  2124275
  cc79227
  d91661e
  4759a48
  f28d995
Comment 39 Jakub Hrozek 2018-06-14 19:44:30 UTC 
I'm going to move the bug back to ASSIGNED. I know the upstream tickets were really less than clear, cross-referenced each other and so did the bugzillas.

So to make it clear what was pushed to upstream:
 - the Kerberos locator plugin is now able to consume multiple addresses from the kdcinfo files. This will most probably be in 7.6 (modulo the usual disclaimer about future releases..)
 - BUT there is no mechanism yet in SSSD which would write multiple addresses into the kdcinfo files

 - the patches add the ability to generate the kdcinfo files for trusted domains of directly joined AD clients and IPA masters in a trust relationship with an AD domain
  - BUT there is no mechanism yet for writing the kdcinfo files on IPA clients in an IPA-AD trust setup. I'm working on a subsequent patch that would implement this and I hope that the patch makes 7.6 (modulo the usual disclaimer about future releases..)

We're now working on another, subsequent patchset that would write multiple addresses into kdcinfo files, but /only in the IPA-AD client case/.

I hope it is clearer what was implemented and what is still in the works.
Comment 40 Jakub Hrozek 2018-06-20 19:10:47 UTC 
I'm moving the bug to 7.7, because the scope of the work *as described in the subject* is still not implemented and not even started. But at the same time, I believe that the use-case tracked in most of the customer cases is either implemented in the 7.6 codebase already or being worked on and planned.

To summarize the previous comment again:
 - the Kerberos locator plugin can consume multiple addresses

 - clients joined directly to AD generate kdcinfo files to be consumed by the kdcinfo files. So far the kdcinfo files would only contain one address, but previously the files were not created at all. This is tracked in https://bugzilla.redhat.com/show_bug.cgi?id=1494690

 - the previous point applies also for IPA servers in a trust relationship with an AD domain

 - currently I'm working on generating the kdcinfo files for IPA clients in a trust relationship. The patches work, but need a little cleanup. I'm quite hopeful they would make 7.6 and this work is tracked with https://bugzilla.redhat.com/show_bug.cgi?id=1416528
Comment 42 Jakub Hrozek 2019-03-19 22:40:57 UTC 
Upstream ticket:
https://pagure.io/SSSD/sssd/issue/3973
Comment 43 Jakub Hrozek 2019-03-19 22:44:37 UTC 
First ticket fixed:
    master: 63ccbfe
    sssd-1-16: 96e4d71
Comment 44 Jakub Hrozek 2019-03-23 19:38:11 UTC 
Another master PR:
* master: 208a79a
* sssd-1-16: c225ed7
Comment 45 Jakub Hrozek 2019-04-02 20:40:51 UTC 
Final PR:
 * master: e8d806d9bbb1ba288ed6a83158113f4d8f8a8929
 * sssd-1-16: ec0c31a71ef8ec61e11b1c6d21edcf9b923ce4ca
Comment 47 anuja 2019-05-15 11:18:07 UTC 
Hi thalman,
could you add verification steps for this.
Comment 48 Tomas Halman 2019-05-16 07:32:30 UTC 
in sssd.conf specify more servers, Some of them doesn't have to exist for the test, put at least 4 servers and at least 2 backup servers

krb5_server = server1, server2, server3, server4
krb5_backup_server = bck1, bck2
krb5_use_kdcinfo = true

You can do the same for krb5_kpasswd, krb5_backup_kpasswd options

* re/start sssd
* log into system with kerberos credentials
* check /var/lib/sss/pubconf for created kdcifo file. 
Expected result:
** KDC info file should have 3 servers + one backup server, one server per line.
** Some servers can have their name in file, some can be already resolved to IP address
** Order of servers in file can change depending on failover mechanism

* set krb5_kdcinfo_lookahead to different values (default is "3:1", so try "4:0", "4:2") 
Expected result:
** Number of servers in KDC info file changes accordingly
Comment 50 anuja 2019-05-17 13:43:47 UTC 
Hi, Jackub, Tomas 
Could you have a look in comment #49
If steps are wrong could you add  correct steps based on above test.
Comment 57 errata-xmlrpc 2019-08-06 13:02:00 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2019:2177
Note You need to log in before you can comment on or make changes to this bug.