Bug 1598457

Moving to component sssd as the utility 'sss_ssh_authorizedkeys' is provided by sssd packages.

There is no default place in the AD LDAP schema to store SSH public keys hence SSSD on the IPA server does not read any related attribute by default.

To tell SSSD on the IPA server to read this attribute please add

[domain/your.ipa.domain/your.ad.domain]
ldap_user_ssh_public_key = sshPublicKey

to sssd.conf to read and store the sshPublicKey attribute for AD users from your.ad.domain. If there are multiple domains in you trusted AD forest with user with ssh keys you have to add a similar entry in sssd.conf for the other domains as well.

This must be done on all IPA servers (it would be sufficient to add it on the trust agents but it might lead to less confusion to do it on all servers). The IPA client do not need it because they will get the data directly from the IPA servers.

HTH

bye,
Sumit

Hi,

Unfortunately it didn't help.

Excerpt from /var/logs/sssd/sssd.log:

(Mon Jul  9 10:29:43:850778 2018) [sssd] [sss_ini_call_validators] (0x0020): [rule/allowed_subdomain_options]: Attribute 'ldap_user_ssh_public_key' is not allowed in section 'domain/IPA.DOMAIN/AD.DOMAIN'. Check for typos

sssd.conf
[domain/IPA.DOMAIN]
cache_credentials = True
krb5_store_password_if_offline = True
ipa_domain = rl.lan
id_provider = ipa
auth_provider = ipa
access_provider = ipa
ipa_hostname = HOSTNAME.IPA.DOMAIN
chpass_provider = ipa
ipa_server = HOSTNAME.IPA.DOMAIN
ipa_server_mode = True
ldap_tls_cacert = /etc/ipa/ca.crt

[domain/IPA.DOMAIN/AD.DOMAIN]
ldap_user_ssh_public_key = sshPublicKey

[sssd]
services = sudo, nss, ifp, pam, ssh
domains = IPA.DOMAIN

[nss]
memcache_timeout = 600
homedir_substring = /home

[pam]

[sudo]

[autofs]

[ssh]

[pac]

[ifp]

[secrets]

The two domain name given in the sub-domain section name must match exactly what SSSD use as domain names. Please call 'sssctl domain-list' to get the expected spelling. E.g.:

# sssctl domain-list
rhel75.devel
ChIlD.ad.devel
ad.devel

-> sssd.conf
...
[domain/rhel75.devel/ChIlD.ad.devel]
ldap_user_ssh_public_key = myChildAttrbuteName


If you switch on debug_level=9 you should see in the domain log the requested attributes:

...
(Mon Jul  9 12:28:12 2018) [sssd[be[rhel75.devel]]] [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [accountExpires]
(Mon Jul  9 12:28:12 2018) [sssd[be[rhel75.devel]]] [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [userAccountControl]
(Mon Jul  9 12:28:12 2018) [sssd[be[rhel75.devel]]] [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [myChildAttrbuteName]
(Mon Jul  9 12:28:12 2018) [sssd[be[rhel75.devel]]] [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [userCertificate;binary]
(Mon Jul  9 12:28:12 2018) [sssd[be[rhel75.devel]]] [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [mail]
(Mon Jul  9 12:28:12 2018) [sssd[be[rhel75.devel]]] [sdap_get_generic_ext_step] (0x2000): ldap_search_ext called, msgid = 5
...

HTH

bye,
Sumit

Hi,

Checked the spelling - everything as expected (no typos, all lowercase)
sudo sssctl domain-list
ipa.domain
ad.domain

The error in the logs is the same:

(Mon Jul  9 13:30:40:945930 2018) [sssd] [sss_ini_call_validators] (0x0020): [rule/allowed_subdomain_options]: Attribute 'ldap_user_ssh_public_key' is not allowed in section 'domain/ipa.domain/ad.domain'. Check for typos.


However, clearing cache helped...
sss_cache -E


And now that I see it, I can only login once after clearing cache. Is this expected?

The debug message is shown because the validator is too picky.

Do I understand correctly that you can log in with ssh and pubkey authentication once and before you can log in a second time you have to call 'sss_cache -E'?

If run 'ssh ...', 'exit' cycles multiple times without issues. Can you added 'debug_level=9' to the [ssh] section of sssd.conf, restart SSSD, run your test again and attach /var/log/sssd/sssd_ssh.log to this ticket?

bye,
Sumit

Hi Sumit,

Yes, you understand correctly.

Logs attached. (only logs between running ssh and after successful/unsuccessful login) 

Success.txt:
1) cleared cache
2) restarted sssd.conf
3) run ssh
4) got bash

Fail.txt
1) logout
1) run ssh again
3) get prompted for password

Created attachment 1457811 [details]
Failed ssh with public-key from AD (after initial success)

Created attachment 1457813 [details]
Successful log to ssh server via AD public-key

Additionally, this only happens on ipa-servers (trust agents), and not other linux servers (ipa-clients).

Difference in configuration between ipa-servers and ipa-clients is in sssd.conf

[domain/ipa.domain/ad.domain]
ldap_user_ssh_public_key = sshPublicKey

(and of course ipa_server_mode = True, etc...)

Thanks the logs do not show anything suspicious. Next would be to check the sshd logs. For this please add 'LogLevel DEBUG3' to /etc/ssh/sshd_config and restart sshd.

sshd will use the /usr/bin/sss_ssh_authorizedkeys command to get the allowed public keys from SSSD. I can see the following for a successful authentication in my logs:

Jul  9 16:30:12 ipaserver75 sshd[4503]: debug3: subprocess: AuthorizedKeysCommand command "/usr/bin/sss_ssh_authorizedkeys tu1" running as nobody
Jul  9 16:30:12 ipaserver75 sshd[4503]: debug1: temporarily_use_uid: 99/99 (e=0/0)
Jul  9 16:30:12 ipaserver75 sshd[4503]: debug1: restore_uid: 0/0
Jul  9 16:30:12 ipaserver75 sshd[4503]: debug3: subprocess: AuthorizedKeysCommand pid 4506
Jul  9 16:30:12 ipaserver75 sshd[4503]: debug1: temporarily_use_uid: 99/99 (e=0/0)
Jul  9 16:30:12 ipaserver75 sshd[4503]: debug1: matching key found: file /usr/bin/sss_ssh_authorizedkeys, line 1 RSA SHA256:0yvy65NczeZGoKC7vDsfGdFie613hJu4FXDYm+HytNs
Jul  9 16:30:12 ipaserver75 sshd[4503]: debug1: restore_uid: 0/0
Jul  9 16:30:12 ipaserver75 sshd[4503]: debug3: mm_answer_keyallowed: key 0x55f499fb36a0 is allowed
Jul  9 16:30:12 ipaserver75 sshd[4503]: debug3: mm_request_send entering: type 23
Jul  9 16:30:12 ipaserver75 sshd[4503]: debug3: mm_key_verify entering [preauth]
Jul  9 16:30:12 ipaserver75 sshd[4503]: debug3: mm_request_send entering: type 24 [preauth]
Jul  9 16:30:12 ipaserver75 sshd[4503]: debug3: mm_key_verify: waiting for MONITOR_ANS_KEYVERIFY [preauth]
Jul  9 16:30:12 ipaserver75 sshd[4503]: debug3: mm_request_receive_expect entering: type 25 [preauth]
Jul  9 16:30:12 ipaserver75 sshd[4503]: debug3: mm_request_receive entering [preauth]
Jul  9 16:30:12 ipaserver75 sshd[4503]: debug3: mm_request_receive entering
Jul  9 16:30:12 ipaserver75 sshd[4503]: debug3: monitor_read: checking request 24
Jul  9 16:30:12 ipaserver75 sshd[4503]: debug3: mm_answer_keyverify: key 0x55f499fb3510 signature verified
Jul  9 16:30:12 ipaserver75 sshd[4503]: debug3: mm_request_send entering: type 25
Jul  9 16:30:12 ipaserver75 sshd[4503]: debug3: mm_request_receive_expect entering: type 102
Jul  9 16:30:12 ipaserver75 sshd[4503]: debug3: mm_request_receive entering
Jul  9 16:30:12 ipaserver75 sshd[4503]: debug1: do_pam_account: called
Jul  9 16:30:12 ipaserver75 sshd[4503]: debug3: PAM: do_pam_account pam_acct_mgmt = 0 (Success)
Jul  9 16:30:12 ipaserver75 sshd[4503]: debug3: mm_request_send entering: type 103
Jul  9 16:30:12 ipaserver75 sshd[4503]: Accepted publickey for tu1 from ::1 port 54020 ssh2: RSA SHA256:0yvy65NczeZGoKC7vDsfGdFie613hJu4FXDYm+HytNs

Do you see any errors here?

Thank you for the ssh logs. I assume the key gets removed from the cache during the access control check where SSSD unconditionally refreshes the user entry to make sure the group memberships of the user are up-to-date for the access control checks.

You could confirm this by calling "/usr/bin/sss_ssh_authorizedkeys USER" manually after logging in. Here no key should be returned. If that's the case I'd like to ask you to add debug_level=9 to the [domain/...] section of sssd.conf and restart SSSD.

Then login with ssh and public-key authentication and call the sss_ssh_authorizedkeys. Please attach the SSSD domain log file as a private attachement to this ticket (or send it to me directly if you prefer).

I have an idea, maybe the second user lookup uses the Global Catalog, which does not have the sshPublicKey attribute replicated. I'll try to verify this in my setup as well.

Hi,

Yes, that is the case. (No key is returned after logging in, when manually calling sss_ssh_authorizedkeys USER).

Will upload the logs now.

So, I tried your suggestion:

sshPublicKey is set to "Replicate To Global Catalog" on ActiveDirectory.

It works now on ipa-servers as well. :)

Thank you!

Oh, great, if it is easy for you to replicate sshPublicKey to the Global Catalog then this is the best solution.

The logs you send say 'Removing attribute [sshPublicKey]' when reading from the global catalog. This is kind of a known issue with the Global Catalog https://pagure.io/SSSD/sssd/issue/2474. But since we recently add support the read the list of replicated attributes (partial attribure set) we might now have a proper way to decide when to remove an attribute and when to keep it when getting data from the Global Catalog.

So there are two thing to fix:
 - the config validator should accept ldap_user_ssh_public_key (and I guess all other ldap_user_* and ldap_group_* options) for subdomain configuration
 - https://pagure.io/SSSD/sssd/issue/2474

(In reply to Sumit Bose from comment #18)
> Oh, great, if it is easy for you to replicate sshPublicKey to the Global
> Catalog then this is the best solution.
> 
> The logs you send say 'Removing attribute [sshPublicKey]' when reading from
> the global catalog. This is kind of a known issue with the Global Catalog
> https://pagure.io/SSSD/sssd/issue/2474. But since we recently add support
> the read the list of replicated attributes (partial attribure set) we might
> now have a proper way to decide when to remove an attribute and when to keep
> it when getting data from the Global Catalog.
> 

Thank you for your detective work on this. I renamed the bug so that it's easier for us to see what is going on in the future.

> So there are two thing to fix:
>  - the config validator should accept ldap_user_ssh_public_key (and I guess
> all other ldap_user_* and ldap_group_* options) for subdomain configuration

https://pagure.io/SSSD/sssd/issue/3772

>  - https://pagure.io/SSSD/sssd/issue/2474

OK, I'll link this BZ with https://pagure.io/SSSD/sssd/issue/3538 as well.

Upstream ticket:
https://pagure.io/SSSD/sssd/issue/2474

Upstream ticket:
https://pagure.io/SSSD/sssd/issue/3538

*** Bug 1666042 has been marked as a duplicate of this bug. ***

Upstream fixes:
    master:
        3cb9a3d
        b2352a0
        eaece8b
    sssd-1-16:
        f80dad6
        0b5a359
        8ffc64c

Verified as per steps from comment #26 and comment #27

Verified Using Version:
ipa-server-4.8.0-8.module+el8.1.0+3977+ec23ef34.x86_64
sssd-ipa-2.2.0-11.el8.x86_64
------------------------------------------------------

[root@ipaqavmd ~]# grep "subdomain_homedir = %o" /etc/sssd/sssd.conf 
subdomain_homedir = %o
[root@ipaqavmd ~]# service sssd stop; rm -rf /var/lib/sss/{db,mc}/*; service sssd start
Redirecting to /bin/systemctl stop sssd.service
Redirecting to /bin/systemctl start sssd.service
[root@ipaqavmd ~]# 
[root@ipaqavmd ~]# getent initgroups -s sss bz1194345
bz1194345 1577600513
[root@ipaqavmd ~]# getent passwd bz1194345
bz1194345:*:1577620053:1577620053:bz1194345:/my/test/homedir:
[root@ipaqavmd ~]# 
[root@ipaqavmd ~]# grep "get_subdomain_homedir_of_user failed" /var/log/sssd/sssd_testrelm.test.log 
[root@ipaqavmd ~]# echo $?
1
[root@ipaqavmd ~]# 

As per fix after 'getent initgroups -s sss' there are no errors are reported in logs.
And the home directory is as expected after 'getent initgroups -s sss'
Based on this marking bz as verified.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2019:3651