Red Hat Bugzilla – Bug 1007969
sss tools do not have an option to remove the sssd database
Last modified: 2016-11-04 03:09:45 EDT
Description of problem: realmd should clean sssd database after leave. The database is located in /var/lib/sss/db/cache_$domain.ldb. Version-Release number of selected component (if applicable): realmd-0.14.6-1.el7 How reproducible: always Steps to Reproduce: 1. Join via sssd 2. leave 3. check the content of /var/lib/sss/db/ Additional info: So far only one case was found where this matter and the case is not really typical for production environment so this has low priority. The case is like this: 1/ join to a server e.g. ipa.baseos.qe 2/ login as a remote user e.g. amy@ipa.baseos.qe 3/ leave 4/ join again but to another server that by the way server the same doman ipa.baseos.qe, but it is a completely independent server 5/ login as amy@ipa.baseos.qe does not work unless the have the same ID on booth servers
We do run sss_cache. Could you provide verbose 'realm leave' output? Also, how are you examining the contents of the cache?
I hope the test below make the case clearer. I'm cc-ing also jhrozek. 0 [root@rhel7 ~ ]# cat /etc/resolv.conf # Generated by NetworkManager nameserver 10.34.37.24 0 [root@rhel7 ~ ]# ls -l /var/lib/sss/db/ total 0 0 [root@rhel7 ~ ]# realm -v join ipa.baseos.qe * Resolving: _ldap._tcp.ipa.baseos.qe * Performing LDAP DSE lookup on: 10.34.37.24 * Successfully discovered: ipa.baseos.qe Password for admin: * Required files: /usr/sbin/ipa-client-install, /usr/sbin/oddjobd, /usr/libexec/oddjob/mkhomedir, /usr/sbin/sssd * LANG=C /usr/sbin/ipa-client-install --domain ipa.baseos.qe --realm IPA.BASEOS.QE --mkhomedir --enable-dns-updates --unattended --force-join --principal admin -W --force-ntpd Discovery was successful! Hostname: rhel7.pkis.net Realm: IPA.BASEOS.QE DNS Domain: ipa.baseos.qe IPA Server: sec-ipa1.ipa.baseos.qe BaseDN: dc=ipa,dc=baseos,dc=qe Synchronizing time with KDC... Successfully retrieved CA cert Subject: CN=Certificate Authority,O=IPA.BASEOS.QE Issuer: CN=Certificate Authority,O=IPA.BASEOS.QE Valid From: Tue Jul 23 12:18:48 2013 UTC Valid Until: Sat Jul 23 12:18:48 2033 UTC Enrolled in IPA realm IPA.BASEOS.QE Created /etc/ipa/default.conf Configured /etc/sssd/sssd.conf Configured /etc/krb5.conf for IPA realm IPA.BASEOS.QE Hostname (rhel7.pkis.net) not found in DNS Failed to update DNS records. Adding SSH public key from /etc/ssh/ssh_host_dsa_key.pub Adding SSH public key from /etc/ssh/ssh_host_rsa_key.pub host_mod: Unknown option: no_members Failed to upload host SSH public keys. SSSD enabled Configured /etc/openldap/ldap.conf NTP enabled Configured /etc/ssh/ssh_config Configured /etc/ssh/sshd_config Client configuration complete. * /usr/bin/systemctl enable sssd.service * /usr/bin/systemctl restart sssd.service * /usr/bin/sh -c /usr/sbin/authconfig --update --enablesssd --enablesssdauth --enablemkhomedir --nostart && /usr/bin/systemctl enable oddjobd.service * Successfully enrolled machine in realm 0 [root@rhel7 ~ ]# 0 [root@rhel7 ~ ]# ls -l /var/lib/sss/db/ total 3772 -rw-------. 1 root root 1286144 Sep 17 09:01 cache_ipa.baseos.qe.ldb -rw-------. 1 root root 1121 Sep 17 09:01 ccache_IPA.BASEOS.QE -rw-------. 1 root root 1286144 Sep 17 09:01 config.ldb -rw-------. 1 root root 1286144 Sep 17 09:01 sssd.ldb 0 [root@rhel7 ~ ]# 0 [root@rhel7 ~ ]# getent amy@ipa.baseos.qe Unknown database: amy@ipa.baseos.qe Try `getent --help' or `getent --usage' for more information. 1 [root@rhel7 ~ ]# getent passwd amy@ipa.baseos.qe amy@ipa.baseos.qe:*:1365200005:1365200005:Amy Amy:/home/amy:/bin/sh 0 [root@rhel7 ~ ]# ssh amy@ipa.baseos.qe@localhost The authenticity of host 'localhost (<no hostip for proxy command>)' can't be established. RSA key fingerprint is c5:2e:97:10:26:7a:6d:f5:9e:a7:44:92:4f:0b:d7:a8. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'localhost' (RSA) to the list of known hosts. amy@ipa.baseos.qe@localhost's password: Creating home directory for amy@ipa.baseos.qe. -sh-4.2$ exit logout Connection to localhost closed. 0 [root@rhel7 ~ ]# 0 [root@rhel7 ~ ]# ls -l /tmp/krb5cc_1365200005_zJcbJ6 -rw-------. 1 amy@ipa.baseos.qe amy@ipa.baseos.qe 530 Sep 17 09:03 /tmp/krb5cc_1365200005_zJcbJ6 0 [root@rhel7 ~ ]# 0 [root@rhel7 ~ ]# realm -v leave ipa.baseos.qe * LANG=C /usr/sbin/ipa-client-install --uninstall --unattended Unenrolling client from IPA server Removing Kerberos service principals from /etc/krb5.keytab Disabling client Kerberos and LDAP configurations Restoring client configuration files nscd daemon is not installed, skip configuration nslcd daemon is not installed, skip configuration Client uninstall complete. * Removing entries from keytab for realm * /usr/sbin/sss_cache --users --groups --netgroups --services --autofs-maps * Removing domain configuration from sssd.conf * /usr/sbin/authconfig --update --disablesssdauth --nostart * /usr/bin/systemctl disable sssd.service rm '/etc/systemd/system/multi-user.target.wants/sssd.service' * /usr/bin/systemctl stop sssd.service * Successfully unenrolled machine from realm 0 [root@rhel7 ~ ]# 0 [root@rhel7 ~ ]# ls -l /var/lib/sss/db/ total 4088 -rw-------. 1 root root 1609728 Sep 17 09:03 cache_ipa.baseos.qe.ldb -rw-------. 1 root root 1121 Sep 17 09:01 ccache_IPA.BASEOS.QE -rw-------. 1 root root 1286144 Sep 17 09:01 config.ldb -rw-------. 1 root root 1286144 Sep 17 09:01 sssd.ldb 0 [root@rhel7 ~ ]# // Now I change the IPA server 0 [root@rhel7 ~ ]# echo 'nameserver 10.34.24.252' >/etc/resolv.conf 0 [root@rhel7 ~ ]# 0 [root@rhel7 ~ ]# 0 [root@rhel7 ~ ]# realm -v join ipa.baseos.qe * Resolving: _ldap._tcp.ipa.baseos.qe * Performing LDAP DSE lookup on: 10.34.24.252 * Successfully discovered: ipa.baseos.qe Password for admin: * Required files: /usr/sbin/ipa-client-install, /usr/sbin/oddjobd, /usr/libexec/oddjob/mkhomedir, /usr/sbin/sssd * LANG=C /usr/sbin/ipa-client-install --domain ipa.baseos.qe --realm IPA.BASEOS.QE --mkhomedir --enable-dns-updates --unattended --force-join --principal admin -W --force-ntpd Discovery was successful! Hostname: rhel7.pkis.net Realm: IPA.BASEOS.QE DNS Domain: ipa.baseos.qe IPA Server: server.ipa.baseos.qe BaseDN: dc=ipa,dc=baseos,dc=qe Synchronizing time with KDC... Successfully retrieved CA cert Subject: CN=Certificate Authority,O=IPA.BASEOS.QE Issuer: CN=Certificate Authority,O=IPA.BASEOS.QE Valid From: Tue Apr 30 14:33:21 2013 UTC Valid Until: Sat Apr 30 14:33:21 2033 UTC Enrolled in IPA realm IPA.BASEOS.QE Created /etc/ipa/default.conf Configured /etc/sssd/sssd.conf Configured /etc/krb5.conf for IPA realm IPA.BASEOS.QE Hostname (rhel7.pkis.net) not found in DNS Failed to update DNS records. Adding SSH public key from /etc/ssh/ssh_host_dsa_key.pub Adding SSH public key from /etc/ssh/ssh_host_rsa_key.pub host_mod: Unknown option: no_members Failed to upload host SSH public keys. SSSD enabled Configured /etc/openldap/ldap.conf NTP enabled Configured /etc/ssh/ssh_config Configured /etc/ssh/sshd_config Client configuration complete. * /usr/bin/systemctl enable sssd.service * /usr/bin/systemctl restart sssd.service * /usr/bin/sh -c /usr/sbin/authconfig --update --enablesssd --enablesssdauth --enablemkhomedir --nostart && /usr/bin/systemctl enable oddjobd.service * Successfully enrolled machine in realm 0 [root@rhel7 ~ ]# 0 [root@rhel7 ~ ]# getent passwd amy@ipa.baseos.qe amy@ipa.baseos.qe:*:903600006:903600006:Amy Amy:/home/amy:/bin/sh 0 [root@rhel7 ~ ]# ssh amy@ipa.baseos.qe@localhost amy@ipa.baseos.qe@localhost's password: Permission denied, please try again. amy@ipa.baseos.qe@localhost's password: 130 [root@rhel7 ~ ]# 130 [root@rhel7 ~ ]# ls -l /tmp/krb5cc* -rw-------. 1 1365200005 1365200005 530 Sep 17 09:03 /tmp/krb5cc_1365200005_zJcbJ6 0 [root@rhel7 ~ ]# // Notice the UID/GID; they are the original ones and not 903600006:903600006 0 [root@rhel7 ~ ]# mv /tmp/krb5cc_1365200005_zJcbJ6 /tmp/krb5cc_1365200005_zJcbJ6_BACKUP 0 [root@rhel7 ~ ]# ls -l /tmp/krb5cc* -rw-------. 1 1365200005 1365200005 530 Sep 17 09:03 /tmp/krb5cc_1365200005_zJcbJ6_BACKUP 0 [root@rhel7 ~ ]# 0 [root@rhel7 ~ ]# ssh amy@ipa.baseos.qe@localhost amy@ipa.baseos.qe@localhost's password: Last failed login: Tue Sep 17 09:07:19 CEST 2013 from localhost on ssh:notty There was 1 failed login attempt since the last successful login. -sh-4.2$ exit logout Connection to localhost closed. 0 [root@rhel7 ~ ]# ls -l /tmp/krb5cc* -rw-------. 1 1365200005 1365200005 530 Sep 17 09:03 /tmp/krb5cc_1365200005_zJcbJ6_BACKUP -rw-------. 1 amy@ipa.baseos.qe amy@ipa.baseos.qe 530 Sep 17 09:08 /tmp/krb5cc_903600006_9EiQ8f 0 [root@rhel7 ~ ]# // Let's connect again to the first IPA server 0 [root@rhel7 ~ ]# echo 'nameserver 10.34.37.24' >/etc/resolv.conf 0 [root@rhel7 ~ ]# realm -v join ipa.baseos.qe * Resolving: _ldap._tcp.ipa.baseos.qe * Performing LDAP DSE lookup on: 10.34.37.24 * Successfully discovered: ipa.baseos.qe Password for admin: * Required files: /usr/sbin/ipa-client-install, /usr/sbin/oddjobd, /usr/libexec/oddjob/mkhomedir, /usr/sbin/sssd ... SNIP ... * Successfully enrolled machine in realm 0 [root@rhel7 ~ ]# 0 [root@rhel7 ~ ]# getent passwd amy@ipa.baseos.qe amy@ipa.baseos.qe:*:1365200005:1365200005:Amy Amy:/home/amy:/bin/sh 0 [root@rhel7 ~ ]# ssh amy@ipa.baseos.qe@localhost amy@ipa.baseos.qe@localhost's password: Permission denied, please try again. amy@ipa.baseos.qe@localhost's password: 130 [root@rhel7 ~ ]# 0 [root@rhel7 ~ ]# sss_cache -E 0 [root@rhel7 ~ ]# ssh amy@ipa.baseos.qe@localhost amy@ipa.baseos.qe@localhost's password: Permission denied, please try again. amy@ipa.baseos.qe@localhost's password: Permission denied, please try again. amy@ipa.baseos.qe@localhost's password: 130 [root@rhel7 ~ ]# 130 [root@rhel7 ~ ]# systemctl restart sssd 0 [root@rhel7 ~ ]# ssh amy@ipa.baseos.qe@localhost amy@ipa.baseos.qe@localhost's password: Permission denied, please try again. amy@ipa.baseos.qe@localhost's password: 130 [root@rhel7 ~ ]# rm -f /var/lib/sss/db/* 0 [root@rhel7 ~ ]# systemctl restart sssd 0 [root@rhel7 ~ ]# ssh amy@ipa.baseos.qe@localhost amy@ipa.baseos.qe@localhost's password: Last failed login: Tue Sep 17 09:14:32 CEST 2013 from localhost on ssh:notty There were 5 failed login attempts since the last successful login. Last login: Tue Sep 17 09:03:13 2013 from localhost -sh-4.2$ exit logout Connection to localhost closed. 0 [root@rhel7 ~ ]# ls -l /tmp/krb5cc* -rw-------. 1 amy@ipa.baseos.qe amy@ipa.baseos.qe 530 Sep 17 09:14 /tmp/krb5cc_1365200005_M1a3OI -rw-------. 1 amy@ipa.baseos.qe amy@ipa.baseos.qe 530 Sep 17 09:03 /tmp/krb5cc_1365200005_zJcbJ6_BACKUP -rw-------. 1 903600006 903600006 530 Sep 17 09:08 /tmp/krb5cc_903600006_9EiQ8f 0 [root@rhel7 ~ ]#
This is arguably an edge-case but on leave, I think it would be OK to also rm the database. AFAIK the domain is removed from sssd.conf as well and SSSD is restarted. That way, even root who could have installed ldb-tools won't have any access to cached domain data in the orphaned ldb files.
I think we should bump this to the next release. Do you agree Patrik?
(In reply to Stef Walter from comment #4) > I think we should bump this to the next release. Do you agree Patrik? The described use case is quite a corner case and so far noting really was found that might cause problems. So yes, I agree.
So why doesn't sss_cache remove the ccache database? There are complaints against realmd about this: https://bugs.freedesktop.org/show_bug.cgi?id=90810
(In reply to Stef Walter from comment #6) > So why doesn't sss_cache remove the ccache database? There are complaints > against realmd about this: > > https://bugs.freedesktop.org/show_bug.cgi?id=90810 There's an upstream bug for that - https://fedorahosted.org/sssd/ticket/1691
Jakub, is that really the same thing? It seems like people want the entire database file removed.
(In reply to Stef Walter from comment #8) > Jakub, is that really the same thing? It seems like people want the entire > database file removed. You're right, it's not, thanks for catching that. I could swear we had an upstream ticket to rm the whole cache, but I can't find it now...so I filed a new one - https://fedorahosted.org/sssd/ticket/2671
This needs to be solved by sssd properly. realmd shouldn't be screwing around with sssd internal database cache manually. We run sss_cache and that should be enough. If no domains exist when running sss_cache, then sss_cache should just remove the database.
Upstream ticket: https://fedorahosted.org/sssd/ticket/2671
Implemented upstream as part of: e157b9f6cb370e1b94bcac2044d26ad66d640fba 9e9ad4cb181c6c0ec70caacfb31319753f889e98 bf83a0faacf16196ab9bd37dcf6190b4209ccaf7 586fa3571753ab4a607d40fc31503fc0e8effd70 2f18b8d67c86a1a277b59894f24ea6e09b41b7ea d6f1b16baf8106d709e3fac585a12789dcb6bd29 725c291ccfa46b08d2713133c227ac8d7203eb2f 2f75ad013f8410397e4efbf0adadc2e69621f12a edaadf8de0c86a2cfff2d29215775d42919476f3 47ce713ef8c7b32f2ce19cc3ace8e88f123fafac 7bf750f6b3b47dcc8a192cc7bcbdecfb94e6cefb d2d8f342cd5e90bb9fd947c448492225f959aa86 aea1d5c0ca9bb1470759b024c8b97b6c1f577193 e98ccef2609811186711b79d8ef5d0a4450ab6e0 81cde110402e088508053aea79670b38d450cb83 b03ccb2764a4ccdadb77599cb624b6a17b633438 3bc651a611a3e5be508875f3ae58bfb5ece2525c a6cd927f298ff5c9a603db5acb6c1b0ebea178c0 b963ed8079a4a284611d50d1b79695116c40295d cf3ba77997dfbd076a1f30fdbb33c7973766ac03 36e262020c80479baa09b2c4c8dd045c7a0f32a1 12d99da163b1efef7e982f04e03049e012857bae 2a45f13e3139063d3a5842119e7377c8c98aea1d 7f0b01bf0a8f5c5b3ef145e81511b6db2cb4f98f b420aae3becdbf501deb2637e2a06636bd6ce1fe Note that this was not implemented as part of sss_cache, but sssctl. sssctl talks to SSSD over D-Bus. Long-term, we would prefer to fold all the existing command-line tools to be driven through the D-Bus interface rather than touching the database directly.
I replaced the Doc Text according to #c19. Jakub, can you please review the Doc Text to make sure it's correct like this?
(In reply to Marc Muehlfeld from comment #20) > I replaced the Doc Text according to #c19. > > Jakub, can you please review the Doc Text to make sure it's correct like > this? sss_cache doesn't have the ability to restart sssd, only sssctl does.
(In reply to Jakub Hrozek from comment #21) > (In reply to Marc Muehlfeld from comment #20) > > I replaced the Doc Text according to #c19. > > > > Jakub, can you please review the Doc Text to make sure it's correct like > > this? > > sss_cache doesn't have the ability to restart sssd, only sssctl does. Once again I got confused by the title of this bugzilla. I really need to change it.. The doc text is good, thank you Marc. ACK.
Verified, the sssctl tool now does this. [root@qe-blade-05 db]# sssctl cache-remove SSSD must not be running. Stop SSSD now? (yes/no) [yes] yes Creating backup of local data... SSSD backup of local data already exist, override? (yes/no) [no] yes Removing cache files... SSSD needs to be running. Start SSSD now? (yes/no) [yes]
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHEA-2016-2476.html