Hide Forgot
I've noticed that ssh logins to a server using krb5 in sssd take much longer than they should (5-40 seconds). I started sssd with debugging enabled (sssd --debug-level=9 --debug-timestamps=1 -c /etc/sssd/sssd.conf) and noticed the following errors on the console output: (Thu Mar 1 12:31:05 2012) [sssd] [check_file] (1): lstat for [/var/run/nscd/socket] failed: [2][No such file or directory]. (Thu Mar 1 12:31:05 2012) [sssd] [ldb] (6): server_sort:Unable to register control with rootdse! (Thu Mar 1 12:31:05 2012) [sssd] [confdb_get_domain_internal] (1): No enumeration for [default]! (Thu Mar 1 12:31:05 2012) [sssd] [server_setup] (3): Becoming a daemon. Running ssh -v on the client had the following output: $ ssh -v server OpenSSH_5.8p1 Debian-7ubuntu1, OpenSSL 1.0.0e 6 Sep 2011 debug1: Reading configuration data /home/user/.ssh/config debug1: Applying options for * debug1: Reading configuration data /etc/ssh/ssh_config debug1: Applying options for * debug1: Connecting to server [1.2.3.4] port 22. debug1: Connection established. debug1: identity file /home/user/.ssh/id_rsa type 1 debug1: Checking blacklist file /usr/share/ssh/blacklist.RSA-2048 debug1: Checking blacklist file /etc/ssh/blacklist.RSA-2048 debug1: identity file /home/user/.ssh/id_rsa-cert type -1 debug1: identity file /home/user/.ssh/id_dsa type -1 debug1: identity file /home/user/.ssh/id_dsa-cert type -1 debug1: identity file /home/user/.ssh/id_ecdsa type -1 debug1: identity file /home/user/.ssh/id_ecdsa-cert type -1 debug1: Remote protocol version 2.0, remote software version OpenSSH_5.3 debug1: match: OpenSSH_5.3 pat OpenSSH* debug1: Enabling compatibility mode for protocol 2.0 debug1: Local version string SSH-2.0-OpenSSH_5.8p1 Debian-7ubuntu1 debug1: SSH2_MSG_KEXINIT sent debug1: SSH2_MSG_KEXINIT received debug1: kex: server->client aes128-ctr hmac-md5 none debug1: kex: client->server aes128-ctr hmac-md5 none debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP debug1: SSH2_MSG_KEX_DH_GEX_INIT sent debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY debug1: Server host key: RSA 24:4b:96:a5:07:71:d3:e5:63:e3:f6:b1:e9:aa:ba:2f debug1: Host 'server' is known and matches the RSA host key. debug1: Found key in /home/user/.ssh/known_hosts:503 debug1: ssh_rsa_verify: signature correct debug1: SSH2_MSG_NEWKEYS sent debug1: expecting SSH2_MSG_NEWKEYS debug1: SSH2_MSG_NEWKEYS received debug1: Roaming not allowed by server debug1: SSH2_MSG_SERVICE_REQUEST sent debug1: SSH2_MSG_SERVICE_ACCEPT received debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password LONG PAUSE HERE if NOT using GSSAPI debug1: Next authentication method: gssapi-keyex debug1: No valid Key exchange context debug1: Next authentication method: gssapi-with-mic LONG PAUSE HERE if using GSSAPI debug1: Authentication succeeded (gssapi-with-mic). Authenticated to server ([1.2.3.4]:22). debug1: channel 0: new [client-session] debug1: Requesting no-more-sessions debug1: Entering interactive session. debug1: Requesting X11 forwarding with authentication spoofing. debug1: Sending environment. debug1: Sending env LC_MESSAGES = en_US.UTF-8 debug1: Sending env LC_COLLATE = en_US.UTF-8 debug1: Sending env LANG = en_US.UTF-8 debug1: Sending env LC_CTYPE = en_US.UTF-8 Last login: Thu Mar 1 11:34:02 2012 from 1.2.3.5 Since I didn't have nscd installed, I tried just creating the /var/run/nscd directory & restarted sssd but there was no improvement. I then installed nscd and started it - the /var/run/nscd/socket was created. Afterwards I restarted sssd again. Now logins take less than a second. Before starting nscd: $ time ssh server : real 0m21.829s user 0m0.016s sys 0m0.016s After starting nscd: $ time ssh server : real 0m0.851s user 0m0.016s sys 0m0.012s I would be happy with this as a solution, but the redhat sssd documentation clearly recommends that it should NOT be run with nscd side-by-side. "SSSD is not designed to be used with the NSCD daemon. Even though SSSD does not directly conflict with NSCD, using both services can result in unexpected behavior, especially with how long entries are cached." http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/6/html/Deployment_Guide/usingnscd-sssd.html Versions: Server- RHEL 6.1 glibc-2.12-1.25.el6_1.3.x86_64 sssd-1.5.1-34.el6_1.3.x86_64 nscd-2.12-1.25.el6_1.3.x86_64 krb5-workstation-1.9-9.el6_1.2.x86_64 krb5-libs-1.9-9.el6_1.2.x86_64 pam_krb5-2.3.11-6.el6.x86_64 Client- Ubuntu 11.04 x64 OpenSSH_5.8p1 Debian-7ubuntu1, OpenSSL 1.0.0e 6 Sep 2011 How reproducible: Always. With or without client gssapi authentication Steps to Reproduce: 1. Configure sssd with auth_provider = krb5. We are using Windows 2008R2 for LDAP/kerberos. 2. Attempt to log in or run an ssh command. The time to authenticate will very wildly - 5 to over 40 seconds 3. Install nscd and start it. 4. Restart sssd 5. Attempt to log in - logins are now much quicker - generally under a second. Actual results: $ time ssh server : real 0m21.829s user 0m0.016s sys 0m0.016s Expected results: $ time ssh server : real 0m0.851s user 0m0.016s sys 0m0.012s Additional info: We are using LDAPS for directory lookups. Client is configured to use gssapi but problem happens with gssapi disabled too. sssd.conf: [sssd] config_file_version = 2 reconnection_retries = 3 services = nss, pam domains = default [nss] filter_groups = root filter_users = root reconnection_retries = 3 [pam] reconnection_retries = 3 [domain/default] auth_provider = krb5 cache_credentials = True ldap_group_object_class = group ldap_id_use_start_tls = False ldap_user_uid_number = UidNumber chpass_provider = krb5 enumerate = False ldap_schema = rfc2307bis ldap_force_upper_case_realm = True ldap_user_principal = userPrincipalName ldap_user_object_class = person ldap_user_gid_number = GidNumber ldap_group_name = msSFU30Name ldap_user_name = uid ldap_group_gid_number = GidNumber ldap_default_authtok_type = password ldap_search_base = dc=mydomain,dc=com id_provider = ldap ldap_default_bind_dn = CN=linuxuser,OU=accounts,DC=mydomain,DC=com ldap_user_shell = LoginShell krb5_server = server1.mydomain.com,server2.mydomain.com,server3.mydomain.com,server4.mydomain.com min_id = 1000 ldap_uri = ldaps://server1.mydomain.com,ldaps://server2.mydomain.com,ldaps://server3.mydomain.com,ldaps://server4.mydomain.com krb5_realm = MYDOMAIN.COM krb5_kdcip = server1.mydomain.com,server2.mydomain.com,server3.mydomain.com,server4.mydomain.com ldap_user_home_directory = unixHomeDirectory ldap_default_authtok = ******** ldap_tls_cacertdir = /etc/openldap/cacerts dns_discovery_domain = mydomain.com
Sorry, but you need to include more information. Primarily the SSSD version you are running and the SSSD logs. > (Thu Mar 1 12:31:05 2012) [sssd] [check_file] (1): lstat for > [/var/run/nscd/socket] failed: [2][No such file or directory]. > (Thu Mar 1 12:31:05 2012) [sssd] [ldb] (6): server_sort:Unable to register > control with rootdse! > (Thu Mar 1 12:31:05 2012) [sssd] [confdb_get_domain_internal] (1): No > enumeration for [default]! > (Thu Mar 1 12:31:05 2012) [sssd] [server_setup] (3): Becoming a daemon. These are just warnings. Also, the format of the debug logs shows that you are running sssd older than 1.7. Because nscd gives you a much better login time, I assume the problem is slow initgroups operation. There have been many performance improvements in both SSSD 1.7 and 1.8 related to faster initgroups. RHEL-6.3 is going to rebase the SSSD to 1.8 which might be a good place to try the performance improvements on your RHEL server. > I would be happy with this as a solution, but the redhat sssd documentation > clearly recommends that it should NOT be run with nscd side-by-side. > > "SSSD is not designed to be used with the NSCD daemon. Even though SSSD does > not directly conflict with NSCD, using both services can result in unexpected > behavior, especially with how long entries are cached." > http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/6/html/Deployment_Guide/usingnscd-sssd.html nscd and SSSD both include their own caching mechanism and it's better not to mix the two. I'm going to close this bug here with the suggestion that you try out a newer version to see if your performance problem persists even with the fixes we did recently. *** This bug has been marked as a duplicate of bug 743133 ***