Hide Forgot
Created attachment 710661 [details] this is the log of what I see using "ssh -v" Description of problem: Last week I updated to the most current fedora 18 patch level (64 bit). After that SSH stopped using the known host authentication. I am forced to enter my password each time. This is a pain and a major problem as I also use mercurial and the host source machine will no longer auto login (after I enter my password ONCE in the popup that no longer comes up) so updates become painful. Version-Release number of selected component (if applicable): OpenSSH_6.1p1, OpenSSL 1.0.1e-fips 11 Feb 2013 How reproducible: totally every time - since the last "yum update" Steps to Reproduce: 1. ssh to a machine you have setup with automatic login (after 1st time) (copied the local public key to ~/.ssh/id_rsa.pub if I recall) 2. popup does not popup and you must enter a password to the host each and every time (popup USED to be only the 1st time, then directly in) Actual results: ssh works but not nicely as it did before the update Error shown: Agent admitted failure to sign using the key. Expected results: popup requesting password the first time, immediate login on subsequent logins to this target machine or any other that uses the same protocol Additional info: I made the mistake of appending this bug information to another older bug with exactly the same tag line - see Error above I have attached the ssh -v results (edited to remove target machine name and IP) One quick thought: The target machines I am logging into are running older releases of Fedora (8, 12) - we are planning to upgrade all soon. Perhaps this new update made them incompatible.
Having another problem, I was looking at the system log /var/log/messages This looks like another clue - Mar 18 11:28:00 rainbow gnome-keyring-daemon[1491]: Gkm: unsupported key algorithm in certificate: 1.2.840.10045.2.1 Clueless as to what to do - hope this helps.
I'd like to reproduce this. Could you please attach relevant configuration parts and relevant parts of the server's /var/log/secure? Does it work with a client from Fedora 17?
Created attachment 713417 [details] this is a pared for "su" version of /var/log/secure as requested I apologize for not knowing what configuration files you want me to send. Please send file names, I will edit out anything you don't need and send. Including /var/log/secure (clipped of su things) - I don't see anything that will help you - hope there is something there. I did not pause at Fedora 17. I went from 12 -> 16 -> 18. The problem occurred after a "yum update" in 18; before that all worked just fine in 18 (was fine in 16 always). I believe this happened about 1 week ago (~ 3/9/2013 I think).
(In reply to comment #3) > I apologize for not knowing what configuration files you want me to send. > Please send file names, I will edit out anything you don't need and send. server:/etc/ssh/sshd_config client:/etc/ssh/ssh_config > Including /var/log/secure (clipped of su things) - I don't see anything that > will help you - hope there is something there. Unfortunately, it doesn't help since there's only INFO LogLovel > > I did not pause at Fedora 17. I went from 12 -> 16 -> 18. The problem > occurred after a "yum update" in 18; before that all worked just fine in 18 > (was fine in 16 always). I believe this happened about 1 week ago (~ > 3/9/2013 I think). Could you inspect /var/log/yum.log* and find out which versions of openssh had been installed before you updated to the current version?
>server:/etc/ssh/sshd_config $ egrep -v '^#|^$' /etc/ssh/ssh_config Host * GSSAPIAuthentication yes ForwardX11Trusted yes SendEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES SendEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT SendEnv LC_IDENTIFICATION LC_ALL >client:/etc/ssh/ssh_config # egrep -v '^#|^$' /etc/ssh/ssh_config Host * GSSAPIAuthentication yes ForwardX11Trusted yes SendEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES SendEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT SendEnv LC_IDENTIFICATION LC_ALL LANGUAGE SendEnv XMODIFIERS BOTH machines are VANILLA out of the installation for the OS box (or yum update?). I have a id_rsa.pub file on the client which is copied to the server id_rsa so that (in the old days), I don't have to enter my password EVERYTIME (just first and this is the popup which no longer appears on my desktop - instead each time I am asked). >> Including /var/log/secure (clipped of su things) >Unfortunately, it doesn't help since there's only INFO LogLovel Tell me how to change the logging level and what more you would like. >Could you inspect /var/log/yum.log* and find out which versions of openssh had been installed before you updated to the current version? # grep ssh /var/log/yum.log Feb 05 11:13:53 Updated: libssh2-1.4.3-1.fc18.x86_64 Feb 16 08:05:14 Updated: openssh-6.1p1-5.fc18.x86_64 Feb 16 08:07:02 Updated: openssh-server-6.1p1-5.fc18.x86_64 Feb 16 08:07:02 Updated: openssh-clients-6.1p1-5.fc18.x86_64 Mar 05 16:28:13 Updated: openssh-6.1p1-6.fc18.x86_64 Mar 05 16:28:16 Updated: openssh-clients-6.1p1-6.fc18.x86_64 Mar 05 16:29:36 Updated: openssh-server-6.1p1-6.fc18.x86_64 I think the 3/5 update is the problem.
(In reply to comment #5) >server:/etc/ssh/sshd_config > $ egrep -v '^#|^$' /etc/ssh/ssh_config please run egrep -v '^#|^$' /etc/ssh/sshd_config ssh_config is for ssh client and sshd_config is for sshd server
my eyes missed it - sorry - Protocol 2 sticks out like a sore thumb, but I am not familiar with options Client: # egrep -v '^#|^$' /etc/ssh/sshd_config SyslogFacility AUTHPRIV AuthorizedKeysFile .ssh/authorized_keys PasswordAuthentication yes ChallengeResponseAuthentication no GSSAPIAuthentication yes GSSAPICleanupCredentials yes UsePAM yes X11Forwarding yes UsePrivilegeSeparation sandbox # Default for new installations. AcceptEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES AcceptEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT AcceptEnv LC_IDENTIFICATION LC_ALL LANGUAGE AcceptEnv XMODIFIERS Subsystem sftp /usr/libexec/openssh/sftp-server Server: # egrep -v '^#|^$' /etc/ssh/sshd_config Protocol 2 PermitRootLogin no SyslogFacility AUTHPRIV PermitRootLogin yes PasswordAuthentication yes ChallengeResponseAuthentication no GSSAPIAuthentication yes GSSAPICleanupCredentials yes UsePAM yes AcceptEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES AcceptEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT AcceptEnv LC_IDENTIFICATION LC_ALL X11Forwarding yes Subsystem sftp /usr/libexec/openssh/sftp-server if you need any more (like more than INFO level set, let me know what and how)
Sorry, I somehow posted this twice and some of my answers targeted to one have gone to the other (and vice versa) - I also did not get several messages posted due to my confusion. *** This bug has been marked as a duplicate of bug 844488 ***
Sorry I get the silly award. The other bug I marked this as a duplicate of was not mine. It was filed by someone else over a year ago. The symptoms are the same but the root cause is probably quite different. The other bug was closed and this one should be open.
new yum update run - current kernel vmlinuz-3.8.4-202.fc18.x86_64, libevent_openssl-2.0.so.5 libssl3.so libssl.so.1.0.1e libevent_openssl-2.0.so.5.1.6 libssl.so.10 Problem persists.
3/31/13 yum update run - reboot and problem persists
4/5/13 yum update - no changes - will wait till some post here
I'm seeing the exact same thing. sshd and ssh worked fine with the same config for over a decade. The recent yum update broke agent-based logins. $ egrep -v '^#|^$' /etc/ssh/sshd_config Protocol 2 SyslogFacility AUTHPRIV LoginGraceTime 10 PermitRootLogin without-password AuthorizedKeysFile .ssh/authorized_keys PasswordAuthentication no ChallengeResponseAuthentication no KerberosAuthentication no GSSAPIAuthentication no GSSAPICleanupCredentials yes UsePAM no X11Forwarding yes UsePrivilegeSeparation sandbox # Default for new installations. ClientAliveInterval 60 ClientAliveCountMax 30 AllowUsers root wolfgang Ciphers aes256-cbc $ egrep -v '^#|^$' /etc/ssh/ssh_config Host *.wsrcc.com localhost ancho arbol bertha capsicum pearl poblano sonic ForwardX11 yes ForwardX11Trusted yes Host * ConnectTimeout 10 IdentityFile ~/.ssh/id_rsa IdentityFile ~/.ssh/id_dsa Protocol 2 ServerAliveInterval 60 ServerAliveCountMax 30 VerifyHostKeyDNS yes
I am glad that someone else is seeing this. Too bad for all of us with this condition. I continue to check for "fixed" after every week when I do a full yum update, but will only post if the problem fixes or developers want more information. :=[
I had a terrible time with this problem as well until I disabled SELinux. Looking in /var/log/audit/audit.log I'm no expert in SELinux but it would appear that SELinux is denying access to the key files!! type=AVC msg=audit(1368025308.551:1191): avc: denied { read } for pid=27473 comm="sshd" name="authorized_keys" dev="sda5" ino=4196458 scontext=system_u:system_r:sshd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:home_root_t:s0 tclass =file type=SYSCALL msg=audit(1368025308.551:1191): arch=c000003e syscall=2 success=no exit=-13 a0=7f7dfffd38b0 a1=800 a2=0 a3=c items=0 ppid=23164 pid=27473 auid=4294967295 uid=0 gid=0 euid=1014 suid=0 fsuid=1014 egid=1015 sgid=0 fsgid=1015 ses =4294967295 tty=(none) comm="sshd" exe="/usr/sbin/sshd" subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 key=(null) type=USER_AUTH msg=audit(1368025308.551:1192): pid=27473 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=pubkey acct="euan" exe="/usr/sbin/sshd" hostname=? addr=::1 terminal=ssh res=failed'
Thanks for your input Euan. I think you are advocating turning off SELinux protection. I can do that and my system is pretty well protected so maybe that is the thing to do. If you are a developer and suggest this, I will do it. I can try it anyway, but it gives me NO WAY to find out if it is fixed. Right now, I try after every yum update. This problem started to appear in Linux 18 after some yum update. It has been working for a long time before that. I also note that a directory of files (forgot name - sorry) used to be created in /tmp (---key--- I think) and they are no longer being created. There are NO errors in the troubleshooter than seem related, but YES the audit.log has entries for sshd. In my case they are ALL of type USER_LOGIN e.g. type=USER_LOGIN msg=audit(1367003928.970:463): pid=16752 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=login acct=28696E76616C6964207573657229 exe="/usr/sbin/sshd" hostname=? addr=192.168.101.153 terminal=ssh res=failed The IP address is interesting as it is within my local home network but not one I know (most are "hard" wired) - this is in the dynamic range for me. IT IS NOT THE ADDRESS OF THE MACHINE I am trying to get to (208.x.x.x)
I don't see any AVC's here. The problem I'm seeing appears to be a failure of the local agent, not the remote sshd. I'll append an ssh -v transcript showing the failure. It is after everything has been negotiated and the agent (keystore) tries to sign something. One other odd thing I notice that may be related to the failure to sign is that the key can't be deleted. If I add extra keys by hand with ssh-add, these can be removed, but the pre-existing one from the login can't be. I wonder if there are two separate keystores involved. One functional, the other not so much. [wolfgang@arbol ~]$ ssh-add -D All identities removed. [wolfgang@arbol ~]$ ssh-add -l 2048 90:be:33:a9:c0:9e:2f:ec:b4:c4:78:06:50:94:6b:88 wolfgang+20130119@arbol.wsrcc.com (RSA) [wolfgang@arbol ~]$ [wolfgang@arbol ~]$ ssh -vvvvv poblano OpenSSH_6.1p1, OpenSSL 1.0.1e-fips 11 Feb 2013 debug1: Reading configuration data /home/wolfgang/.ssh/config debug1: Reading configuration data /etc/ssh/ssh_config debug1: /etc/ssh/ssh_config line 35: Applying options for poblano debug1: /etc/ssh/ssh_config line 44: Applying options for * debug2: ssh_connect: needpriv 0 debug1: Connecting to poblano [fdb1:c796:3f3:2c2b:2e0:81ff:fe56:8d66] port 22. debug2: fd 3 setting O_NONBLOCK debug1: connect to address fdb1:c796:3f3:2c2b:2e0:81ff:fe56:8d66 port 22: No route to host debug1: Connecting to poblano [192.83.197.4] port 22. debug2: fd 3 setting O_NONBLOCK debug1: fd 3 clearing O_NONBLOCK debug1: Connection established. debug3: timeout: 10000 ms remain after connect debug3: Incorrect RSA1 identifier debug3: Could not load "/home/wolfgang/.ssh/id_rsa" as a RSA1 public key debug1: identity file /home/wolfgang/.ssh/id_rsa type 1 debug1: identity file /home/wolfgang/.ssh/id_rsa-cert type -1 debug1: identity file /home/wolfgang/.ssh/id_dsa type -1 debug1: identity file /home/wolfgang/.ssh/id_dsa-cert type -1 debug1: Remote protocol version 2.0, remote software version OpenSSH_6.1 debug1: match: OpenSSH_6.1 pat OpenSSH* debug1: Enabling compatibility mode for protocol 2.0 debug1: Local version string SSH-2.0-OpenSSH_6.1 debug2: fd 3 setting O_NONBLOCK debug3: load_hostkeys: loading entries for host "poblano" from file "/home/wolfgang/.ssh/known_hosts" debug3: load_hostkeys: loaded 0 keys debug3: load_hostkeys: loading entries for host "poblano" from file "/etc/ssh/ssh_known_hosts" debug3: load_hostkeys: found key type RSA in file /etc/ssh/ssh_known_hosts:7 debug3: load_hostkeys: loaded 1 keys debug3: order_hostkeyalgs: prefer hostkeyalgs: ssh-rsa-cert-v01@openssh.com,ssh-rsa-cert-v00@openssh.com,ssh-rsa debug1: SSH2_MSG_KEXINIT sent debug1: SSH2_MSG_KEXINIT received debug2: kex_parse_kexinit: diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1 debug2: kex_parse_kexinit: ssh-rsa-cert-v01@openssh.com,ssh-rsa-cert-v00@openssh.com,ssh-rsa,ssh-dss-cert-v01@openssh.com,ssh-dss-cert-v00@openssh.com,ssh-dss debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96 debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96 debug2: kex_parse_kexinit: none,zlib@openssh.com,zlib debug2: kex_parse_kexinit: none,zlib@openssh.com,zlib debug2: kex_parse_kexinit: debug2: kex_parse_kexinit: debug2: kex_parse_kexinit: first_kex_follows 0 debug2: kex_parse_kexinit: reserved 0 debug2: kex_parse_kexinit: diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1 debug2: kex_parse_kexinit: ssh-rsa,ssh-dss debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96 debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96 debug2: kex_parse_kexinit: none,zlib@openssh.com debug2: kex_parse_kexinit: none,zlib@openssh.com debug2: kex_parse_kexinit: debug2: kex_parse_kexinit: debug2: kex_parse_kexinit: first_kex_follows 0 debug2: kex_parse_kexinit: reserved 0 debug2: mac_setup: found hmac-md5 debug1: kex: server->client aes128-ctr hmac-md5 none debug2: mac_setup: found hmac-md5 debug1: kex: client->server aes128-ctr hmac-md5 none debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP debug2: dh_gen_key: priv key bits set: 132/256 debug2: bits set: 506/1024 debug1: SSH2_MSG_KEX_DH_GEX_INIT sent debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY debug1: Server host key: RSA 2d:07:17:cc:7f:72:61:ac:36:58:1d:03:2d:82:96:c1 debug3: verify_host_key_dns DNS lookup error: name does not exist debug3: load_hostkeys: loading entries for host "poblano" from file "/home/wolfgang/.ssh/known_hosts" debug3: load_hostkeys: loaded 0 keys debug3: load_hostkeys: loading entries for host "poblano" from file "/etc/ssh/ssh_known_hosts" debug3: load_hostkeys: found key type RSA in file /etc/ssh/ssh_known_hosts:7 debug3: load_hostkeys: loaded 1 keys debug3: load_hostkeys: loading entries for host "192.83.197.4" from file "/home/wolfgang/.ssh/known_hosts" debug3: load_hostkeys: loaded 0 keys debug3: load_hostkeys: loading entries for host "192.83.197.4" from file "/etc/ssh/ssh_known_hosts" debug3: load_hostkeys: found key type RSA in file /etc/ssh/ssh_known_hosts:15 debug3: load_hostkeys: loaded 1 keys debug1: Host 'poblano' is known and matches the RSA host key. debug1: Found key in /etc/ssh/ssh_known_hosts:7 debug2: bits set: 504/1024 debug1: ssh_rsa_verify: signature correct debug2: kex_derive_keys debug2: set_newkeys: mode 1 debug1: SSH2_MSG_NEWKEYS sent debug1: expecting SSH2_MSG_NEWKEYS debug2: set_newkeys: mode 0 debug1: SSH2_MSG_NEWKEYS received debug1: Roaming not allowed by server debug1: SSH2_MSG_SERVICE_REQUEST sent debug2: service_accept: ssh-userauth debug1: SSH2_MSG_SERVICE_ACCEPT received debug2: key: /home/wolfgang/.ssh/id_rsa (0x7f1a456e74c0) debug2: key: /home/wolfgang/.ssh/id_dsa ((nil)) debug3: input_userauth_banner ************************************************************************ RSA and DSA authentication only. Password logins are NOT accepted. ************************************************************************ debug1: Authentications that can continue: publickey debug3: start over, passed a different list publickey debug3: preferred publickey,keyboard-interactive,password debug3: authmethod_lookup publickey debug3: remaining preferred: keyboard-interactive,password debug3: authmethod_is_enabled publickey debug1: Next authentication method: publickey debug1: Offering RSA public key: /home/wolfgang/.ssh/id_rsa debug3: send_pubkey_test debug2: we sent a publickey packet, wait for reply debug1: Server accepts key: pkalg ssh-rsa blen 279 debug2: input_userauth_pk_ok: fp 90:be:33:a9:c0:9e:2f:ec:b4:c4:78:06:50:94:6b:88 debug3: sign_and_send_pubkey: RSA 90:be:33:a9:c0:9e:2f:ec:b4:c4:78:06:50:94:6b:88 Agent admitted failure to sign using the key. debug1: Trying private key: /home/wolfgang/.ssh/id_dsa debug3: no such identity: /home/wolfgang/.ssh/id_dsa debug2: we did not send a packet, disable method debug1: No more authentication methods to try. Permission denied (publickey).
More on the odd thing that I noted in /tmp When this was working (fedora 8, 12, 16 and 18 BEFORE some patch from yum) There was a directory in /tmp called "keyring-XXXXXX" where XXXXXX is some combination of letters and numbers. This was created the first time I did an ssh (some other things created it too) and I was asked for my password in a popup in the window system. Subsequent ssh (or other things) did not ask - just connected. NO SUCH DIRECTORY IS NOW MADE. I suspect this is a symptom not a cause, but ...
This appears to be a bug in gnome-keyring or how the initial login password is used to unlock the ssh rsa/dsa credentials. Disabling gnome-keyring's ssh subsection via gnome-session-properties fixes the problem. In fact, There appears to be a default ssh-agent already running too, which will happily allow the rsa/dsa keys to be loaded via ssh-add. Why are there two keyring daemons running in the first place?
OK Wofgang, I see two ssh things running but they are different. One is the daemon, one is the agent. Here is the local picture, the remote works fine when I login from another (third) machine. root 1171 1 0 May07 ? 00:00:00 /usr/sbin/sshd -D ray 1656 1506 0 May07 ? 00:00:00 /usr/bin/ssh-agent /bin/sh -c exec -l /bin/bash -c "/usr/bin/gnome-session --session=cinnamon" If you are recommending that I disable gnome-keyring and all will work, I will do that. However, then I will NOT know if this is ever fixed. On the other hand, I can type my password many times and wait. Please advise.
(In reply to comment #20) > If you are recommending that I disable gnome-keyring and all will work, I > will do that. However, then I will NOT know if this is ever fixed. gnome-keyring shows up several times in the gnome-session-properties tool. You only have to disable the ssh functionality, not the whole thing. > On the other hand, I can type my password many times and wait. You don't need to type your password repeatedly. Run ssh-add to add your password to the ssh-agent. You only need to do this once per login on the local machine.
Fixed in release installed 5/29/2013 - thanks I would mark as fixed and closed, but cannot find the options on this bugzilla page
OK, missed that box - this one is fixed and done