Created attachment 1143276 [details] Client Session Description of problem: When I upgrade openssh to openssh-7.2p2-1.fc23 Version-Release number of selected component (if applicable): openssh-7.2p2-1.fc23 How reproducible: Everytime Steps to Reproduce: 1.With a valid Kerberos ticket ssh to F23 machine running 7.2p2-1 2.Client errors with "Disconnecting: Hash's MIC didn't verify" and doesn't connect Actual results: "Disconnecting: Hash's MIC didn't verify" Expected results: A SSH session to this machine It works again if downgraded to openssh-7.2p2-1.fc23 Additional info: Attached the debug from client and server.
Created attachment 1143277 [details] Server Session
Sorry typo, should have read "it works again if we downgrade to openssh-7.1p1-3.fc23"
Hi, thank you for the report. There was a lot of releases since openssh-7.1p1-3.fc23 (since 2015-09-25). All of the versions since that date do not work? Can you please verify that? There was a lot of changes to go through. I will try to reproduce it in our environment. I am wondering how did it slip through our testing, because we certainly have test for GSSAPI with MIC and it looks like a reproducible one.
You are using non-default "GSSAPIDelegateCredentials" which makes a difference from my default test case. But even though I can't reproduce your behaviour with latest openssh (I am connecting from openssh-7.2p2-1. Can you please provide what other non-default configuration do you have on client and server? I tried various configurations and I am always able to connect without any problem.
I can't see how you get the other intermediate versions? yum list openssh --showduplicates Yum command has been deprecated, redirecting to '/usr/bin/dnf list openssh --showduplicates'. See 'man dnf' and 'man yum2dnf' for more information. To transfer transaction metadata from yum to DNF, run: 'dnf install python-dnf-plugins-extras-migrate && dnf-2 migrate' Last metadata expiration check: 2:18:43 ago on Mon Apr 4 11:17:54 2016. Installed Packages openssh.x86_64 7.2p2-1.fc23 @updates Available Packages openssh.x86_64 7.1p1-3.fc23 fedora openssh.x86_64 7.2p2-1.fc23 @updates openssh.x86_64 7.2p2-1.fc23 updates I thought this would be it, and can't see them in the repo directly from a browser. What did you have in mind for non-standard options? I'm thinking the sshd_config and ssh_config and maybe krb5.conf? Can there be much else that would influence this error?
Sorry I forgot to add a link to koji. It looks like the previous versions are gone from mirrors already and I don't know why there is only that old one (the version packaged when Fedora 23 was released?). You can download all available versions from koji [1], but you need to go through the builds to your architecture and download packages by hand (at least openssh-clients, openssh-server and openssh rpms, depending on your installation) and then (example version somewhere in the middle): dnf update *7.1p2-4.fc23.x86_64.rpm or dnf downgrade *7.1p2-4.fc23.x86_64.rpm respective if you are going up or down in numbers. I would try to bisect the interval of releases. Yes, about the config I meant client ssh_config, server sshd_config (it is partially visible from the log) and kerberos (krb5.conf) probably too. [1] http://koji.fedoraproject.org/koji/packageinfo?packageID=96
Works with openssh-7.1p2-4 breaks with openssh-7.2p1-1
Thanks for narrowing the problem to one update. openssh 7.2 version reworked rekey handling which is probably causing the issue (still I am wondering why it does not cause problems in my setup). Can you verify that it works for you if you drop "GSSAPIDelegateCredentials" from your your client? Basically you can do it with inline switch: ssh -vvv -k fedtest
Ahh maybe why you can't replicate is that it works if you used OpenSSH 7.2 to 7.2 but if one end is an earlier version 7.1 (or the version on RHEL7), it errors. Maybe you have both ends (clients and server) on the new version?
Created attachment 1143859 [details] Proposed patch Yes, you are right. We test only on the single machine with same client and server versions. Now I set up kerberos principals between rhel7 and fedora23 and I see your described behaviour. Digging deep enough showed, that part of MIC hash is also min uint32 min, minimal size in bits of an acceptable group which is increased in openssh-7.2 and therefore the MIC does not verify (client and server are using different minimal values). From RFC4462 The server keeps a list of safe primes and corresponding generators that it can select from. These are chosen as described in Section 3 of [GROUP-EXCHANGE]. The client requests a modulus from the server, indicating the minimum, maximum, and preferred sizes; the server responds with a suitable modulus and generator. The exchange then proceeds as described in Section 2.1 above. [...] o min and max are the minimal and maximal sizes of p in bits that are acceptable to the client This means that there is a bug in server code, sending wrong MIC signature (not based on client range, but based on subset of client and server ranges). If I read the RFC right, there is no way for server to enforce minimal key size and doing: min = MAX(DH_GRP_MIN, min); is plain wrong in this case. Server can only verify, that the nbits value is in the server proposal range, but it can not propose different range for a client. See proposed patch. It fixed the issue for me. Can you verify it with this scratch build: http://koji.fedoraproject.org/koji/taskinfo?taskID=13565607
Your patched packages work on the test systems I have here too. Both F23 to F23 plus RHEL7 to F23 work fine. Thanks
openssh-7.2p2-2.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2016-8629d3fbb0
openssh-7.2p2-2.fc23 has been submitted as an update to Fedora 23. https://bodhi.fedoraproject.org/updates/FEDORA-2016-90c69a9e2a
openssh-7.2p2-2.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-8629d3fbb0
openssh-7.2p2-2.fc23 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-90c69a9e2a
openssh-7.2p2-2.fc23 has been pushed to the Fedora 23 stable repository. If problems still persist, please make note of it in this bug report.
openssh-7.2p2-2.fc24 has been pushed to the Fedora 24 stable repository. If problems still persist, please make note of it in this bug report.