Bug 2164016 - Unhelpful "Bad server host key: Invalid key length" error for old entry in known_hosts
Summary: Unhelpful "Bad server host key: Invalid key length" error for old entry in kn...
Keywords:
Status: NEW
Alias: None
Product: Fedora
Classification: Fedora
Component: openssh
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Dmitry Belyavskiy
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2023-01-24 14:08 UTC by Jonathan Wakely
Modified: 2024-02-05 13:23 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Type: Bug
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker FC-719 0 None None None 2023-01-24 14:08:53 UTC

Description Jonathan Wakely 2023-01-24 14:08:17 UTC
Description of problem:

If you have an old ssh-rsa host key in known_hosts and upgrade to F37, you will be unable to connect to the host. The error is very unhelpful:

  "Bad server host key: Invalid key length"


Version-Release number of selected component (if applicable):

openssh-8.8p1-7.fc37.x86_64

How reproducible:

Always


Steps to Reproduce:
1. Use DEFAULT crypto policiies.
2. Have an old host key in known_hosts that is disallowed by the policy.
3. Try to ssh to the host.

Actual results:

Bad server host key: Invalid key length

It seems that the ssh client first finds a matching host key in known_hosts, and *then* checks whether that key is allowed by policy.

debug1: Remote protocol version 2.0, remote software version OpenSSH_8.0
debug1: compat_banner: match: OpenSSH_8.0 pat OpenSSH* compat 0x04000000
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to gcc.gnu.org:22 as 'jwakely'
debug3: record_hostkey: found key type RSA in file /home/jwakely/.ssh/known_hosts:58
debug3: load_hostkeys_file: loaded 1 keys from gcc.gnu.org
...
...
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: rsa-sha2-512
debug1: kex: server->client cipher: aes256-gcm MAC: <implicit> compression: zlib
debug1: kex: client->server cipher: aes256-gcm MAC: <implicit> compression: zlib
debug1: kex: curve25519-sha256 need=32 dh_need=32
debug1: kex: curve25519-sha256 need=32 dh_need=32
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug3: receive packet: type 31
debug1: SSH2_MSG_KEX_ECDH_REPLY received
Bad server host key: Invalid key length



Expected results:

Ideally, the client would not even load a host key from known_hosts if it's a type that isn't allowed. That way a new host key would be received, and the user would be prompted to accept a new one of a type that the client can use:

The authenticity of host 'gcc.gnu.org (8.43.85.97)' can't be established.
ED25519 key fingerprint is SHA256:5DrqonN/NcBalJRSSpE86TZRuNiBYukggX/j02fe1qA.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'gcc.gnu.org' (ED25519) to the list of known hosts.

Alternatively, the error message should make it clear that the problem is the host key read from ~/.ssh/known_hosts **not** the one being received from the server. To somebody unfamiliar with the openssh internal implementation details, it's not at all clear that "Bad server host key" means a problem with your local known_hosts file rather than a problem with the server. The reaction is simply "argh, another fedora crypto policy change ... I need to allow the server's key somehow".


Additional info:

The fix is simply to remove the offending line from known_hosts and accept a new key of a type allowed by the policy.  But because there's no clue about known_hosts in the error, people are downgrading their systemwide crypto policy to LEGACY to be able to connect to servers that they need to reach. This does not improve security!

If improving the error message is not possible, could we at least add something to https://docs.fedoraproject.org/en-US/quick-docs/dnf-system-upgrade/#sect-resolving-post-upgrade-issues pointing out that old host keys might be unusable and suggesting that the solution is to remove those entries from known_hosts and accept a new host key.

There is currently no info I could find in fedora docs about the right fix for this error.

There is no mention of known_hosts in any of 
https://fedoraproject.org/wiki/Changes/CryptoPolicy
https://fedoraproject.org/wiki/Changes/StrongCryptoSettings
https://fedoraproject.org/wiki/Changes/StrongCryptoSettings2
https://fedoraproject.org/wiki/Changes/StrongCryptoSettings3

https://fedoraproject.org/wiki/Releases/37/ChangeSet#Strong_crypto_settings:_phase_3,_forewarning_1/2 specifically says F37 doesn't change any defaults, but that doesn't seem to be true since upgrading f36 to f37 broke this use case.

Comment 1 Jonathan Wakely 2023-01-24 14:17:37 UTC
Here's somebody noting this error when testing F37 beta:

https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org/thread/7UXGQ3U24BASVT4HNDYHYEAHGQRHZ3NF/#7UXGQ3U24BASVT4HNDYHYEAHGQRHZ3NF

The suggested remedies are "Is it possible to generate a longer key pair?" and "Check if you can connect to the remote side with the LEGACY policy".

Neither of those is the right solution.

The second reply links to https://gitlab.com/redhat-crypto/fedora-crypto-policies/-/commit/e4ed8604ba69650f002229e29b7ca54768cafef5 which certainly seems like a change in crypto policies for F37. So why do the release notes say there wasn't one?

Comment 2 Dmitry Belyavskiy 2023-01-24 15:49:53 UTC
Legacy policy probably already also forbids so short keys.

Regarding the 2nd link, we have forbidden too short RSA keys for a long time, but OpenSSH implemented this restriction recently. Technically it's not a change of the defaults but enlarging the scope. I agree we'd better add this announcement.

Comment 3 Jonathan Wakely 2023-01-24 15:54:44 UTC
(In reply to Dmitry Belyavskiy from comment #2)
> Legacy policy probably already also forbids so short keys.

It works in F36 with DEFAULT policy and doesn't work in F37 ¯\_(ツ)_/¯

> Regarding the 2nd link, we have forbidden too short RSA keys for a long
> time, but OpenSSH implemented this restriction recently. Technically it's
> not a change of the defaults but enlarging the scope.

As a user I just see something that works in F36 and fails in F37.

> I agree we'd better
> add this announcement.

That would be helpful, thanks.

Comment 4 Dmitry Belyavskiy 2023-01-24 15:58:44 UTC
(In reply to Jonathan Wakely from comment #3)
> (In reply to Dmitry Belyavskiy from comment #2)
> > Legacy policy probably already also forbids so short keys.
> 
> It works in F36 with DEFAULT policy and doesn't work in F37 ¯\_(ツ)_/

Exactly. In F36 we had the same key length in DEFAULT CP but no way to enforce it for OpenSSH.
In F37 we got this option.

> > I agree we'd better
> > add this announcement.
> 
> That would be helpful, thanks.

I'm not sure if it is possible to update RN retroactively, I'll take a look.

Comment 5 Mark Wielaard 2023-01-24 17:25:42 UTC
Note that more documentation would be helpful but it is really the error message that is misleading and causes people to unnecessarily switch to the LEGACY policy.
As the description says the ssh client should not load a host key from known_hosts if it's a type that isn't allowed.
Or the error message should make it clear that the problem is the host key read from known_hosts but the server does support a newer/longer alg/key.

Comment 6 Aoife Moloney 2023-11-23 01:07:07 UTC
This message is a reminder that Fedora Linux 37 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora Linux 37 on 2023-12-05.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
'version' of '37'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, change the 'version' 
to a later Fedora Linux version. Note that the version field may be hidden.
Click the "Show advanced fields" button if you do not see it.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora Linux 37 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora Linux, you are encouraged to change the 'version' to a later version
prior to this bug being closed.

Comment 7 Jonathan Wakely 2023-11-23 08:22:12 UTC
I don't think this was fixed, was it?

Retargetting to rawhide.

Comment 8 Jason Merrill 2024-02-05 13:23:37 UTC
Same problem in Fedora 39 still, the diagnostic is extremely misleading.


Note You need to log in before you can comment on or make changes to this bug.