Bug 1981884

Summary: OpenSSH's ssh-keygen can't parse encrypted PKCS#8 private keys
Product: Red Hat Enterprise Linux 9 Reporter: Hubert Kario <hkario>
Component: opensshAssignee: Dmitry Belyavskiy <dbelyavs>
Status: CLOSED CURRENTRELEASE QA Contact: Pavel Yadlouski <pyadlous>
Severity: high Docs Contact:
Priority: medium    
Version: 9.0CC: jjelen, mhavrila, pyadlous
Target Milestone: betaKeywords: TestBlockerForLayeredProduct, Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-12-07 21:42:00 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
An openssl 3.0-compatibility patch none

Description Hubert Kario 2021-07-13 16:16:24 UTC
Description of problem:
When openssl is used to generate a private key that's protected with a passphrase it creates a PKCS#8 file by default. ssh-keygen is unable to process such files.

Version-Release number of selected component (if applicable):
openssh-8.6p1-5.el9.1.x86_64
openssl-3.0.0-0.alpha16.4.el9.x86_64

How reproducible:
always

Steps to Reproduce:
1. openssl genrsa -aes128 -out my-test-private.key -passout pass:RedHatEnterpriseLinux9.0 2048
2. ssh-keygen -y -f my-test-private.key > public.key.pub

Actual results:
Load key "my-test-private.key": error in libcrypto

Expected results:
SSH public key in public.key.pub file

Additional info:
The issue is most likely caused by the fact that genrsa creates PKCS#8 encrypted files now, not SSLeay-format files. When genrsa is used with -traditional option, the ssh-keygen is able to parse the file.

Comment 1 Jakub Jelen 2021-07-13 22:23:03 UTC
It turned out that the new OpenSSL is sending more error codes than OpenSSH is expecting. The parsing of them is kind of fragile too. When I try to print the errors, there are new errors from the decoder layer and from passphrase, which are not recognized by openssh:

[root@rhel-9-0-0-20210629-1 openssh-portable]# ./ssh-keygen -y -f ../test.key 
4087F1A8C07F0000:error:07880109:common libcrypto routines:do_ui_passphrase:interrupted or cancelled:crypto/passphrase.c:175:
4087F1A8C07F0000:error:04800068:PEM routines:PEM_do_header:bad password read:crypto/pem/pem_lib.c:439:
4087F1A8C07F0000:error:1E08010C:DECODER routines:OSSL_DECODER_from_bio:unsupported:crypto/encode_decode/decoder_lib.c:83:Not supported for the data to decode.

OpenSSH is using ERR_peek_error() and ERR_peek_last_error() to look for errors, but it does not know either of these (it expects the middle one).

There are couple of options what we could do:
 * look into all of the errors in order until we find some familiar error suggesting the passphrase is wrong (probably best)
 * add new error codes that we will consider as a wrong passphrase
 * the cipher/passphrase.c in openssl suggests it can prompt for passphrase itself through some UI, but I do not think this is a good idea.

I tried the following patch, which allowed me to use the key I previously generated on my Fedora and that was failing to be accepted previously in RHEL9, but it will require some more love:

diff --git a/sshkey.c b/sshkey.c
index 51482929..7317549a 100644
--- a/sshkey.c
+++ b/sshkey.c
@@ -4519,6 +4519,18 @@ translate_libcrypto_error(unsigned long pem_err)
 		default:
 			return SSH_ERR_LIBCRYPTO_ERROR;
 		}
+#ifdef ERR_LIB_OSSL_DECODER
+	case ERR_LIB_OSSL_DECODER:
+		switch (pem_reason) {
+#ifdef ERR_R_UNSUPPORTED
+		case ERR_R_UNSUPPORTED:
+			return SSH_ERR_KEY_WRONG_PASSPHRASE;
+#endif
+		default:
+			return SSH_ERR_LIBCRYPTO_ERROR;
+		}
+
+#endif
 	case ERR_LIB_ASN1:
 		return SSH_ERR_INVALID_FORMAT;
 	}

Comment 2 Dmitry Belyavskiy 2021-07-21 11:01:06 UTC
RHEL 8.5 in this case requests the passphrase which I think is the desired behaviour

Comment 3 Dmitry Belyavskiy 2021-07-21 12:52:05 UTC
https://github.com/openssl/openssl/pull/14834 seems to be the relevant OpenSSL changeset

Comment 4 Dmitry Belyavskiy 2021-07-21 14:50:59 UTC
Created attachment 1804145 [details]
An openssl 3.0-compatibility patch

Comment 7 Dmitry Belyavskiy 2021-07-26 11:38:22 UTC
Works for OpenSSL 3.0-beta1 without any patches.