Bug 1467669

Summary:	OpenSSL broken when generating RSA keys 'UI_set_result:result too small:crypto/ui/ui_lib.c:778'
Product:	[Fedora] Fedora	Reporter:	Davi Garcia <dvercill>
Component:	openssl	Assignee:	Tomas Mraz <tmraz>
Status:	CLOSED NOTABUG	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	urgent	Docs Contact:
Priority:	unspecified
Version:	26	CC:	tmraz
Target Milestone:	---
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2017-07-10 11:48:39 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Davi Garcia 2017-07-04 13:43:07 UTC

Description of problem:

OpenSSL is unable to generate file with RSA private keys on Fedora 26 using the command 'openssl genrsa -des3 -passout pass:x -out server.pass.key 2048'. The same command is functional on RHEL 7.3.

Version-Release number of selected component (if applicable):

[dvercill@blackpad ~]$ rpm -qa | grep openssl 
compat-openssl10-pkcs11-helper-1.22-1.fc26.x86_64
compat-openssl10-1.0.2j-6.fc26.x86_64
openssl-1.1.0f-4.fc26.x86_64
xmlsec1-openssl-1.2.23-2.fc26.x86_64
openssl-libs-1.1.0f-4.fc26.x86_64

How reproducible:

Easily.

Steps to Reproduce:
1. Run command 'openssl genrsa -des3 -passout pass:x -out server.pass.key 2048'
2. Check file 'server.pass.key'

Actual results:

The command prints errors messages and generate a empty file.

Expected results:

The command should create a file containing the RSA private key.

Comment 1 Davi Garcia 2017-07-04 13:43:53 UTC

[user@host ~]$ openssl genrsa -des3 -passout pass:x -out server.pass.key 2048
Generating RSA private key, 2048 bit long modulus
...............................................................+++
...................................................................+++
e is 65537 (0x010001)
140107418351360:error:28069065:UI routines:UI_set_result:result too small:crypto/ui/ui_lib.c:778:You must type in 4 to 1023 characters
140107418351360:error:28069065:UI routines:UI_set_result:result too small:crypto/ui/ui_lib.c:778:You must type in 4 to 1023 characters
140107418351360:error:0906906F:PEM routines:PEM_ASN1_write_bio:read key:crypto/pem/pem_lib.c:336:

Comment 2 Tomas Mraz 2017-07-10 11:48:39 UTC

You have to pass a password that is longer than the existing 'x' (minimum is 4 characters) or not use encryption at all - use this command:

openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out server.key

Comment 3 Davi Garcia 2017-07-10 14:55:14 UTC

(In reply to Tomas Mraz from comment #2)
> You have to pass a password that is longer than the existing 'x' (minimum is
> 4 characters) or not use encryption at all - use this command:
> 
> openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out server.key

Alright! Should I consider that as OpenSSL had a behavioral change from version OpenSSL 1.0.1e-fips to OpenSSL 1.1.0f-fips? Because that same command does not generate the same error when running at RHEL 7.

Comment 4 Tomas Mraz 2017-07-10 15:45:46 UTC

Yes, this was intentional behavioral change from 1.0.1 to 1.1.0.