Bug 2059424

Summary:	rpm compiled with openssl fails to import RPM-GPG-KEY-CentOS-SIG-Extras
Product:	Red Hat Enterprise Linux 9	Reporter:	Manu Bretelle <chantr4>
Component:	distribution	Assignee:	Brian Stinson <bstinson>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Brian Stinson <bstinson>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	CentOS Stream	CC:	ajb, bstinson, carl, cllang, daltonminer, davdunc, davide, davidmccheyne, dbelyavs, gary.buhrmaster, jwboyer, lisas, mercer, michel, ngompa13, phil, pmatilai, pmendezh, toracat
Target Milestone:	rc	Keywords:	Reopened
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	centos-stream-release-9.0-12.el9	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2022-03-10 19:08:16 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Manu Bretelle 2022-03-01 02:01:55 UTC

Description of problem:


Version-Release number of selected component (if applicable): rpm-4.16.1.3-11.el9.x86_64


How reproducible: always


Steps to Reproduce:
1. Try to import `RPM-GPG-KEY-CentOS-SIG-Extras`
2. key fails to import: key 1 import failed.

Actual results:

# rpmkeys --import /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Extras 
error: /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Extras: key 1 import failed.



Expected results:

successfully import the key.

Additional info:

pgpdump of RPM-GPG-KEY-centosofficial is available at https://pastebin.com/RMGc1cdw

RPM-GPG-KEY-CentOS-SIG-Extras https://pastebin.com/tbP2nPQb


# rpm -q libgcrypt openssl rpm
libgcrypt-1.10.0-2.el9.x86_64
openssl-3.0.1-12.el9.x86_64
rpm-4.16.1.3-11.el9.x86_64


When swapping `--with-crypto=openssl` to `--with-crypto=libgcrypt`, the problem does not reproduce.

I have not had the opportunity to dig through the exact reason this is failing with openssl, but speculate that the new key (created Wed Dec 15 23:09:47 PST 2021) uses a modern algo or such which is not handled by c9s openssl version.

Comment 1 Neal Gompa 2022-03-01 02:20:48 UTC

Note that the key *does* import on RPM 4.17.0 in Fedora Linux 36 (which links to OpenSSL 3.0)

Fedora Linux 36 reproducer:

ngompa@fedora ~> podman run --pull=always --rm -it fedora:36
Trying to pull registry.fedoraproject.org/fedora:36...
Getting image source signatures
Copying blob 9b4565f05748 done
Copying config c334c89acc done
Writing manifest to image destination
Storing signatures
[root@07581dff93bd /]# rpm -q openssl-libs
openssl-libs-3.0.0-1.fc36.x86_64
[root@07581dff93bd /]# rpm -q rpm
rpm-4.17.0-9.fc36.x86_64
[root@07581dff93bd /]# rpm --verbose --import https://raw.githubusercontent.com/xsuchy/distribution-gpg-keys/main/keys/centos/RPM-GPG-KEY-CentOS-SIG-Extras
[root@07581dff93bd /]# rpm -qi gpg-pubkey-1d997668-61bae63b
Name        : gpg-pubkey
Version     : 1d997668
Release     : 61bae63b
Architecture: (none)
Install Date: Tue Mar  1 02:16:51 2022
Group       : Public Keys
Size        : 0
License     : pubkey
Signature   : (none)
Source RPM  : (none)
Build Date  : Thu Dec 16 07:09:47 2021
Build Host  : localhost
Packager    : CentOS Extras SIG (https://wiki.centos.org/SpecialInterestGroup) <security>
Summary     : CentOS Extras SIG (https://wiki.centos.org/SpecialInterestGroup) <security> public key
Description :
-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: rpm-4.17.0 (NSS-3)

mQENBGG65jsBCADef7Fspss6f2PKrlrxufWlBaQI+kcdSDbY7o/dyyjpT7dcX8t8
Ou73irjiShK3q0pdrh1Wy/mXc7RIJwAbCt9OVgyx4PV6AW5LfU7P7xyEAbTgLhz9
lLPjBGhBvfRpW+7naPqkTcIKxpVR8Khq6fsvThGCNzNkGa46F1srE3mf1zC9wdVR
VtXO7gHEZ2LrNcl195jZkBQOLcXANcSOFh5eRfhumULmk4XgCGmZQT5UNFofqOmn
aWQGBq3XaU7RWjl7RH+IS2EW0rAtz9Le+cH+j0aFhzo7jBMOxGYG62rUaHdxssjV
S1CrfpYT6NeG5i/1hiP4hO9suezJw4yuXNZ3ABEBAAG0VkNlbnRPUyBFeHRyYXMg
U0lHIChodHRwczovL3dpa2kuY2VudG9zLm9yZy9TcGVjaWFsSW50ZXJlc3RHcm91
cCkgPHNlY3VyaXR5QGNlbnRvcy5vcmc+iQE5BBMBAgAjBQJhuuY7AhsvBwsJCAcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQH/aiFx2ZdmjUtwf9GX3exQy6bC/A7miq
I0yfoBR+jvZQKy7+U8vexbr0cgkYDTJ2zN3y+JL1391Y9CS0oDNqYLIv1BwHXAmX
EarpQV/YyEocnYXwcVLugKCnbIN92vMTiyb/NESx1vHbduK+B8wWo3bp3sPK+Ha/
zXrHXWSEgUeCBY/b7Tbl3GW8NX9Pr+yY0zHcvTfLByVH0KpNNLsyOsrCdk4MSKMl
IBZWDaUYVAbyHXB92wZlQOKp+HqRxNhceGHTzeXBymK1LadntlCYTaqsg3ErRq8p
ZwkpeyAi/avjIPYc53QE3dKGw2cUjZxkMOe6BoMbeLlO3+INdJBc/gcW4xUsQ28Y
QtY8jLkBDQRhuuY7AQgAs+enJDbwE/Iln3BnxodDQ3/1t9ULlMLJLiV+FgS7yREZ
QvhVQxFWaJqbiPV6EJVxEP5lUHND2DAE2ZTr60y0rI3ZAY52go+QYHXb+M5HC12H
HbhIDTWaETNo5heq/qyVSRT1u0g/yKCxQdyqnVsL86bro0wgrpj7XuApQifFhy16
AkDjhcB0C0dXkfvEnHJylWiHpp7upfSgOcGwQ+yRHOZWJnyF+OMrFfNiwD74/zEN
4RoNFgpqJZ81TF0qCdllTYGAXXUdYsJlg64dH0u84naTOFIuInywCmNyPmC8e8/0
g56hCV2L7bRJGjBCa6VH+TgvVGnkFsoMM9ijhuTIIQARAQABiQI+BBgBAgAJBQJh
uuY7AhsuASkJEB/2ohcdmXZowF0gBBkBAgAGBQJhuuY7AAoJEItcgRH8pdD/Zv0I
AIdipgQOIf7JcWmXli29nFedwdY1RJbKxABP2vMioYXDXjTkLBFsE9v4upnVnIZy
EM6IyrN8qaEVGMbCAiG+G+ABc3P/2mIWjlk4TUR4dFMGrnjbtGAalRd1EVJX/x8N
Vr9er+EMJDYR0PhKIQkZbUdjwYwB0R1wyJsvTsVf7k52Y013oI9iZd7nkOZxeKZX
qlHLbf1fSY6szmHEGVu/LNY8gUc3n2z8UByiqeJ2unrU5iL4FQ45DAoUWT0Y2WBF
WTYvJCTNGmiuKroBqC+nFgUekWJGvv3hBTHd5Eq54PWoWzV9BeM7XWdmpT66xOgb
ye5/zLerWSoC+8fVdBqW7LSEqggAzRaID+mLFPTe2LbQkaBkmIpqeoDOy700Xy6K
VW05GndH0E0t86DbQClFyzucYLzX2dXyV2DWjoWDIevQnS51zzsd+lWyuKPICKte
+K+yk5QEiwgaDf5oPmI8WL2zIAfiwVHlU0epMLU1pZLAQYotsQ6m5qPPMVXcfIqF
3UJwZEnZRccfOKq1hHSS2/Ns4ihAfkrfes1IFLSzbyvinXQUqFVrY8oZCKhNPSRd
IXXPIx0KvnlI9e0EittvsrQxebAa2MwLXOVYL8WVvOLY7oNTrOxe45jOdzMz2+rK
dodVWwuBwNKuwSE6b5A0dwUj8ZEo/5L4noufZF6aGOLdbVcoUg==
=RGYd
-----END PGP PUBLIC KEY BLOCK-----


CentOS Stream 9 reproducer:

ngompa@fedora ~> podman run --pull=always --rm -it centos:stream9
Trying to pull quay.io/centos/centos:stream9...
Getting image source signatures
Copying blob 972147051161 skipped: already exists
Copying config 1ddb9bedee done
Writing manifest to image destination
Storing signatures
[root@f367491929dc /]# rpm -q openssl-libs
openssl-libs-3.0.1-12.el9.x86_64
[root@f367491929dc /]# rpm -q rpm
rpm-4.16.1.3-11.el9.x86_64
[root@f367491929dc /]# rpm --verbose --import https://raw.githubusercontent.com/xsuchy/distribution-gpg-keys/main/keys/centos/RPM-GPG-KEY-CentOS-SIG-Extras
error: https://raw.githubusercontent.com/xsuchy/distribution-gpg-keys/main/keys/centos/RPM-GPG-KEY-CentOS-SIG-Extras: key 1 import failed.

The difference between Fedora and CentOS/RHEL here is that CS9 is further along on OpenSSL 3.0 (with 3.0.1) and Fedora has RPM 4.16.1.3 with some 4.17.0 backports.

Comment 2 Neal Gompa 2022-03-01 02:31:23 UTC

> The difference between Fedora and CentOS/RHEL here is that CS9 is further along on OpenSSL 3.0 (with 3.0.1) and Fedora has RPM 4.16.1.3 with some 4.17.0 backports.

Ugh, wow. I mean CS9 has OpenSSL 3.0.1 and RPM 4.16.1.3 with 4.17.0 backports, while Fedora has OpenSSL 3.0.0 and RPM 4.17.0.

Comment 3 Manu Bretelle 2022-03-01 02:34:46 UTC

As another datapoint, building rpm master on FC35 against `openssl-1.1.1l-2.fc35.x86_64`, the issue does not reproduce.

Comment 4 Carl George 🤠 2022-03-01 05:11:12 UTC

I tried a several recent openssl builds, and it appears that -9 is where this got broken.

openssl-3.0.1-14.el9	fails
openssl-3.0.1-13.el9	fails
openssl-3.0.1-12.el9	fails
openssl-3.0.1-11.el9	fails
openssl-3.0.1-10.el9	fails
openssl-3.0.1-9.el9	fails
openssl-3.0.1-7.el9	works
openssl-3.0.1-5.el9	works
openssl-3.0.1-4.el9	works

I'm not sure if it's relevant or not, but I noticed that the RPM-GPG-KEY-centosofficial key is rsa4096, but the RPM-GPG-KEY-CentOS-SIG-Extras key is rsa2048.  Could openssl be blocking it because of this?

Comment 5 Manu Bretelle 2022-03-01 06:04:24 UTC

https://gitlab.com/redhat/centos-stream/rpms/openssl/-/commit/d79f404164cf13ca54b1e8aef814d22007b77fbd
https://gitlab.com/redhat/centos-stream/rpms/openssl/-/commit/53b85f538ce686d9753c04f4ac7a75d117b8f32e
https://gitlab.com/redhat/centos-stream/rpms/openssl/-/commit/78fb78d30755ae18fdaef28ef392f4e67c662ff6

are the patchesthat made it in between openssl-3.0.1-7.el9 and openssl-3.0.1-9.el9.

Comment 6 Panu Matilainen 2022-03-01 09:29:25 UTC


*** This bug has been marked as a duplicate of bug 2059101 ***

Comment 7 Clemens Lang 2022-03-01 10:14:23 UTC

This isn't a duplicate of the rpm or openssl bug, it's a bug in the centos-release package, which uses a SHA1 signature of a subkey that can also be used for signing. It's the equivalent of https://bugzilla.redhat.com/show_bug.cgi?id=2058497, except it shouldn't be assigned to redhat-release, but centos-release. There isn't a centos-release component in bugzilla as far as I can see, so I'm not sure where to send it.

The change that caused this to fail is https://bugzilla.redhat.com/show_bug.cgi?id=2031742, btw.

Comment 8 Clemens Lang 2022-03-01 10:29:49 UTC

I didn't want to change the resolution or component, but since I accidentally did that now, might as well re-open so that the redhat-release maintainers can hopefully re-assign this correctly.

Comment 9 Neal Gompa 2022-03-01 16:54:50 UTC

Can someone please remove the automatic redhat-private flag on this? There's nothing sensitive in this BZ.

Comment 11 Clemens Lang 2022-03-03 08:53:45 UTC

https://gitlab.com/redhat/centos-stream/rpms/centos-release/-/commit/d11bd36e8491652d21ceca69e81ef7aeee433445
https://gitlab.com/redhat/centos-stream/rpms/centos-release/-/commit/ba747b459dd4929fef03102438aef3d4c015ebd1

Comment 12 Carl George 🤠 2022-03-10 19:08:16 UTC

This is fixed in centos-stream-release-9.0-12.el9, which is now available on the mirrors and in the latest container image.

[carl@teal:~]$ podman run -it --rm --pull always centos:stream9
Trying to pull quay.io/centos/centos:stream9...
Getting image source signatures
Copying blob f1891b8c8dcd skipped: already exists  
Copying config 44ffcc4ace done  
Writing manifest to image destination
Storing signatures
[root@9155e2b271cd /]# rpm -q centos-stream-release
centos-stream-release-9.0-12.el9.noarch
[root@9155e2b271cd /]# dnf --quiet repoquery --nvr --latest-limit 1 centos-stream-release
centos-stream-release-9.0-12.el9
[root@9155e2b271cd /]# rpm -q gpg-pubkey
package gpg-pubkey is not installed
[root@9155e2b271cd /]# rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Extras-SHA512
[root@9155e2b271cd /]# rpm -q gpg-pubkey
gpg-pubkey-1d997668-621e3cac