Bug 2126100 - Pulling Red Hat container images on CentOS 8 stream problem due to /etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-beta
Summary: Pulling Red Hat container images on CentOS 8 stream problem due to /etc/pki/r...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: containers-common
Version: CentOS Stream
Hardware: x86_64
OS: Linux
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Jindrich Novy
QA Contact: David Darrah/Red Hat QE
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-09-12 12:18 UTC by Lennart Schedin
Modified: 2023-07-04 06:47 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-07-04 06:47:48 UTC
Type: Bug
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHELPLAN-133742 0 None None None 2022-09-12 12:29:40 UTC

Description Lennart Schedin 2022-09-12 12:18:36 UTC
How to reproduce:
1. Install a new CentOS 8 Stream VM. I used http://mirror.zetup.net/CentOS/8-stream/isos/x86_64/CentOS-Stream-8-x86_64-20220831-boot.iso as a base and selected “Server”

2. Install podman:
sudo dnf -y install podman

3. Pull the Red Hat Ubi8 image:
podman pull registry.access.redhat.com/ubi8:8.6-903.1661794351

Actual:
[root@localhost ~]# podman pull registry.access.redhat.com/ubi8:8.6-903.1661794351
Trying to pull registry.access.redhat.com/ubi8:8.6-903.1661794351...
Error: copying system image from manifest list: Source image rejected: None of the signatures were accepted, reasons: open /etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-beta: no such file or directory; open /etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-beta: no such file or directory; open /etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-beta: no such file or directory; open /etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-beta: no such file or directory; open /etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-beta: no such file or directory; open /etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-beta: no such file or directory

Expected:
The Red Hat Ubi8 image should be pulled and successfully stored in the local container image storage.

Environment:
[root@localhost ~]# cat /etc/redhat-release
CentOS Stream release 8
[root@localhost ~]# rpm -q centos-stream-release
centos-stream-release-8.6-1.el8.noarch
[root@localhost ~]# rpm -q containers-common
containers-common-1-40.module_el8.7.0+1196+721f4eb0.x86_64
[root@localhost ~]# rpm -q podman
podman-4.2.0-1.module_el8.7.0+1196+721f4eb0.x86_64

Analysis:
The root cause of the problem is somewhere related to the commit https://git.centos.org/rpms/containers-common/c/38a67aceb2ae0a4000ce2f29e8ebbccc2042215d where the file /etc/containers/policy.json was changed to also include the key /etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-beta for the registry.access.redhat.com registry.

The sibling key /etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release is included in containers-common RPM in CentOS 8 Stream, but on RHEL 8 both keys are distributed in the redhat-release RPM. The (half-)corresponding RPM on CentOS side (centos-stream-release) does not have the keys.

My guess is that CentOS 8 Stream has an inconsistency with respect to the keys and what is part of the policy.json file. Either this inconsistency needs to be solved, or making the podman pull command not crash when pulling images for where some key does not exist on disk.

(Ugly) workaround 1:
Create a dummy key file: 
[root@localhost ~]# touch /etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-beta

(Ugly) workaround 2:
Edit the file /etc/containers/policy.json and remove /etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-beta from it.

Comment 2 Lennart Schedin 2023-05-08 06:21:40 UTC
This ticket has been solved because the file /etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release has been added to containers-common 1-46 for CentOS 8 stream:

[builder@7f7981bdc33c ~]$ cat  /etc/redhat-release
CentOS Stream release 8

[builder@7f7981bdc33c ~]$ rpm -q -l containers-common-1-49.module_el8.8.0+1254+78119b6e.x86_64 | grep RPM-GPG-KEY
/etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-beta
/etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release

Comment 3 Jindrich Novy 2023-07-04 06:47:48 UTC
Thank you for your comment Lennart.


Note You need to log in before you can comment on or make changes to this bug.