Bug 2020026
| Summary: | Unable to pull UBI 9 images from non-RHEL | |||
|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 9 | Reporter: | Shion Tanaka <shtanaka> | |
| Component: | doc-Release_Notes-9-en-US | Assignee: | Gabi Fialová <gfialova> | |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | RHEL DPM <rhel-docs> | |
| Severity: | high | Docs Contact: | Gabriela Nečasová <gnecasov> | |
| Priority: | high | |||
| Version: | 9.0 | CC: | agerstmayr, dmoppert, dornelas, dwalsh, gfialova, gnecasov, jnovy, jwboyer, lfriedma, mhofmann, mitr, ngompa13, rhel-docs, tsweeney, vrothber | |
| Target Milestone: | rc | Keywords: | Documentation, Reopened, Triaged | |
| Target Release: | 9.1 | Flags: | pm-rhel:
mirror+
|
|
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | ||||
| Fixed In Version: | Doc Type: | No Doc Update | ||
| Doc Text: | Story Points: | --- | ||
| Clone Of: | ||||
| : | 2020301 2094015 (view as bug list) | Environment: | ||
| Last Closed: | 2022-06-06 13:28:38 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | ||||
| Bug Blocks: | 2020301, 2094015 | |||
I got the same error in rhel9-beta/toolbox. It seems to be the same of other tags(rhel9-beta/*).
```
$ podman pull registry.access.redhat.com/rhel9-beta/toolbox
Trying to pull registry.access.redhat.com/rhel9-beta/toolbox:latest...
Error: Source image rejected: None of the signatures were accepted, reasons: Invalid GPG signature: gpgme.Signature{Summary:128, Fingerprint:"938A80CAF21541EB", Status:gpgme.Error{err:0x9}, Timestamp:time.Time{wall:0x0, ext:63771441427, loc:(*time.Location)(0x55d730c21680)}, ExpTimestamp:time.Time{wall:0x0, ext:62135596800, loc:(*time.Location)(0x55d730c21680)}, WrongKeyUsage:false, PKATrust:0x0, ChainModel:false, Validity:0, ValidityReason:error(nil), PubkeyAlgo:1, HashAlgo:8}; Invalid GPG signature: gpgme.Signature{Summary:128, Fingerprint:"938A80CAF21541EB", Status:gpgme.Error{err:0x9}, Timestamp:time.Time{wall:0x0, ext:63771441427, loc:(*time.Location)(0x55d730c21680)}, ExpTimestamp:time.Time{wall:0x0, ext:62135596800, loc:(*time.Location)(0x55d730c21680)}, WrongKeyUsage:false, PKATrust:0x0, ChainModel:false, Validity:0, ValidityReason:error(nil), PubkeyAlgo:1, HashAlgo:8}; Invalid GPG signature: gpgme.Signature{Summary:128, Fingerprint:"938A80CAF21541EB", Status:gpgme.Error{err:0x9}, Timestamp:time.Time{wall:0x0, ext:63771441428, loc:(*time.Location)(0x55d730c21680)}, ExpTimestamp:time.Time{wall:0x0, ext:62135596800, loc:(*time.Location)(0x55d730c21680)}, WrongKeyUsage:false, PKATrust:0x0, ChainModel:false, Validity:0, ValidityReason:error(nil), PubkeyAlgo:1, HashAlgo:8}; Invalid GPG signature: gpgme.Signature{Summary:128, Fingerprint:"938A80CAF21541EB", Status:gpgme.Error{err:0x9}, Timestamp:time.Time{wall:0x0, ext:63771441428, loc:(*time.Location)(0x55d730c21680)}, ExpTimestamp:time.Time{wall:0x0, ext:62135596800, loc:(*time.Location)(0x55d730c21680)}, WrongKeyUsage:false, PKATrust:0x0, ChainModel:false, Validity:0, ValidityReason:error(nil), PubkeyAlgo:1, HashAlgo:8}; Invalid GPG signature: gpgme.Signature{Summary:128, Fingerprint:"938A80CAF21541EB", Status:gpgme.Error{err:0x9}, Timestamp:time.Time{wall:0x0, ext:63771441429, loc:(*time.Location)(0x55d730c21680)}, ExpTimestamp:time.Time{wall:0x0, ext:62135596800, loc:(*time.Location)(0x55d730c21680)}, WrongKeyUsage:false, PKATrust:0x0, ChainModel:false, Validity:0, ValidityReason:error(nil), PubkeyAlgo:1, HashAlgo:8}; Invalid GPG signature: gpgme.Signature{Summary:128, Fingerprint:"938A80CAF21541EB", Status:gpgme.Error{err:0x9}, Timestamp:time.Time{wall:0x0, ext:63771441430, loc:(*time.Location)(0x55d730c21680)}, ExpTimestamp:time.Time{wall:0x0, ext:62135596800, loc:(*time.Location)(0x55d730c21680)}, WrongKeyUsage:false, PKATrust:0x0, ChainModel:false, Validity:0, ValidityReason:error(nil), PubkeyAlgo:1, HashAlgo:8}
```
The ubi9 Beta images are signed with the Red Hat Beta GPG key, which can be found on https://access.redhat.com/security/team/key New enough versions of podman, such as those found in Fedora CentOS Stream 8/9 or RHEL 9 Beta, default to validating signatures of images. The current builds are configured to trust production keys for the Red Hat registries but do not trust the Beta keys by default. We're looking into this issue, but a workaround is to ensure the Red Hat Beta key is on the local system and use podman image trust to set the Beta key as a trusted key for the ubi9-beta namespace. Here is an example: [jwboyer@zod ~]$ wget https://www.redhat.com/security/data/f21541eb.txt [jwboyer@zod ~]$ sudo podman image trust set -f ./f21541eb.txt registry.access.redhat.com/ubi9-beta [jwboyer@zod ~]$ podman pull registry.access.redhat.com/ubi9-beta/ubi Trying to pull registry.access.redhat.com/ubi9-beta/ubi:latest... Getting image source signatures Checking if image destination supports signatures Copying blob c3aca5a03ade done Copying blob 76a1729eef6b done Copying config 28b0a4b69d done Writing manifest to image destination Storing signatures 28b0a4b69d9b5dc8b55a9639dc372803a3145c65d2a0c36e80b52a208e18b2f6 [jwboyer@zod ~]$ @Josh Boyer Thank you for your comments. I can confirm that the workaround works.(Both ubi9-beta and rhel9-beta) ``` $ wget https://www.redhat.com/security/data/f21541eb.txt $ sudo podman image trust set -f ./f21541eb.txt registry.access.redhat.com/ubi9-beta $ sudo podman image trust set -f ./f21541eb.txt registry.access.redhat.com/rhel9-beta $ podman image trust show default accept registry.access.redhat.com signedBy security https://access.redhat.com/webassets/docker/content/sigstore registry.access.redhat.com/rhel9-beta signedBy security https://access.redhat.com/webassets/docker/content/sigstore registry.access.redhat.com/ubi9-beta signedBy security https://access.redhat.com/webassets/docker/content/sigstore registry.redhat.io signedBy security https://registry.redhat.io/containers/sigstore insecureAcceptAnything $ podman pull ubi9-beta/ubi $ podman pull rhel9-beta/toolbox $ podman images|grep -e ubi9 -e rhel9 registry.access.redhat.com/rhel9-beta/toolbox latest b4b8016fce23 3 weeks ago 591 MB registry.access.redhat.com/ubi9-beta/ubi latest 28b0a4b69d9b 3 weeks ago 229 MB ``` One note: the file passed with -f should be an absolute path rather than a relative path. Otherwise podman pull commands must be run from the directory that contains the key file. For systems that lack the Red Hat Beta key, it is best to copy the key to /etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-beta and specify that path. I moved the key and reassigned it to an absolute path. It's working perfectly. ``` $ sudo wget -O /etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-beta https://www.redhat.com/security/data/f21541eb.txt $ sudo podman image trust set -f /etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-beta registry.access.redhat.com/ubi9-beta $ sudo podman image trust set -f /etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-beta registry.access.redhat.com/rhel9-beta $ podman image trust show --raw |grep beta "registry.access.redhat.com/rhel9-beta": [ "keyPath": "/etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-beta" "registry.access.redhat.com/ubi9-beta": [ "keyPath": "/etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-beta" ``` Keeping this assigned to Jindrich for now, but have added Valentin to the cc list in case he has a thought. The question is should we ship an updated version of Podman for RHEL8, Fedora and CoreOS Stream with the beta key preinstalled? Tom, please, is this issue fixed? Thank you in advance. |
Description of problem: Cannot pull UBI 9 images with Invalid GPG signature error. ``` $ podman pull registry.redhat.io/ubi9-beta/ubi Trying to pull registry.redhat.io/ubi9-beta/ubi:latest... Error: Source image rejected: None of the signatures were accepted, reasons: Invalid GPG signature: gpgme.Signature{Summary:128, Fingerprint:"938A80CAF21541EB", Status:gpgme.Error{err:0x9}, Timestamp:time.Time{wall:0x0, ext:63771440989, loc:(*time.Location)(0x5599955cd680)}, ExpTimestamp:time.Time{wall:0x0, ext:62135596800, loc:(*time.Location)(0x5599955cd680)}, WrongKeyUsage:false, PKATrust:0x0, ChainModel:false, Validity:0, ValidityReason:error(nil), PubkeyAlgo:1, HashAlgo:8}; Invalid GPG signature: gpgme.Signature{Summary:128, Fingerprint:"938A80CAF21541EB", Status:gpgme.Error{err:0x9}, Timestamp:time.Time{wall:0x0, ext:63771440989, loc:(*time.Location)(0x5599955cd680)}, ExpTimestamp:time.Time{wall:0x0, ext:62135596800, loc:(*time.Location)(0x5599955cd680)}, WrongKeyUsage:false, PKATrust:0x0, ChainModel:false, Validity:0, ValidityReason:error(nil), PubkeyAlgo:1, HashAlgo:8}; Invalid GPG signature: gpgme.Signature{Summary:128, Fingerprint:"938A80CAF21541EB", Status:gpgme.Error{err:0x9}, Timestamp:time.Time{wall:0x0, ext:63771440990, loc:(*time.Location)(0x5599955cd680)}, ExpTimestamp:time.Time{wall:0x0, ext:62135596800, loc:(*time.Location)(0x5599955cd680)}, WrongKeyUsage:false, PKATrust:0x0, ChainModel:false, Validity:0, ValidityReason:error(nil), PubkeyAlgo:1, HashAlgo:8}; Invalid GPG signature: gpgme.Signature{Summary:128, Fingerprint:"938A80CAF21541EB", Status:gpgme.Error{err:0x9}, Timestamp:time.Time{wall:0x0, ext:63771440991, loc:(*time.Location)(0x5599955cd680)}, ExpTimestamp:time.Time{wall:0x0, ext:62135596800, loc:(*time.Location)(0x5599955cd680)}, WrongKeyUsage:false, PKATrust:0x0, ChainModel:false, Validity:0, ValidityReason:error(nil), PubkeyAlgo:1, HashAlgo:8}; Invalid GPG signature: gpgme.Signature{Summary:128, Fingerprint:"938A80CAF21541EB", Status:gpgme.Error{err:0x9}, Timestamp:time.Time{wall:0x0, ext:63771440991, loc:(*time.Location)(0x5599955cd680)}, ExpTimestamp:time.Time{wall:0x0, ext:62135596800, loc:(*time.Location)(0x5599955cd680)}, WrongKeyUsage:false, PKATrust:0x0, ChainModel:false, Validity:0, ValidityReason:error(nil), PubkeyAlgo:1, HashAlgo:8}; Invalid GPG signature: gpgme.Signature{Summary:128, Fingerprint:"938A80CAF21541EB", Status:gpgme.Error{err:0x9}, Timestamp:time.Time{wall:0x0, ext:63771440992, loc:(*time.Location)(0x5599955cd680)}, ExpTimestamp:time.Time{wall:0x0, ext:62135596800, loc:(*time.Location)(0x5599955cd680)}, WrongKeyUsage:false, PKATrust:0x0, ChainModel:false, Validity:0, ValidityReason:error(nil), PubkeyAlgo:1, HashAlgo:8} ``` ``` podman pull registry.access.redhat.com/ubi9-beta/ubi ``` Almost the same result. Version-Release number of selected component (if applicable): on CentOS Stream 8 ``` $ cat /etc/redhat-release CentOS Stream release 8 $ uname -a Linux phenex 4.18.0-348.el8.x86_64 #1 SMP Tue Oct 19 15:14:17 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux $ podman --version podman version 3.4.1-dev ``` on Fedora 35 ``` $ cat /etc/redhat-release Fedora release 35 (Thirty Five) $ uname -a Linux fedora35 5.14.14-300.fc35.x86_64 #1 SMP Wed Oct 20 16:14:50 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux $ podman --version podman version 3.4.1 ``` How reproducible: All the time. Steps to Reproduce: 1. podman pull registry.redhat.io/ubi9-beta/ubi or 1. podman pull registry.access.redhat.com/ubi9-beta/ubi Actual results: Error above. Expected results: Additional info: