Bug 2027576
| Summary: | podman fail to create container for "x509: certificate signed by unknown authority" issue | |||
|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 9 | Reporter: | yanpliu <yanpliu> | |
| Component: | podman | Assignee: | Gabriela Nečasová <gnecasov> | |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | atomic-bugs <atomic-bugs> | |
| Severity: | medium | Docs Contact: | Gabriela Nečasová <gnecasov> | |
| Priority: | unspecified | |||
| Version: | 9.0 | CC: | bbaude, dornelas, dwalsh, gnecasov, jnovy, jwboyer, lkuprova, lsm5, mheon, pthomas, qianzhan, smccarty, tsweeney, umohnani | |
| Target Milestone: | rc | Keywords: | Triaged | |
| Target Release: | --- | |||
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | ||||
| Fixed In Version: | Doc Type: | Bug Fix | ||
| Doc Text: |
.Podman no longer fails to pull a container "X509: certificate signed by unknown authority"
Previously, if you had your own internal registry signed by our own CA certificate, then you had to import the certificate onto your host machine. Otherwise, an error occurs:
----
x509: certificate signed by unknown authority
----
With this update, the problem has been fixed.
|
Story Points: | --- | |
| Clone Of: | ||||
| : | 2029912 2029913 (view as bug list) | Environment: | ||
| Last Closed: | 2022-03-17 08:06:08 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | ||||
| Bug Blocks: | 2029912, 2029913 | |||
|
Description
yanpliu
2021-11-30 06:14:37 UTC
Jindrich, I'm keeping this assigned to you for now, but I think this may be something Josh or Scott will need to change/handle in the ub9 container. I've added a few others to the cc list to. I'm guessing this is because podman is trying to validate the certificate the internal registry uses for encryption and it doesn't trust the cert. This doesn't seem like it has anything to do with the image. Yanpliu, I think your example is missing a step. WHen I try: # subscription-manager register --username=admin --password=password --org=Default_Organization --environment=Library --force --auto-attach Unregistering from: subscription.rhsm.redhat.com:443/subscription The system with UUID a3207486-aece-4d76-9353-34ae51729386 has been unregistered All local data removed Registering to: subscription.rhsm.redhat.com:443/subscription Error: Server does not support environments. Which looks like I need a hosted cert first. Do you know how/where I would do that? I found references in the BZ, but the certs don't appear to be available from the locations stated there. And/or @jwboyer do you happen to know? Hello Tom, My bug reproduce is on Satellite environment, it specified the org and environment, your steps are registered to custom portal. I reproduce it on Stage candlepin [root@kvm-02-guest24 ~]# subscription-manager register Registering to: subscription.rhsm.stage.redhat.com:443/subscription Username: rhsm_integration_sca1 Password: The system has been registered with ID: 896b08a7-fa73-47bb-9692-ac372f79307f The registered system name is: kvm-02-guest24.lab.eng.rdu2.redhat.com [root@kvm-02-guest24 ~]# subscription-manager register --force --auto-attach Unregistering from: subscription.rhsm.stage.redhat.com:443/subscription The system with UUID 896b08a7-fa73-47bb-9692-ac372f79307f has been unregistered All local data removed Registering to: subscription.rhsm.stage.redhat.com:443/subscription Username: rhsm_integration_sca1 Password: The system has been registered with ID: d757738b-dd9e-4df9-ba12-125749d4aea7 The registered system name is: kvm-02-guest24.lab.eng.rdu2.redhat.com Installed Product Current Status: Product Name: Red Hat Enterprise Linux for x86_64 Beta Status: Subscribed [root@kvm-02-guest24 ~]# ls /etc/pki/entitlement/ 1379435967326307764-key.pem 1379435967326307764.pem [root@kvm-02-guest24 ~]# ls -l /etc/pki/entitlement/ total 12 -rw-r--r--. 1 root root 3243 Dec 12 21:41 1379435967326307764-key.pem -rw-r--r--. 1 root root 7689 Dec 12 21:41 1379435967326307764.pem [root@kvm-02-guest24 ~]# podman create -t --name test_container registry-proxy.engineering.redhat.com/rh-osbs/ubi9 && podman start test_container Trying to pull registry-proxy.engineering.redhat.com/rh-osbs/ubi9:latest... Error: initializing source docker://registry-proxy.engineering.redhat.com/rh-osbs/ubi9:latest: pinging container registry registry-proxy.engineering.redhat.com: Get "https://registry-proxy.engineering.redhat.com/v2/": x509: certificate signed by unknown authority (In reply to Tom Sweeney from comment #4) > Yanpliu, > > I think your example is missing a step. WHen I try: > > # subscription-manager register --username=admin --password=password > --org=Default_Organization --environment=Library --force --auto-attach > Unregistering from: subscription.rhsm.redhat.com:443/subscription > The system with UUID a3207486-aece-4d76-9353-34ae51729386 has been > unregistered > All local data removed > Registering to: subscription.rhsm.redhat.com:443/subscription > Error: Server does not support environments. > > Which looks like I need a hosted cert first. Do you know how/where I would > do that? I found references in the BZ, but the certs don't appear to be > available from the locations stated there. And/or @jwboyer do > you happen to know? I don't know. I don't think that's the problem anyway. (In reply to yanpliu from comment #5) > Hello Tom, > > My bug reproduce is on Satellite environment, it specified the org and > environment, your steps are registered to custom portal. > I reproduce it on Stage candlepin > > [root@kvm-02-guest24 ~]# subscription-manager register > Registering to: subscription.rhsm.stage.redhat.com:443/subscription > Username: rhsm_integration_sca1 > Password: > The system has been registered with ID: 896b08a7-fa73-47bb-9692-ac372f79307f > The registered system name is: kvm-02-guest24.lab.eng.rdu2.redhat.com > [root@kvm-02-guest24 ~]# subscription-manager register --force > --auto-attach > Unregistering from: subscription.rhsm.stage.redhat.com:443/subscription > The system with UUID 896b08a7-fa73-47bb-9692-ac372f79307f has been > unregistered > All local data removed > Registering to: subscription.rhsm.stage.redhat.com:443/subscription > Username: rhsm_integration_sca1 > Password: > The system has been registered with ID: d757738b-dd9e-4df9-ba12-125749d4aea7 > The registered system name is: kvm-02-guest24.lab.eng.rdu2.redhat.com > Installed Product Current Status: > Product Name: Red Hat Enterprise Linux for x86_64 Beta > Status: Subscribed > > [root@kvm-02-guest24 ~]# ls /etc/pki/entitlement/ > 1379435967326307764-key.pem 1379435967326307764.pem > [root@kvm-02-guest24 ~]# ls -l /etc/pki/entitlement/ > total 12 > -rw-r--r--. 1 root root 3243 Dec 12 21:41 1379435967326307764-key.pem > -rw-r--r--. 1 root root 7689 Dec 12 21:41 1379435967326307764.pem > [root@kvm-02-guest24 ~]# podman create -t --name test_container > registry-proxy.engineering.redhat.com/rh-osbs/ubi9 && podman start > test_container > Trying to pull registry-proxy.engineering.redhat.com/rh-osbs/ubi9:latest... > Error: initializing source > docker://registry-proxy.engineering.redhat.com/rh-osbs/ubi9:latest: pinging > container registry registry-proxy.engineering.redhat.com: Get > "https://registry-proxy.engineering.redhat.com/v2/": x509: certificate > signed by unknown authority This is still just saying it doesn't trust the cert that the registry is signed with. That registry is an internal proxy, which is different from access.redhat.com or quay.io. If you look at that cert, it's signed by the Red Hat CA. If that's not in your trust chain on that machine, it's going to reject it. Common Name (CN) registry-proxy.engineering.redhat.com Organization (O) Red Hat, Inc. Organizational Unit (OU) PnT DevOps Common Name (CN) Certificate Authority Organization (O) Red Hat Organizational Unit (OU) prod This is a host setup issue. You need to import the Red Hat CA certs on that host. [jwboyer@zod ~]$ podman pull registry-proxy.engineering.redhat.com/rh-osbs/ubi9 Trying to pull registry-proxy.engineering.redhat.com/rh-osbs/ubi9:latest... Error: initializing source docker://registry-proxy.engineering.redhat.com/rh-osbs/ubi9:latest: pinging container registry registry-proxy.engineering.redhat.com: Get "https://registry-proxy.engineering.redhat.com/v2/": x509: certificate signed by unknown authority [jwboyer@zod ~]$ sudo su - [root@zod ~]# cd /etc/pki/ca-trust/source/anchors/ [root@zod anchors]# curl -O https://password.corp.redhat.com/RH-IT-Root-CA.crt % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 1517 100 1517 0 0 1447 0 0:00:01 0:00:01 --:--:-- 1447 [root@zod anchors]# update-ca-trust [root@zod anchors]# exit logout [jwboyer@zod ~]$ podman pull registry-proxy.engineering.redhat.com/rh-osbs/ubi9 Trying to pull registry-proxy.engineering.redhat.com/rh-osbs/ubi9:latest... Getting image source signatures Copying blob 0eea3ccadc41 done Copying blob aa43eab6a004 done Copying config a4b4bef98e done Writing manifest to image destination Storing signatures a4b4bef98ee0514be6e1d1ce034c674f432148c37192c6624085ea2ee673d8d4 [jwboyer@zod ~]$ Hello Josh, Thanks for your update. This step can pass after manually update RH-IT-Root-CA.crt. We test this scenarios as customer, customer do not need to do this on rhel guest. And this scenario works well on RHEL9.0 Alpha/Beta and RHEl8.6/8.5 etc, all do not need to manually update RH-IT-Root-CA.crt. (In reply to yanpliu from comment #7) > Hello Josh, > Thanks for your update. > This step can pass after manually update RH-IT-Root-CA.crt. > We test this scenarios as customer, customer do not need to do this on rhel > guest. > And this scenario works well on RHEL9.0 Alpha/Beta and RHEl8.6/8.5 etc, all > do not need to manually update RH-IT-Root-CA.crt. Customers do not have access to a registry that is internal only to Red Hat and signed with RH-IT-Root-CA.crt. Testing against that registry isn't a valid customer scenario. The equivalent customer scenario is if the customer has their own internal registry signed with their own CA cert, where they would have to perform similar steps to get podman to pull images. This should probably be documented in the release notes if it isn't already. @gnecasov Do we have anything in the release notes that covers this situation? Hello Lenka, Thank you very much for telling me this information. I can find Doc Text field in "Show advanced fields". Thank you all! Retest on DISTRO=RHEL-9.1.0-20220309.3 Satellite6.10, this issue did not reproduce. [root@kvm-04-guest16 ~]# subscription-manager register --username=admin --password=admin --org=Default_Organization --environment=Library Registering to: hpe-nehalem-02.hpe2.lab.eng.bos.redhat.com:443/rhsm The system has been registered with ID: 64f361c9-d187-4c2c-89f7-df7a345544f7 The registered system name is: kvm-04-guest16.lab.eng.rdu2.redhat.com [root@kvm-04-guest16 ~]# subscription-manager register --username=admin --password=admin --org=Default_Organization --environment=Library --force --auto-attach Unregistering from: hpe-nehalem-02.hpe2.lab.eng.bos.redhat.com:443/rhsm The system with UUID 64f361c9-d187-4c2c-89f7-df7a345544f7 has been unregistered All local data removed Registering to: hpe-nehalem-02.hpe2.lab.eng.bos.redhat.com:443/rhsm The system has been registered with ID: 37760171-9f35-48a1-9c4a-2a7459c8d9d6 The registered system name is: kvm-04-guest16.lab.eng.rdu2.redhat.com Installed Product Current Status: Product Name: Red Hat Enterprise Linux for x86_64 Beta Status: Subscribed [root@kvm-04-guest16 ~]# ls /etc/pki/entitlement/ 5695550158646135769-key.pem 5695550158646135769.pem [root@kvm-04-guest16 ~]# ls -l /etc/pki/entitlement/ total 12 -rw-r--r--. 1 root root 3243 Mar 14 21:19 5695550158646135769-key.pem -rw-r--r--. 1 root root 4386 Mar 14 21:19 5695550158646135769.pem [root@kvm-04-guest16 ~]# podman create -t --name test_container registry-proxy.engineering.redhat.com/rh-osbs/ubi9 && podman start test_container Trying to pull registry-proxy.engineering.redhat.com/rh-osbs/ubi9:latest... Getting image source signatures Copying blob ac30c696644e done Copying blob 0765046881e5 done Copying config 6c694af905 done Writing manifest to image destination Storing signatures 914654dc2913059e1ba4ed15a4ed1b224b90c1653352633960b59358e85f3dd2 test_container @ @ Yanpliu, can you verify that this is now working as it should or if the error still exists and we still need to document. I'm not sure which is reproducing. (apologies for the two empty comments, Bugzilla is not my friend this morning). Hello Tom, I have verified this issue on RHEL-9.1.0-20220309.3 register against Satellite6.10, create and start container pass with manually update RH-IT-Root-CA.crt [root@kvm-04-guest16 ~]# rpm -qa |grep podman podman-catatonit-4.0.0-6.el9.x86_64 podman-4.0.0-6.el9.x86_64 cockpit-podman-42-1.el9.noarch You can see the verify steps on Comment 19. (In reply to Tom Sweeney from comment #22) > Yanpliu, can you verify that this is now working as it should or if the > error still exists and we still need to document. I'm not sure which is > reproducing. (apologies for the two empty comments, Bugzilla is not my > friend this morning). (In reply to Tom Sweeney from comment #22) > Yanpliu, can you verify that this is now working as it should or if the > error still exists and we still need to document. I'm not sure which is > reproducing. (apologies for the two empty comments, Bugzilla is not my > friend this morning). @gnecasov the doc text needs some tweaking. " the them on your host" seems off. @yanpliu I think you are saying that the bug is NOT reproducible now? Is that correct? If so, should we close this as fixed in Current Release and ignore the Release note? (In reply to Tom Sweeney from comment #25) > @yanpliu I think you are saying that the bug is NOT reproducible > now? Is that correct? If so, should we close this as fixed in Current > Release and ignore the Release note? Yes, you are correct. This issue fixed in the Current Release. The release note has been republished: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9/html-single/9.0_release_notes/index |