RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2216079 - Curl error (77): Problem with the SSL CA cert when using UBI9 image on OpenShift container platform 4
Summary: Curl error (77): Problem with the SSL CA cert when using UBI9 image on OpenSh...
Keywords:
Status: CLOSED DUPLICATE of bug 2203096
Alias: None
Product: Red Hat Enterprise Linux 9
Classification: Red Hat
Component: subscription-manager
Version: 9.2
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: CSI Client Tools Bugs
QA Contact: Red Hat subscription-manager QE Team
URL:
Whiteboard:
: 2226728 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2023-06-20 04:03 UTC by Wei Liu
Modified: 2023-07-26 23:01 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-06-29 07:37:08 UTC
Type: Bug
Target Upstream Version:
Embargoed:
pm-rhel: mirror+


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHELPLAN-160243 0 None None None 2023-06-20 04:04:51 UTC

Description Wei Liu 2023-06-20 04:03:21 UTC
Description of problem:
Curl error (77): Problem with the SSL CA cert when using using UBI9 image on OpenShift container platform 4 

Version-Release number of selected component (if applicable):
OCP 4.13 
registry.access.redhat.com/ubi9/ubi:latest

How reproducible:
Always

Steps to Reproduce:
1. Use the following to create one buildconfig

oc create -f buildconfig.yaml
buildconfig.build.openshift.io/my-csi-bc-s2i created


cat buildconfig.yaml
apiVersion: build.openshift.io/v1
kind: BuildConfig
metadata:
  name: my-csi-bc-s2i
  namespace: ent-test
spec:
  runPolicy: Serial
  source:
    dockerfile: |
      FROM registry.access.redhat.com/ubi9/ubi:latest
      RUN ls -la /etc/pki/entitlement &&\
          rm /etc/rhsm-host &&\
          yum install -y yum-utils 
      RUN  echo rhocp-4.13-for-rhel-9-x86_64-rpms >>repo.txt;  echo rhocp-4.13-for-rhel-9-x86_64-debug-rpms >>repo.txt; 
      RUN cat repo.txt | xargs -I {} subscription-manager repos --enable={}
      RUN cat repo.txt | xargs -I {} repoquery --available --quiet --all --repoid={} >> packages.txt  
      RUN shuf -n 3 packages.txt > installed_pck &&\
          cat installed_pck 
      RUN cat installed_pck | xargs -I {} dnf install -y --setopt=*.module_hotfixes=True --skip-broken {}
  strategy:
    type: Docker
    dockerStrategy:
      volumes:
        - mounts:
            - destinationPath: "/etc/pki/entitlement"
          name: my-csi-shared-secret
          source:
            csi:
              driver: csi.sharedresource.openshift.io
              readOnly: true
              volumeAttributes:
                sharedSecret: my-share 
            type: CSI
  output:
    to:
      kind: "ImageStreamTag"
      name: "sample-custom:latest"

2. Start the buildconfig to create container image
oc start-build my-csi-bc-s2i -F

time="2023-06-20T02:11:26Z" level=info msg="Not using native diff for overlay, this may cause degraded performance for building images: kernel has CONFIG_OVERLAY_FS_REDIRECT_DIR enabled"
I0620 02:11:26.814758       1 defaults.go:112] Defaulting to storage driver "overlay" with options [mountopt=metacopy=on].
Caching blobs under "/var/cache/blobs".

Pulling image registry.access.redhat.com/ubi9/ubi:latest ...
Trying to pull registry.access.redhat.com/ubi9/ubi:latest...
Getting image source signatures
Copying blob sha256:33b9f09cff46d8b03b1185767b94e3881ea4cb8a671dd16ca1403f599fb2ed8f
Copying config sha256:e7236a3e070f267713ad79c451b8628166abc0bc9c855f624619e099ec3faa99
Writing manifest to image destination
Storing signatures
Adding transient rw bind mount for /run/secrets/rhsm
STEP 1/9: FROM registry.access.redhat.com/ubi9/ubi:latest
STEP 2/9: RUN ls -la /etc/pki/entitlement &&    rm /etc/rhsm-host &&    yum install -y yum-utils
total 0
drwxrwxrwt.  3 root root 120 Jun 20 02:11 .
drwxr-xr-x. 10 root root 154 May  3 09:06 ..
drwxr-xr-x.  2 root root  80 Jun 20 02:11 ..2023_06_20_02_11_24.158959106
lrwxrwxrwx.  1 root root  31 Jun 20 02:11 ..data -> ..2023_06_20_02_11_24.158959106
lrwxrwxrwx.  1 root root  26 Jun 20 02:11 entitlement-key.pem -> ..data/entitlement-key.pem
lrwxrwxrwx.  1 root root  22 Jun 20 02:11 entitlement.pem -> ..data/entitlement.pem
Updating Subscription Management repositories.
Unable to read consumer identity
Subscription Manager is operating in container mode.

This system is not registered with an entitlement server. You can use subscription-manager to register.

Red Hat Enterprise Linux 9 for x86_64 - AppStre 0.0  B/s |   0  B     00:00    
Errors during downloading metadata for repository 'rhel-9-for-x86_64-appstream-rpms':
  - Curl error (77): Problem with the SSL CA cert (path? access rights?) for https://cdn.redhat.com/content/dist/rhel9/9/x86_64/appstream/os/repodata/repomd.xml [error setting certificate file: %(ca_cert_dir)sredhat-uep.pem]
Error: Failed to download metadata for repo 'rhel-9-for-x86_64-appstream-rpms': Cannot download repomd.xml: Curl error (77): Problem with the SSL CA cert (path? access rights?) for https://cdn.redhat.com/content/dist/rhel9/9/x86_64/appstream/os/repodata/repomd.xml [error setting certificate file: %(ca_cert_dir)sredhat-uep.pem]
error: build error: building at STEP "RUN ls -la /etc/pki/entitlement &&    rm /etc/rhsm-host &&    yum install -y yum-utils": while running runtime: exit status 1


Actual results:
One error will be shown


Expected results:
No error and the package can be installed successfully

Additional info:
It is OK if try with the UBI8:
[root@ocp4-rhel8-ent-slave-1 entitlement-tests]# oc start-build my-csi-bc-s2i -F
build.build.openshift.io/my-csi-bc-s2i-1 started
time="2023-06-20T03:52:13Z" level=info msg="Not using native diff for overlay, this may cause degraded performance for building images: kernel has CONFIG_OVERLAY_FS_REDIRECT_DIR enabled"
I0620 03:52:13.435155       1 defaults.go:112] Defaulting to storage driver "overlay" with options [mountopt=metacopy=on].
Caching blobs under "/var/cache/blobs".

Pulling image registry.access.redhat.com/ubi8/ubi:latest ...
Trying to pull registry.access.redhat.com/ubi8/ubi:latest...
Getting image source signatures
Copying blob sha256:0fa65fe5c23e8b1745b1f39aa3735f2f3ce77cad9e470bfbb1468cb45a886bbe
Copying config sha256:817f060b4672f886292b297d96d2288dec751013210f35a4c89cd9499866e7a5
Writing manifest to image destination
Storing signatures
Adding transient rw bind mount for /run/secrets/rhsm
STEP 1/9: FROM registry.access.redhat.com/ubi8/ubi:latest
STEP 2/9: RUN ls -la /etc/pki/entitlement &&    ls -la /etc/rhsm/ca &&    rm /etc/rhsm-host &&    yum install -y yum-utils
total 0
drwxrwxrwt.  3 root root 120 Jun 20 03:52 .
drwxr-xr-x. 10 root root 154 May  3 15:08 ..
drwxr-xr-x.  2 root root  80 Jun 20 03:52 ..2023_06_20_03_52_10.110422034
lrwxrwxrwx.  1 root root  31 Jun 20 03:52 ..data -> ..2023_06_20_03_52_10.110422034
lrwxrwxrwx.  1 root root  26 Jun 20 03:52 entitlement-key.pem -> ..data/entitlement-key.pem
lrwxrwxrwx.  1 root root  22 Jun 20 03:52 entitlement.pem -> ..data/entitlement.pem
total 12
drwxr-xr-x. 2 root root   68 May  3 15:07 .
drwxr-xr-x. 6 root root  104 May  3 15:08 ..
-rw-r--r--. 1 root root 2305 Feb 23 06:16 redhat-entitlement-authority.pem
-rw-r--r--. 1 root root 7411 Feb 23 06:16 redhat-uep.pem
Updating Subscription Management repositories.
Unable to read consumer identity

This system is not registered with an entitlement server. You can use subscription-manager to register.

Red Hat Enterprise Linux 8 for x86_64 - AppStre 107 MB/s |  57 MB     00:00    
Red Hat Enterprise Linux 8 for x86_64 - BaseOS  103 MB/s |  61 MB     00:00    
Red Hat Universal Base Image 8 (RPMs) - BaseOS  5.1 MB/s | 839 kB     00:00    
Red Hat Universal Base Image 8 (RPMs) - AppStre  14 MB/s | 3.3 MB     00:00    
Red Hat Universal Base Image 8 (RPMs) - CodeRea 918 kB/s | 106 kB     00:00    
Dependencies resolved.
================================================================================
 Package          Arch   Version            Repository                     Size
================================================================================
Installing:
 yum-utils        noarch 4.0.21-19.el8_8    rhel-8-for-x86_64-baseos-rpms  75 k
Installing dependencies:
 dnf-plugins-core noarch 4.0.21-19.el8_8    rhel-8-for-x86_64-baseos-rpms  75 k

Transaction Summary
================================================================================
Install  2 Packages

Total download size: 150 k
Installed size: 44 k
Downloading Packages:
(1/2): dnf-plugins-core-4.0.21-19.el8_8.noarch. 695 kB/s |  75 kB     00:00    
.....

Comment 1 Pino Toscano 2023-06-26 07:35:05 UTC
The changes done for bug 2108549 (fixed in 9.2, and also in 9.1z as bug 2151829, and 9.0z as bug 2151830) were supposed to fix exactly this situation. AFAICT, that bug was fixed months ago, tested also by OpenShift QE (Lu Liu), and I'd be surprised if it was still broken and nobody had noticed for months.

So:
- what is the exact version of the UBI 9 used?
- what is the version of subscription-manager in the UBI 9 used?
- please check your steps with what was done in bug 2108549 -- anything different?

Comment 2 Wei Liu 2023-06-26 08:02:56 UTC
- what is the exact version of the UBI 9 used?

UBI Image:registry.access.redhat.com/ubi9/ubi:latest 

- what is the version of subscription-manager in the UBI 9 used?

sh-5.1# cat /etc/redhat-release 
Red Hat Enterprise Linux release 9.2 (Plow)
sh-5.1# 
sh-5.1# subscription-manager version
server type: This system is currently not registered.
subscription management server: 4.2.15-1
subscription management rules: 5.43
subscription-manager: 1.29.33.1-1.el9_2

- please check your steps with what was done in bug 2108549 -- anything different?

The difference with bug 2108549 is as following,

In the bug 2108549, we use the ubi9 image directly to run a container.
But in this bug, we firstly build one container image which is based on ubi9 and try to install some packages, But failed with the cert error in the image build process...

I know that the same error has been fixed in the bug 2108549, and also have a try to run a container with ubi9 directly, there is no problem with the cert.
But when we try to use the buildconfig to build an image, it will show the error again...

Comment 5 Wei Liu 2023-06-27 05:10:21 UTC
Hi Pino,

Seems the workaround works well and I agree to close this bug and go on tracking it by bug 2203096.

[root@ocp4-rhel8-ent-slave-1 entitlement-tests]# oc create -f buildconfig.yaml
buildconfig.build.openshift.io/my-csi-bc-s2i created
[root@ocp4-rhel8-ent-slave-1 entitlement-tests]# oc start-build my-csi-bc-s2i -F
build.build.openshift.io/my-csi-bc-s2i-1 started
time="2023-06-27T05:06:36Z" level=info msg="Not using native diff for overlay, this may cause degraded performance for building images: kernel has CONFIG_OVERLAY_FS_REDIRECT_DIR enabled"
I0627 05:06:36.376633       1 defaults.go:112] Defaulting to storage driver "overlay" with options [mountopt=metacopy=on].
Caching blobs under "/var/cache/blobs".

Pulling image registry.access.redhat.com/ubi9/ubi:latest ...
Trying to pull registry.access.redhat.com/ubi9/ubi:latest...
Getting image source signatures
Copying blob sha256:7b3dd25bf011f6e84d1eaf4cce367d6d7c3d1d82385a65ebb394b5bf096f8d7a
Copying config sha256:663a35613bf10445ee2fd4af90024ccb83aecf3985b7e98c9e9a87f0f3131865
Writing manifest to image destination
Storing signatures
Adding transient rw bind mount for /run/secrets/rhsm
STEP 1/9: FROM registry.access.redhat.com/ubi9/ubi:latest
STEP 2/9: RUN ls -la /etc/pki/entitlement &&    ls -la /etc/rhsm/ca &&    rm /etc/rhsm-host &&    env SMDEV_CONTAINER_OFF=1  yum install -y yum-utils
total 0
drwxrwxrwt.  3 root root 120 Jun 27 05:06 .
drwxr-xr-x. 10 root root 154 Jun 15 01:44 ..
drwxr-xr-x.  2 root root  80 Jun 27 05:06 ..2023_06_27_05_06_27.1510652602
lrwxrwxrwx.  1 root root  32 Jun 27 05:06 ..data -> ..2023_06_27_05_06_27.1510652602
lrwxrwxrwx.  1 root root  26 Jun 27 05:06 entitlement-key.pem -> ..data/entitlement-key.pem
lrwxrwxrwx.  1 root root  22 Jun 27 05:06 entitlement.pem -> ..data/entitlement.pem
total 12
drwxr-xr-x. 2 root root   68 Jun 15 01:44 .
drwxr-xr-x. 6 root root   84 Jun 15 01:44 ..
-rw-r--r--. 1 root root 2305 Jun 23  2022 redhat-entitlement-authority.pem
-rw-r--r--. 1 root root 7411 Jun 23  2022 redhat-uep.pem
Updating Subscription Management repositories.
Unable to read consumer identity

This system is not registered with an entitlement server. You can use subscription-manager to register.

Red Hat Enterprise Linux 9 for x86_64 - AppStre  13 MB/s |  22 MB     00:01    
Red Hat Enterprise Linux 9 for x86_64 - BaseOS   23 MB/s |  13 MB     00:00    
Red Hat Universal Base Image 9 (RPMs) - BaseOS  1.6 MB/s | 580 kB     00:00    
Red Hat Universal Base Image 9 (RPMs) - AppStre 8.2 MB/s | 1.9 MB     00:00    
Red Hat Universal Base Image 9 (RPMs) - CodeRea 845 kB/s | 195 kB     00:00    
Dependencies resolved.
================================================================================
 Package           Arch    Version         Repository                      Size
================================================================================
Installing:
 yum-utils         noarch  4.3.0-5.el9_2   rhel-9-for-x86_64-baseos-rpms   45 k
Installing dependencies:
 dnf-plugins-core  noarch  4.3.0-5.el9_2   rhel-9-for-x86_64-baseos-rpms   42 k
....

Comment 6 Pino Toscano 2023-06-29 07:37:08 UTC
Thanks! Marking as duplicate then.

*** This bug has been marked as a duplicate of bug 2203096 ***

Comment 7 Wei Liu 2023-07-26 23:01:36 UTC
*** Bug 2226728 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.