This bug has been migrated to another issue tracking site. It has been closed here and may no longer be being monitored.

If you would like to get updates for this issue, or to participate in it, you may do so at Red Hat Issue Tracker .
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2214451 - Running inside a container where rhsm.conf is missing on RHCOS, repo_ca_cert gets set to a bogus value: /etc/rhsm-host-host/ca/redhat-uep.pem
Summary: Running inside a container where rhsm.conf is missing on RHCOS, repo_ca_cert ...
Keywords:
Status: CLOSED MIGRATED
Alias: None
Product: Red Hat Enterprise Linux 9
Classification: Red Hat
Component: librhsm
Version: 9.2
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: rc
: ---
Assignee: Packaging Maintenance Team
QA Contact: swm-qe
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2023-06-13 03:01 UTC by HuijingHei
Modified: 2024-01-01 09:19 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-09-02 00:58:26 UTC
Type: Bug
Target Upstream Version:
Embargoed:
pm-rhel: mirror+

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker   RHEL-1451 0 None Migrated None 2024-01-01 09:19:34 UTC
Red Hat Issue Tracker RHELPLAN-159685 0 None None None 2023-06-13 03:07:29 UTC

Description HuijingHei 2023-06-13 03:01:48 UTC 
Description of problem:

The issue is for entitlement build on OCP, and the workaround is to remove `/etc/rhsm-host`, see https://docs.openshift.com/container-platform/4.13/cicd/builds/running-entitled-builds.html#builds-running-entitled-builds-with-sharedsecret-objects_running-entitled-builds

For rhel-coreos base image we ship `subscription-manager-rhsm-certificates` (but not subscription-manager), if running in container the config file will be set by default `/etc/rhsm-host/rhsm.conf`(which does not exist), then we get the repo ca cert file is `/etc/rhsm-host/ca/redhat-uep.pem` (this file is existed).

According to code(https://github.com/rpm-software-management/librhsm/blob/5e0674cf389f14174208641ec411ba7be448d5e3/rhsm/rhsm-context.c#L542), check conf is under `/etc/rhsm-host`, will update ca cert dir from `/etc/rhsm` to `/etc/rhsm-host`, and finally get `/etc/rhsm-host-host/ca/redhat-uep.pem`, the path is not correct and fail.

Before replace:
conf=/etc/rhsm-host/rhsm.conf, ca=/etc/rhsm-host/ca, repo=/etc/rhsm-host/ca/redhat-uep.pem
After replace:
conf=/etc/rhsm-host/rhsm.conf, ca=/etc/rhsm-host-host/ca, repo=/etc/rhsm-host-host/ca/redhat-uep.pem


Version-Release number of selected component (if applicable):
RHEL 9.2

How reproducible:
100%

Steps to Reproduce:
See https://issues.redhat.com/browse/OCPBUGS-11181?focusedId=22365428&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-22365428

Actual results:
bash-5.1# rpm-ostree install libreswan
error: Updating rpm-md repo 'rhel-9-for-x86_64-baseos-rpms': cannot update repo 'rhel-9-for-x86_64-baseos-rpms': Cannot download repomd.xml: Curl error (77): Problem with the SSL CA cert (path? access rights?) for https://cdn.redhat.com/content/dist/rhel9/9/x86_64/baseos/os/repodata/repomd.xml [error setting certificate file: /etc/rhsm-host-host/ca/redhat-uep.pem]

Expected results:
Install libreswan successfully.

Additional info:
- For latest ubi9 container image, the issue is gone as fixed with BZ#2108549 (from subscription-manager side).
See Derrick's comment https://issues.redhat.com/browse/OCPBUGS-11181?focusedId=22181117&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-22181117

- For ubi8 container image, the issue is existed.
Comment 1 HuijingHei 2023-06-13 03:37:57 UTC 
Additional info:

Same issue https://github.com/rpm-software-management/librhsm/issues/9

Simple workaround in https://github.com/rpm-software-management/librhsm/pull/10:
- Print debug log and skip replacing ca certificate dir if config file not found

We do not have the final solution for the issue now, it might be working for rhcos base image, but I have concern if it will make regression for BZ#2108549, and if it works for ubi8?
Comment 2 HuijingHei 2023-06-27 12:57:20 UTC 
I build the scratch build https://kojihub.stream.rdu2.redhat.com/kojifiles/work/tasks/7640/2407640/librhsm-0.0.3-8.el9.x86_64.rpm with https://github.com/rpm-software-management/librhsm/pull/10, which works on rhcos9 image:

$ oc logs bc-coreos-rhsm-5-build
...
Storing signatures
Adding transient rw bind mount for /run/secrets/rhsm
STEP 1/5: FROM registry.ci.openshift.org/coreos/hhei-rhcos-test:rhsm
STEP 2/5: RUN ls -la /etc/pki/entitlement/
total 0
drwxrwxrwt. 3 root root 120 Jun 27 12:45 .
drwxr-xr-x. 1 root root  25 Jun 27 12:46 ..
drwxr-xr-x. 2 root root  80 Jun 27 12:45 ..2023_06_27_12_45_51.88737879
lrwxrwxrwx. 1 root root  30 Jun 27 12:45 ..data -> ..2023_06_27_12_45_51.88737879
lrwxrwxrwx. 1 root root  26 Jun 27 12:45 entitlement-key.pem -> ..data/entitlement-key.pem
lrwxrwxrwx. 1 root root  22 Jun 27 12:45 entitlement.pem -> ..data/entitlement.pem
--> cf50b34652f
STEP 3/5: RUN rpm-ostree install libreswan
Enabled rpm-md repositories: rhel-9-for-x86_64-baseos-rpms rhel-9-for-x86_64-appstream-rpms
Updating metadata for 'rhel-9-for-x86_64-baseos-rpms'...done
Updating metadata for 'rhel-9-for-x86_64-appstream-rpms'...done
Importing rpm-md...done
rpm-md repo 'rhel-9-for-x86_64-baseos-rpms'; generated: 2023-06-26T13:51:40Z solvables: 4570
rpm-md repo 'rhel-9-for-x86_64-appstream-rpms'; generated: 2023-06-22T18:53:12Z solvables: 14255
Resolving dependencies...done
Will download: 9 packages (3.8?MB)
Downloading from 'rhel-9-for-x86_64-appstream-rpms'...done
Installing 9 packages:
...
Installing: libreswan-4.9-4.el9_2.x86_64 (rhel-9-for-x86_64-appstream-rpms)
Comment 3 Jaroslav Mracek 2023-07-25 07:55:04 UTC 
I would like to ask you for verification of the issue. The bug is reported for RHEL9, but in #Comment0 there is written that `For latest ubi9 container image, the issue is gone`. It sounds to me like the issue is fixed. Even if we deliver the patch to RHEL9 it will not fix old RHEL9 images and it cannot fix RHEL8 images. I would like to ask you for detailed clarification why we need to deliver the patch?
Comment 4 HuijingHei 2023-07-25 08:19:35 UTC 
(In reply to Jaroslav Mracek from comment #3)
> I would like to ask you for verification of the issue. The bug is reported
> for RHEL9, but in #Comment0 there is written that `For latest ubi9 container
> image, the issue is gone`. It sounds to me like the issue is fixed. Even if
> we deliver the patch to RHEL9 it will not fix old RHEL9 images and it cannot
> fix RHEL8 images. I would like to ask you for detailed clarification why we
> need to deliver the patch?

This is a little complicated, the patch is to fix coreos image based on RHEL9 which ships subscription-manager-rhsm-certificates (but not subscription-manager), and ubi9 container image include full subscription-manager.

When this bug was created, the ubi9 container worked well (fixed by BZ#2108549), now it does not work well (tracked by BZ#2216079, and dup to BZ#2203096).
Comment 5 HuijingHei 2023-07-25 08:45:36 UTC 
In summary, rhcos9 container image has the issue is because of none rhsm.conf which is included in subscription-manager(but rhcos image only includes subscription-manager-rhsm-certificates).
But I have no idea about the current problem with the ubi9 container which using subscription-manager, maybe someone from subscription has more context?
Comment 6 Pino Toscano 2023-07-25 11:43:19 UTC 
(In reply to HuijingHei from comment #4)
> When this bug was created, the ubi9 container worked well (fixed by
> BZ#2108549), now it does not work well (tracked by BZ#2216079, and dup to
> BZ#2203096).

(In reply to HuijingHei from comment #5)
> But I have no idea about the current problem with the ubi9 container which
> using subscription-manager, maybe someone from subscription has more context?

Not sure what do you mean with
- "does not work well"
- "problem with the ubi9 container which using subscription-manager"
you please explain a bit more in detail what problematic situation do you see with subscription-manager?

Please specify what is the exact environment, and what is the wanted goal.
Comment 7 HuijingHei 2023-07-25 12:33:27 UTC 
> (In reply to HuijingHei from comment #5)
> > But I have no idea about the current problem with the ubi9 container which
> > using subscription-manager, maybe someone from subscription has more context?
> 
> Not sure what do you mean with
> - "does not work well"

Sorry for the confusion, this means for ubi9 container image, `running entitled builds using SharedSecret objects` does not work as expected, see BZ#2216079

Could you help to review whether the subscription-manager will be affected by https://github.com/rpm-software-management/librhsm/pull/10 ? Just want to make sure the change will not make regression for entitled builds for ubi9. Thanks!
Comment 8 Pino Toscano 2023-07-25 12:53:26 UTC 
(In reply to HuijingHei from comment #7)
> Could you help to review whether the subscription-manager will be affected
> by https://github.com/rpm-software-management/librhsm/pull/10 ? Just want to
> make sure the change will not make regression for entitled builds for ubi9.

TTBOMK, subscription-manager does not use that code at all.
Comment 9 Pino Toscano 2023-07-25 12:56:06 UTC 
FYI:

> The issue is for entitlement build on OCP, and the workaround is to remove `/etc/rhsm-host`

In case your goal is to make sure that, no matter whether there are secrets available either when running a container or when doing a container build, it is possible to run subscription-manager (e.g. to register a container), then removing that symlink hopefully will be the official solution for it. This way, subscription-manager will never see the provided secrets.
Comment 10 HuijingHei 2023-07-25 14:19:45 UTC 
> subscription-manager does not use that code at all.

Thanks for the confirmation, the patch https://github.com/rpm-software-management/librhsm/pull/10 is to fix coreos image based on RHEL9 for entitled builds, and will not make any regression (for entitled builds) for ubi container.

(In reply to Pino Toscano from comment #9)
> FYI:
> 
> > The issue is for entitlement build on OCP, and the workaround is to remove `/etc/rhsm-host`
> 
> In case your goal is to make sure that, no matter whether there are secrets
> available either when running a container or when doing a container build,
> it is possible to run subscription-manager (e.g. to register a container),
> then removing that symlink hopefully will be the official solution for it.
> This way, subscription-manager will never see the provided secrets.

Thanks Pino for the info, the problem is coreos image based on RHEL9/8 which only ships subscription-manager-rhsm-certificates (but not subscription-manager), see https://github.com/rpm-software-management/librhsm/pull/10#discussion_r1223092580


One thing I want to confirm is that, by default the conf path is set to /etc/rhsm-host/rhsm.conf, but it does not exist on rhcos image, in the fixed PR just add checking whether rhsm.conf is existed when replace the ca cert dir and repo dir, if no, skip; if yes, replace.

Maybe it is more safe to check whether ca cert dir (or repo dir) is already under /etc/rhsm-host/ instead of check rhsm.conf existed, if yes, skip; if no, replace.

Before replace:
conf=/etc/rhsm-host/rhsm.conf, ca=/etc/rhsm-host/ca, repo=/etc/rhsm-host/ca/redhat-uep.pem
After replace:
conf=/etc/rhsm-host/rhsm.conf, ca=/etc/rhsm-host-host/ca, repo=/etc/rhsm-host-host/ca/redhat-uep.pem
Comment 11 HuijingHei 2023-07-25 14:25:08 UTC 
> by default the conf path is set to /etc/rhsm-host/rhsm.conf

when running rhcos container on OCP, by default the conf path is set to /etc/rhsm-host/rhsm.conf,
Comment 12 HuijingHei 2023-08-10 09:57:19 UTC 
Hi Jaroslav, is there any updates for this? Thanks!
Comment 13 RHEL Program Management 2023-08-18 07:27:38 UTC 
Issue migration from Bugzilla to Jira is in process at this time. This will be the last message in Jira copied from the Bugzilla bug.
Comment 14 RHEL Program Management 2023-09-02 00:58:26 UTC 
This BZ has been automatically migrated to the issues.redhat.com Red Hat Issue Tracker. All future work related to this report will be managed there.

Due to differences in account names between systems, some fields were not replicated.  Be sure to add yourself to Jira issue's "Watchers" field to continue receiving updates and add others to the "Need Info From" field to continue requesting information.

To find the migrated issue, look in the "Links" section for a direct link to the new issue location. The issue key will have an icon of 2 footprints next to it, and begin with "RHEL-" followed by an integer.  You can also find this issue by visiting https://issues.redhat.com/issues/?jql= and searching the "Bugzilla Bug" field for this BZ's number, e.g. a search like:

"Bugzilla Bug" = 1234567

In the event you have trouble locating or viewing this issue, you can file an issue by sending mail to rh-issues.
Comment 15 Red Hat Bugzilla 2024-01-01 04:25:10 UTC 
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days
Note You need to log in before you can comment on or make changes to this bug.