Bug 1461546 - ppc64le: unable to use host entitlement in containers
ppc64le: unable to use host entitlement in containers
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: releng (Show other bugs)
7.4
ppc64le Linux
high Severity high
: rc
: ---
Assigned To: Lubos Kocman
Release Test Team
:
Depends On:
Blocks: 1428576
  Show dependency treegraph
 
Reported: 2017-06-14 14:29 EDT by CAI Qian
Modified: 2018-03-15 20:34 EDT (History)
15 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1468271 (view as bug list)
Environment:
Last Closed: 2017-08-01 13:41:57 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description CAI Qian 2017-06-14 14:29:55 EDT
Description of problem:
# subscription-manager repos --list-enabled
+----------------------------------------------------------+
    Available Repositories in /etc/yum.repos.d/redhat.repo
+----------------------------------------------------------+
Repo ID:   rhel-7-for-power-le-rpms
Repo Name: Red Hat Enterprise Linux 7 for IBM Power LE (RPMs)
Repo URL:  https://cdn.stage.redhat.com/content/dist/rhel/power-le/7/$releasever
           /$basearch/os
Enabled:   1

# docker run -it brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhel7-ppc64le bash
# yum repolist
Loaded plugins: ovl, product-id, search-disabled-repos, subscription-manager
repolist: 0

# find /run/secrets/
/run/secrets/
/run/secrets/etc-pki-entitlement
/run/secrets/etc-pki-entitlement/5204424257941389053-key.pem
/run/secrets/etc-pki-entitlement/5204424257941389053.pem
/run/secrets/rhel7.repo
/run/secrets/rhsm
/run/secrets/rhsm/ca
/run/secrets/rhsm/ca/redhat-entitlement-authority.pem
/run/secrets/rhsm/ca/redhat-uep.pem
/run/secrets/rhsm/logging.conf
/run/secrets/rhsm/pluginconf.d
/run/secrets/rhsm/pluginconf.d/container_content.ContainerContentPlugin.conf
/run/secrets/rhsm/rhsm.conf

# vi /run/secrets/rhel7.repo
...
[rhel-7-for-power-le-rpms]
metadata_expire = 86400
sslclientcert = /etc/pki/entitlement/5204424257941389053.pem
baseurl = https://cdn.stage.redhat.com/content/dist/rhel/power-le/7/$releasever/$basearch/os
ui_repoid_vars = releasever basearch
sslverify = 1
name = Red Hat Enterprise Linux 7 for IBM Power LE (RPMs)
sslclientkey = /etc/pki/entitlement/5204424257941389053-key.pem
gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release
enabled = 1
sslcacert = /etc/rhsm/ca/redhat-uep.pem
gpgcheck = 1
...

# find /etc/pki/  
/etc/pki/
/etc/pki/product-default
/etc/pki/rpm-gpg
/etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-beta
/etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-legacy-former
/etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-legacy-release
/etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-legacy-rhx
/etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release
/etc/pki/ca-trust
/etc/pki/ca-trust/README
/etc/pki/ca-trust/ca-legacy.conf
/etc/pki/ca-trust/extracted
/etc/pki/ca-trust/extracted/README
/etc/pki/ca-trust/extracted/java
/etc/pki/ca-trust/extracted/java/README
/etc/pki/ca-trust/extracted/java/cacerts
/etc/pki/ca-trust/extracted/openssl
/etc/pki/ca-trust/extracted/openssl/README
/etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt
/etc/pki/ca-trust/extracted/pem
/etc/pki/ca-trust/extracted/pem/README
/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
/etc/pki/ca-trust/extracted/pem/email-ca-bundle.pem
/etc/pki/ca-trust/extracted/pem/objsign-ca-bundle.pem
/etc/pki/ca-trust/source
/etc/pki/ca-trust/source/README
/etc/pki/ca-trust/source/anchors
/etc/pki/ca-trust/source/blacklist
/etc/pki/ca-trust/source/ca-bundle.legacy.crt
/etc/pki/java
/etc/pki/java/cacerts
/etc/pki/tls
/etc/pki/tls/cert.pem
/etc/pki/tls/certs
/etc/pki/tls/certs/ca-bundle.crt
/etc/pki/tls/certs/ca-bundle.trust.crt
/etc/pki/tls/misc
/etc/pki/tls/openssl.cnf
/etc/pki/tls/private
/etc/pki/nss-legacy
/etc/pki/nss-legacy/nss-rhel7.config
/etc/pki/nssdb
/etc/pki/nssdb/cert8.db
/etc/pki/nssdb/cert9.db
/etc/pki/nssdb/key3.db
/etc/pki/nssdb/key4.db
/etc/pki/nssdb/pkcs11.txt
/etc/pki/nssdb/secmod.db
/etc/pki/consumer
/etc/pki/entitlement
/etc/pki/entitlement-host
/etc/pki/product

# cat /var/log/rhsm/rhsm.log 
2017-06-14 18:23:08,313 [INFO] yum:16:MainThread @connection.py:819 - Connection built: host=subscription.rhn.stage.redhat.com port=443 handler=/subscription auth=identity_cert ca_dir=/etc/rhsm-host/ca/ insecure=False
2017-06-14 18:23:08,315 [INFO] yum:16:MainThread @repolib.py:329 - repos updated: Repo updates

Total repo updates: 0
Updated
    <NONE>
Added (new)
    <NONE>
Deleted
    <NONE>

Version-Release number of selected component (if applicable):
Pegas-7.4-20170425.0
subscription-manager-plugin-container-1.19.9-1.el7.ppc64le
subscription-manager-1.19.9-1.el7.ppc64le
docker-1.12.6-33.1.git3a6eaeb.el7.ppc64le
python-rhsm-certificates-1.19.5-1.el7.ppc64le
python-rhsm-1.19.5-1.el7.ppc64le

How reproducible:
always

Actual results:
yum repolist inside container shows 0.

Expected results:
yum repolist inside container shows non-zero.
Comment 1 CAI Qian 2017-06-14 15:17:00 EDT
Same problem for RHEL 7.4 snapshot 3. Confirmed that subscription-manager inside the container caused issue. Bisecting indicating the following version introduced the problem.

subscription-manager-1.19.13-1.el7.ppc64le

Likely, one of the following commits is the culprit.

* Mon May 15 2017 Kevin Howell <khowell@redhat.com> 1.19.13-1
- 1447722: use socket.getaddrinfo() to mimic hostname -f cmd
  (jhnidek@redhat.com)
- 1427069: Add secondary file to determine external repo file changes
  (wpoteat@redhat.com)
- 1444453: set bin scripts file encoding to utf-8 (khowell@redhat.com)
- 1445204: Update timestamp during intitial cert check. (jhnidek@redhat.com)
- 1444453: Set default encoding for gui to UTF-8 (khowell@redhat.com)
Comment 2 Yaakov Selkowitz 2017-06-15 14:36:49 EDT
CAI Qian, are you able to do the same test on x86_64?
Comment 3 CAI Qian 2017-06-15 14:42:03 EDT
x86_64 works fine.
Comment 4 Josh Boyer 2017-06-15 16:40:53 EDT
Neil and Yaakov,

Can you work with Kevin to see which change introduced this?

CAI, to confirm: you did the bisection on the host machine using the same container throughout, correct?
Comment 5 CAI Qian 2017-06-15 16:43:36 EDT
(In reply to Josh Boyer from comment #4)
> Neil and Yaakov,
> 
> Can you work with Kevin to see which change introduced this?
> 
> CAI, to confirm: you did the bisection on the host machine using the same
> container throughout, correct?

Yes.
Comment 6 Yaakov Selkowitz 2017-06-16 03:21:25 EDT
Adding Kevin Howell.  Please note that container support is a new deliverable for ppc64le in 7.4.
Comment 7 Kevin Howell 2017-06-16 11:17:17 EDT
So notice that in comment 0, `find /etc/pki/` shows nothing in /etc/pki/product or /etc/pki/product-default . Without a product certificate, the repository will be filtered out from the container.

The image should have had the 279.pem baked in.

I tried the following to show how the container behaves when the product cert is present.

Reassigning to releng accordingly.

[root@ibm-p8-01-lp6 ~]# docker run --rm -ti -v /etc/pki/product/279.pem:/etc/pki/product-default/279.pem brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhel7-ppc64le /bin/bash
[root@2a507b5a198f /]# yum repolist         
Loaded plugins: ovl, product-id, search-disabled-repos, subscription-manager
rhel-7-for-power-le-rpms                                                                                                                                                                                           
(1/3): rhel-7-for-power-le-rpms/7Server/ppc64le/group                                                                                                                                                              
(2/3): rhel-7-for-power-le-rpms/7Server/ppc64le/updateinfo                                                                                                                                                         
(3/3): rhel-7-for-power-le-rpms/7Server/ppc64le/primary_db                                                                                      6% [========                                                       (3/3): rhel-7-for-power-le-rpms/7Server/ppc64le/primary_db                                                                                      7% [==========                                                     (3/3): rhel-7-for-power-le-rpms/7Server/ppc64le/primary_db                                                                                      9% [============-                                                  (3/3): rhel-7-for-power-le-rpms/7Server/ppc64le/primary_db                                                                                                                                  |  29 MB  00:00:20     
repo id                                                                                          repo name                                                                                                   status
rhel-7-for-power-le-rpms/7Server/ppc64le                                                         Red Hat Enterprise Linux 7 for IBM Power LE (RPMs)                                                          12183
repolist: 12183
Comment 8 John Sefler 2017-06-16 11:42:57 EDT
FYI: The absent product certs provided by the redhat-release-* packages for non-x86_64 arches for Snap1, Snap2, was reported in...

Bug 1455820 - productid files are missing except of x86_64/Server and x86_64/Workstation
Comment 9 Lubos Kocman 2017-06-21 13:39:14 EDT
Fixed the actual issue here
https://code.engineering.redhat.com/gerrit/#/c/109704/

lkocman@rcm-dev:~/redhat-release/redhat-release-server$ /mnt/redhat/scripts/rel-eng/utility/rhel_common/mk-redhat-release-productid --family htb --version 7.4 --source-dir /home/brq/lkocman/rcm-metadata/product_ids/rhel-7.4-htb/
Copying /home/brq/lkocman/rcm-metadata/product_ids/rhel-7.4-htb/Server-x86_64-b65a0155040c-230.pem -> /tmp/redhat-release-productids-7.4-htbFz9gvL/redhat-release-productids-7.4-htb/x86_64/230.pem
Copying /home/brq/lkocman/rcm-metadata/product_ids/rhel-7.4-htb/Server-Server-ppc64le-0d5f7748f9eb-279.pem -> /tmp/redhat-release-productids-7.4-htbFz9gvL/redhat-release-productids-7.4-htb/ppc64le/279.pem
Copying /home/brq/lkocman/rcm-metadata/product_ids/rhel-7.4-htb/Server-Server-s390x-d46b6382a134-72.pem -> /tmp/redhat-release-productids-7.4-htbFz9gvL/redhat-release-productids-7.4-htb/s390x/72.pem
Copying /home/brq/lkocman/rcm-metadata/product_ids/rhel-7.4-htb/Server-Server-aarch64-90a6503223cd-294.pem -> /tmp/redhat-release-productids-7.4-htbFz9gvL/redhat-release-productids-7.4-htb/aarch64/294.pem
Created /home/brq/lkocman/redhat-release/redhat-release-server/redhat-release-productids-7.4-htb.tar.gz

Please run 'rhpkg upload redhat-release-productids-7.4-htb.tar.gz'
Note: Please make sure that 'sources' file contains only one file with product certificates (redhat-release-productids*). If there are multiple ones, keep only the newest one. (See: https://pagure.io/rpkg/issue/204)

Creation seems to work as expected now

I did rebuild all builds due Bug 1380694.

redhat-release-client-7.4-19.el7  (beta certs)
redhat-release-workstation-7.4-17.el7  (beta/htb certs)
redhat-release-server-7.4-17.el7 (beta/htb certs)
redhat-release-computenode-7.4-15.el7 (beta certs)
Comment 11 Jan Stodola 2017-06-27 04:01:14 EDT
CAI Qian, could you please check if this is working fine in Snapshot 5 (RHEL-7.4-20170621.0) ?
Thank you.
Comment 12 CAI Qian 2017-06-27 08:39:35 EDT
Someone needs to rebuild the power base image to include redhat-release-server-7.4-17.el7. I manually upgrade it and everything works fine.
Comment 13 Wei Shi 2017-06-28 01:50:18 EDT
I cannot see /etc/pki/product-default/69.pem in redhat-release-server-7.4-17.el7.x86_64
Is this what we expected?
Comment 16 errata-xmlrpc 2017-08-01 13:41:57 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:1850

Note You need to log in before you can comment on or make changes to this bug.