Bug 1366799

Summary: failed to use host entitlement in containers
Product: Red Hat Enterprise Linux 7 Reporter: Qian Cai <qcai>
Component: subscription-managerAssignee: candlepin-bugs
Status: CLOSED ERRATA QA Contact: John Sefler <jsefler>
Severity: medium Docs Contact:
Priority: urgent    
Version: 7.2CC: aweiteka, bkearney, csnyder, dwalsh, jgalipea, jmolet, miabbott, qcai, redakkan, sct, skallesh
Target Milestone: rcKeywords: Triaged
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-11-03 20:30:51 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Qian Cai 2016-08-12 21:00:49 UTC
Description of problem:
After enable the entitlement in the host, there is no entitlement inside the containers.

# subscription-manager status
+-------------------------------------------+
   System Status Details
+-------------------------------------------+
Overall Status: Current

# subscription-manager repos --list-enabled
+----------------------------------------------------------+
    Available Repositories in /etc/yum.repos.d/redhat.repo
+----------------------------------------------------------+
Repo ID:   rhel-7-server-extras-rpms
Repo Name: Red Hat Enterprise Linux 7 Server - Extras (RPMs)
Repo URL:  https://cdn.redhat.com/content/dist/rhel/server/7/7Server/$basearch/e
           xtras/os
Enabled:   1

Repo ID:   rhel-7-server-eus-rpms
Repo Name: Red Hat Enterprise Linux 7 Server - Extended Update Support (RPMs)
Repo URL:  https://cdn.redhat.com/content/eus/rhel/server/7/$releasever/$basearc
           h/os
Enabled:   1

Repo ID:   rhel-7-server-rpms
Repo Name: Red Hat Enterprise Linux 7 Server (RPMs)
Repo URL:  https://cdn.redhat.com/content/dist/rhel/server/7/$releasever/$basear
           ch/os
Enabled:   1

# docker run -it rhel7 bash
# yum install wget
Loaded plugins: ovl, product-id, search-disabled-repos, subscription-manager
[Errno 2] No such file or directory: '/etc/pki/consumer/cert.pem'
There are no enabled repos.
 Run "yum repolist all" to see the repos you have.
 You can enable repos with yum-config-manager --enable <repo>

Version-Release number of selected component (if applicable):
# rpm -qa | grep docker
docker-common-1.10.3-46.el7.10.x86_64
docker-rhel-push-plugin-1.10.3-46.el7.10.x86_64
docker-selinux-1.10.3-46.el7.10.x86_64
docker-latest-1.12.0-4.el7.x86_64

How reproducible:
always

Comment 3 Daniel Walsh 2016-08-15 12:43:25 UTC
Are you seeing the entitlement in docker-1.10 but not in docker-latest?

Do you see a directory in /run/secrets?

Comment 5 Qian Cai 2016-08-15 15:27:39 UTC
(In reply to Daniel Walsh from comment #3)
> Are you seeing the entitlement in docker-1.10 but not in docker-latest?
No, docker-1.10 has the same issue but I must saw it working before probably docker-1.9.
> 
> Do you see a directory in /run/secrets?
Yes.

# find /run/secrets/
/run/secrets/
/run/secrets/etc-pki-entitlement
/run/secrets/etc-pki-entitlement/1010276342571042975-key.pem
/run/secrets/etc-pki-entitlement/1010276342571042975.pem
/run/secrets/rhel7.repo
/run/secrets/rhsm
/run/secrets/rhsm/ca
/run/secrets/rhsm/ca/redhat-uep.pem
/run/secrets/rhsm/logging.conf
/run/secrets/rhsm/rhsm.conf

Comment 7 Daniel Walsh 2016-08-15 17:37:27 UTC
I think this is more about getting non standard certificates into the container. 
If you volume mount the cert into the container does it work?

Comment 8 Qian Cai 2016-08-15 18:23:36 UTC
# docker run -it -v /etc/pki/:/etc/pki/ rhel7 bash

# yum install gcc
Loaded plugins: ovl, product-id, search-disabled-repos, subscription-manager
(104, 'Connection reset by peer')
There are no enabled repos.
 Run "yum repolist all" to see the repos you have.
 You can enable repos with yum-config-manager --enable <repo>

Comment 9 Daniel Walsh 2016-08-15 18:41:58 UTC
Try with 

docker run --privileged -it -v /etc/pki/:/etc/pki/ rhel7 bash

Comment 10 Qian Cai 2016-08-15 19:10:04 UTC
(In reply to Daniel Walsh from comment #9)
> Try with 
> 
> docker run --privileged -it -v /etc/pki/:/etc/pki/ rhel7 bash
Same error as the one without --privileged .

Comment 11 Daniel Walsh 2016-08-15 19:49:18 UTC
We need to get one of the entitlement guys to look at this and tell us what needs to happen.

Comment 25 Daniel Walsh 2016-09-10 09:26:40 UTC
make sure you have the latest docker-selinux package installed in rhel7.3

docker-1.10.3-53.el7 

Versions before this have issues with SELinux.

Comment 26 Chris Snyder 2016-09-12 21:08:30 UTC
PR with a fix posted as a tracker.

While I've not been able to find the root cause yet for the strange behaviour specifically when running in a container in a vm on CI-OSP (I believe it to be something unique with regards to the networking of that particular set up), I have found as a result of this a change in subman to better enforce the separation between a host and containers running on said host.
The attached PR ensures that our subscription-manager yum plugin does not check for a release value that will not exist from candlepin when run inside a container.

Comment 29 J.C. Molet 2016-09-13 18:40:55 UTC
Still seem to be broken using latest brew candidates for docker and subman:

[root@jmolet-bugtest yum.repos.d]# subscription-manager repos --list-enabled
+----------------------------------------------------------+
    Available Repositories in /etc/yum.repos.d/redhat.repo
+----------------------------------------------------------+

<........ snip for brevity ..........>

Repo ID:   rhel-7-server-rpms
Repo Name: Red Hat Enterprise Linux 7 Server (RPMs)
Repo URL:  https://cdn.redhat.com/content/dist/rhel/server/7/$releasever/$basearch/os
Enabled:   1

<........ snip for brevity ..........>

[root@jmolet-bugtest yum.repos.d]# rpm -qa | grep -e subscription-manager -e python-rhsm
subscription-manager-plugin-container-1.17.15-1.el7.x86_64
python-rhsm-certificates-1.17.9-1.el7.x86_64
subscription-manager-1.17.15-1.el7.x86_64
python-rhsm-1.17.9-1.el7.x86_64

[root@jmolet-bugtest yum.repos.d]# rpm -qa | grep docker
docker-rhel-push-plugin-1.10.3-54.el7.x86_64
docker-1.10.3-54.el7.x86_64
docker-common-1.10.3-54.el7.x86_64
docker-selinux-1.10.3-54.el7.x86_64

[root@jmolet-bugtest yum.repos.d]# systemctl restart rhsmcertd.service docker.service

[root@jmolet-bugtest yum.repos.d]# docker run -it rhel7 bash
[root@fb3c0f9a2da2 /]# yum repolist
Loaded plugins: ovl, product-id, search-disabled-repos, subscription-manager
[Errno 2] No such file or directory: '/etc/pki/consumer/cert.pem'
repolist: 0

[root@fb3c0f9a2da2 /]# find /run/secrets/
/run/secrets/
/run/secrets/rhsm
/run/secrets/rhsm/rhsm.conf
/run/secrets/rhsm/ca
/run/secrets/rhsm/ca/redhat-entitlement-authority.pem
/run/secrets/rhsm/ca/redhat-uep.pem
/run/secrets/rhsm/pluginconf.d
/run/secrets/rhsm/pluginconf.d/container_content.ContainerContentPlugin.conf
/run/secrets/rhsm/logging.conf
/run/secrets/rhel7.repo
/run/secrets/etc-pki-entitlement
/run/secrets/etc-pki-entitlement/3703249394506686825.pem
/run/secrets/etc-pki-entitlement/602520236896205726-key.pem
/run/secrets/etc-pki-entitlement/602520236896205726.pem
/run/secrets/etc-pki-entitlement/3703249394506686825-key.pem

[root@fb3c0f9a2da2 /]# cat /var/log/rhsm/rhsm.log 
2016-09-13 14:36:30,246 [DEBUG] yum:14 @identity.py:131 - Loading consumer info from identity certificates.
2016-09-13 14:36:30,246 [DEBUG] yum:14 @identity.py:146 - Reload of consumer identity cert /etc/pki/consumer/cert.pem raised an exception with msg: [Errno 2] No such file or directory: '/etc/pki/consumer/key.pem'
2016-09-13 14:36:30,272 [INFO] yum:14 @connection.py:778 - Connection built: host=subscription.rhsm.redhat.com port=443 handler=/subscription auth=identity_cert ca_dir=/etc/rhsm-host/ca/ verify=False
2016-09-13 14:36:30,273 [DEBUG] yum:14 @__init__.py:85 - Searching for content of type: yum
2016-09-13 14:36:30,275 [DEBUG] yum:14 @connection.py:475 - Loaded CA certificates from /etc/rhsm-host/ca/: redhat-entitlement-authority.pem, redhat-uep.pem
2016-09-13 14:36:30,275 [DEBUG] yum:14 @connection.py:523 - Making request: GET /subscription/consumers/None/release

Comment 30 Chris Snyder 2016-09-13 19:01:37 UTC
Hey Daniel,

Not sure if you are the right person to ask but as you are fairly active on these docker bugs I figured it was worth a try.

The fix included for this bz (now merged and build for rhel 7.3 snap 4) needs to be included inside the docker containers that come from registry.access.redhat.com. Would you know who to ping to get those rebuilt?


Thanks!

Comment 31 J.C. Molet 2016-09-13 19:08:52 UTC
DOH! I didn't realise this was a client issue as well, after updating the container's subscription manager it works:

[root@jmolet-bugtest rpms]# docker run -v /root/rpms:/root/rpms  -it rhel7 bash
[root@a7274d799613 /]# cd ~/rpms/
[root@a7274d799613 rpms]# ls
python-rhsm-1.17.9-1.el7.x86_64.rpm  python-rhsm-certificates-1.17.9-1.el7.x86_64.rpm  subscription-manager-1.17.15-1.el7.x86_64.rpm  subscription-manager-plugin-container-1.17.15-1.el7.x86_64.rpm
[root@a7274d799613 rpms]# rpm -Uvh ./*.rpm
Preparing...                          ################################# [100%]
Updating / installing...
   1:python-rhsm-certificates-1.17.9-1################################# [ 17%]
   2:python-rhsm-1.17.9-1.el7         ################################# [ 33%]
   3:subscription-manager-1.17.15-1.el################################# [ 50%]
   4:subscription-manager-plugin-conta################################# [ 67%]
Cleaning up / removing...
   5:subscription-manager-1.15.9-15.el################################# [ 83%]
   6:python-rhsm-1.15.4-5.el7         ################################# [100%]
[root@a7274d799613 rpms]# yum repolist
Loaded plugins: ovl, product-id, search-disabled-repos, subscription-manager
<< snip >> 
rhel-7-server-rpms/7Server/x86_64                                              Red Hat Enterprise Linux 7 Server (RPMs)                                                                                      11252
repolist: 23645

[root@a7274d799613 rpms]# yum install wget
Loaded plugins: ovl, product-id, search-disabled-repos, subscription-manager
Resolving Dependencies
--> Running transaction check
---> Package wget.x86_64 0:1.14-10.el7_0.1 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

===================================================================================================================================================================================================================
 Package                                    Arch                                         Version                                                Repository                                                    Size
===================================================================================================================================================================================================================
Installing:
 wget                                       x86_64                                       1.14-10.el7_0.1                                        rhel-7-server-eus-rpms                                       546 k

Transaction Summary
===================================================================================================================================================================================================================
Install  1 Package

Total download size: 546 k
Installed size: 2.0 M
Is this ok [y/d/N]: y
Downloading packages:
warning: /var/cache/yum/x86_64/7Server/rhel-7-server-eus-rpms/packages/wget-1.14-10.el7_0.1.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID fd431d51: NOKEY                   ]  0.0 B/s |    0 B  --:--:-- ETA 
Public key for wget-1.14-10.el7_0.1.x86_64.rpm is not installed
wget-1.14-10.el7_0.1.x86_64.rpm                                                                                                                                                             | 546 kB  00:00:00     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Warning: RPMDB altered outside of yum.
  Installing : wget-1.14-10.el7_0.1.x86_64                                                                                                                                                                     1/1 
  Verifying  : wget-1.14-10.el7_0.1.x86_64                                                                                                                                                                     1/1 

Installed:
  wget.x86_64 0:1.14-10.el7_0.1                                                                                                                                                                                    

Complete!

All works... guess I should wait to see if this makes it into a docker build before setting to VERIFIED?

Comment 32 J.C. Molet 2016-09-13 19:32:54 UTC
Verifying this as the subscription manager component is working and will be included in the standard snapshot release process (snap 4).

Comment 34 errata-xmlrpc 2016-11-03 20:30:51 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2016-2592.html

Comment 35 Red Hat Bugzilla 2023-09-14 03:29:34 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days