Bug 1330852

Summary:	Cannot use 'yum install' in dockerfile for docker build when using rhel7 image
Product:	OpenShift Container Platform	Reporter:	XiuJuan Wang <xiuwang>
Component:	Build	Assignee:	Ben Parees <bparees>
Status:	CLOSED DUPLICATE	QA Contact:	Wenjing Zheng <wzheng>
Severity:	high	Docs Contact:
Priority:	high
Version:	3.2.0	CC:	abenaiss, aos-bugs, bcook, bleanhar, cmilsted, haowang, lsantill, lxia, sgraf, wsun, wzheng, xiuwang
Target Milestone:	---	Keywords:	Regression, Reopened
Target Release:	---	Flags:	xiuwang: needinfo- xiuwang: needinfo- xiuwang: needinfo-
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2018-04-25 18:48:22 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:

Description XiuJuan Wang 2016-04-27 06:59:52 UTC

Description of problem:
After doing subscription management in all nodes, we could use 'yum install' for dockerbuild.
But now, after subscription in node, 'yum' operation can't be used in node,met the [ERROR].We need follow https://access.redhat.com/solutions/69319 to re-register.
#subscription-manager unregister
#subscription-manager clean
#subscription-manager register
#subscription-manager attach --auto
#yum-config-manager --disable rhel-7-server-nfv-rpms

However container still can't inherit the subscription management ability from node. Still met the below [ERROR].

ERROR:
-----------snip-------------
Step 2 : RUN yum-config-manager --disable rhel-7-server-nfv-rpms && INSTALL_PKGS="nss_wrapper java-1.8.0-openjdk" && yum install -y $INSTALL_PKGS && RUN rpm -V $INSTALL_PKGS
---> Running in dd31c6d7eb46
Loaded plugins: ovl, product-id
Loaded plugins: ovl, product-id, search-disabled-repos, subscription-manager
https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/nfv/os/repodata/repomd.xml: [Errno 14] HTTPS Error 403 - Forbidden
Trying other mirror.
To address this issue please refer to the below knowledge base article

https://access.redhat.com/solutions/69319

If above article doesn't help to resolve this issue please open a ticket with Red Hat Support.

One of the configured repositories failed (Red Hat Enterprise Linux for Real Time for NFV (RHEL 7 Server) (RPMs)),
and yum doesn't have enough cached data to continue. At this point the only
safe thing yum can do is fail. There are a few ways to work "fix" this:

1. Contact the upstream for the repository and get them to fix the problem.

2. Reconfigure the baseurl/etc. for the repository, to point to a working
upstream. This is most often useful if you are using a newer
distribution release than is supported by the repository (and the
packages for the previous distribution release still work).

3. Disable the repository, so yum won't use it by default. Yum will then
just ignore the repository until you permanently enable it again or use
--enablerepo for temporary usage:

yum-config-manager --disable rhel-7-server-nfv-rpms

4. Configure the failing repository to be skipped, if it is unavailable.
Note that yum will try to contact the repo. when it runs most commands,
so will have to try and fail each time (and thus. yum will be be much
slower). If it is a very temporary problem though, this is often a nice
compromise:

yum-config-manager --save --setopt=rhel-7-server-nfv-rpms.skip_if_unavailable=true

failure: repodata/repomd.xml from rhel-7-server-nfv-rpms: [Errno 256] No more mirrors to try.
https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/nfv/os/repodata/repomd.xml: [Errno 14] HTTPS Error 403 - Forbidden
F0427 05:41:28.863625 1 builder.go:204] Error: build error: The command '/bin/sh -c yum-config-manager --disable rhel-7-server-nfv-rpms && INSTALL_PKGS="nss_wrapper java-1.8.0-openjdk" && yum install -y $INSTALL_PKGS && RUN rpm -V $INSTALL_PKGS' returned a non-zero code: 1

Version-Release number of selected component (if applicable):
devenv-rhel7_4036

How reproducible:
always

Steps to Reproduce:
1.Do docker build with 'yum install' operation.
2.
3.

Actual results:

Expected results:
Could install rpms during docker build.

Additional info:
Here is my Dockerfile

# The FROM will be replaced when building in OpenShift
FROM openshift/base-rhel7

# Install headless Java
USER root

RUN yum-config-manager --disable rhel-7-server-nfv-rpms && \
INSTALL_PKGS="nss_wrapper java-1.8.0-openjdk" && \
yum install -y $INSTALL_PKGS && \
RUN rpm -V $INSTALL_PKGS
RUN yum clean all
mkdir -p /opt/app-root/jenkins && \
chown -R 1001:0 /opt/app-root/jenkins && \
chmod -R g+w /opt/app-root/jenkins

# Copy the entrypoint
COPY contrib/openshift/* /opt/app-root/jenkins/
USER 1001

# Run the JNLP client by default
# To use swarm client, specify "/opt/app-root/jenkins/run-swarm-client" as Command
ENTRYPOINT ["/opt/app-root/jenkins/run-jnlp-client"]

Comment 1 XiuJuan Wang 2016-04-27 07:32:43 UTC

# openshift version
openshift v3.2.0.20
kubernetes v1.2.0-36-g4a3f9c5
etcd 2.2.5

# docker version
Client:
 Version:         1.9.1
 API version:     1.21
 Package version: docker-1.9.1-25.el7.x86_64
 Go version:      go1.4.2
 Git commit:      78ee77d/1.9.1
 Built:           
 OS/Arch:         linux/amd64

Server:
 Version:         1.9.1
 API version:     1.21
 Package version: docker-1.9.1-25.el7.x86_64
 Go version:      go1.4.2
 Git commit:      78ee77d/1.9.1
 Built:           
 OS/Arch:         linux/amd64

Comment 2 Brenton Leanhardt 2016-04-27 12:06:42 UTC

There are a couple of problems here and neither are actually related to openshift.

1) https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/nfv/os/repodata/repomd.xml has a problem.  I'll report this to RCM.

2) That repository probably shouldn't be enabled by default.  This is likely a backend entitlement issue.  It started showing up on other RHEL hosts I maintain and "breaks yum" there too.

Admins that ever face a similar problem for any reason can add --disablerepo=<broken repo> to their yum CMD as a workaround.

Comment 9 Louis Santillan 2017-05-18 04:33:58 UTC

So this bug is starting to affect other repos and other images.

```
[root@ocp34all ~]# docker version
Client:
 Version:         1.12.6
 API version:     1.24
 Package version: docker-common-1.12.6-16.el7.x86_64
 Go version:      go1.7.4
 Git commit:      3a094bd/1.12.6
 Built:           Tue Mar 21 13:30:59 2017
 OS/Arch:         linux/amd64

Server:
 Version:         1.12.6
 API version:     1.24
 Package version: docker-common-1.12.6-16.el7.x86_64
 Go version:      go1.7.4
 Git commit:      3a094bd/1.12.6
 Built:           Tue Mar 21 13:30:59 2017
 OS/Arch:         linux/amd64
[root@ocp34all ~]# openshift version
openshift v3.4.1.18
kubernetes v1.4.0+776c994
etcd 3.1.0-rc.0
```

Other logging info
```
[root@ocp34all ~]# oc version
oc v3.4.1.18
kubernetes v1.4.0+776c994
features: Basic-Auth GSSAPI Kerberos SPNEGO

Server https://i.truthaddict.org:8443
openshift v3.4.1.18
kubernetes v1.4.0+776c994
[root@ocp34all ~]# docker version
Client:
 Version:         1.12.6
 API version:     1.24
 Package version: docker-common-1.12.6-16.el7.x86_64
 Go version:      go1.7.4
 Git commit:      3a094bd/1.12.6
 Built:           Tue Mar 21 13:30:59 2017
 OS/Arch:         linux/amd64

Server:
 Version:         1.12.6
 API version:     1.24
 Package version: docker-common-1.12.6-16.el7.x86_64
 Go version:      go1.7.4
 Git commit:      3a094bd/1.12.6
 Built:           Tue Mar 21 13:30:59 2017
 OS/Arch:         linux/amd64
[root@ocp34all ~]# openshift version
openshift v3.4.1.18
kubernetes v1.4.0+776c994
etcd 3.1.0-rc.0
[root@ocp34all ~]# subscription-manager unregister
System has been unregistered.
[root@ocp34all ~]# subscription-manager clean
All local data removed
[root@ocp34all ~]# subscription-manager register && subscription-manager status || echo $?
Registering to: subscription.rhsm.redhat.com:443/subscription
Username: rhn-gps-lsantill
Password: 
The system has been registered with ID: 475a48de-1ac2-408f-aa97-3f41768be54d 
+-------------------------------------------+
   System Status Details
+-------------------------------------------+
Overall Status: Invalid

Red Hat OpenShift Container Platform:
- Not supported by a valid subscription.

Red Hat Enterprise Linux Server:
- Not supported by a valid subscription.

1
[root@ocp34all ~]# subscription-manager auto-attach 
Auto-attach preference: enabled
[root@ocp34all ~]# subscription-manager list --available --matches '*OpenShift*'
+-------------------------------------------+
    Available Subscriptions
+-------------------------------------------+
Subscription Name:   30 Day Self-Supported Red Hat OpenShift Container Platform, 2-Core Evaluation
Provides:            Red Hat OpenShift Enterprise Application Node
....[SNIP]....
                     Red Hat Enterprise Linux Server
                     Red Hat Software Collections Beta (for RHEL Server)
                     RHEL for SAP Applications for Power LE
SKU:                 ES0113909
Contract:            11395826
Pool ID:             8a85f9815bd4351c015bd4a6b1694bd6
Provides Management: Yes
Available:           Unlimited
Suggested:           1
Service Level:       Self-Support
Service Type:        L1-L3
Subscription Type:   Standard
Ends:                05/03/2020
System Type:         Virtual

[root@ocp34all ~]#  subscription-manager attach --pool=8a85f9815bd4351c015bd4a6b1694bd6
Successfully attached a subscription for: Employee SKU
[root@ocp34all ~]# subscription-manager repos --disable="*"
Repository 'rhel-7-server-openstack-11-devtools-debug-rpms' is disabled for this system.
....[SNIP]....
Repository 'rhel-7-server-openstack-7.0-tools-source-rpms' is disabled for this system.
Repository 'rhel-7-server-openstack-6.0-installer-rpms' is disabled for this system.
[root@ocp34all ~]# yum repolist
Loaded plugins: product-id, search-disabled-repos, subscription-manager
repo id                                       repo name                                                                       status
!epel/x86_64                                  Extra Packages for Enterprise Linux 7 - x86_64                                  11,659
repolist: 11,659
[root@ocp34all ~]# subscription-manager repos \
>     --enable="rhel-7-server-rpms" \
>     --enable="rhel-7-server-extras-rpms" \
>     --enable="rhel-7-server-ose-3.4-rpms" --enable="rhel-7-server-optional-rpms"
Repository 'rhel-7-server-rpms' is enabled for this system.
Repository 'rhel-7-server-optional-rpms' is enabled for this system.
Repository 'rhel-7-server-ose-3.4-rpms' is enabled for this system.
Repository 'rhel-7-server-extras-rpms' is enabled for this system.
[root@ocp34all ~]# yum update
Loaded plugins: product-id, search-disabled-repos, subscription-manager
epel/x86_64/metalink                                                                                         | 8.2 kB  00:00:00     
epel                                                                                                         | 4.3 kB  00:00:00     
rhel-7-server-extras-rpms                                                                                    | 3.4 kB  00:00:00     
rhel-7-server-optional-rpms                                                                                  | 3.5 kB  00:00:00     
rhel-7-server-ose-3.4-rpms                                                                                   | 4.0 kB  00:00:00     
rhel-7-server-rpms                                                                                           | 3.5 kB  00:00:00     
No packages marked for update
[root@ocp34all ~]# yum repolist
Loaded plugins: product-id, search-disabled-repos, subscription-manager
repo id                                                   repo name                                                           status
*epel/x86_64                                              Extra Packages for Enterprise Linux 7 - x86_64                      11,659
rhel-7-server-extras-rpms/x86_64                          Red Hat Enterprise Linux 7 Server - Extras (RPMs)                      465
rhel-7-server-optional-rpms/7Server/x86_64                Red Hat Enterprise Linux 7 Server - Optional (RPMs)                 10,907
rhel-7-server-ose-3.4-rpms/x86_64                         Red Hat OpenShift Container Platform 3.4 (RPMs)                        704
rhel-7-server-rpms/7Server/x86_64                         Red Hat Enterprise Linux 7 Server (RPMs)                            14,285
repolist: 38,020
[root@ocp34all ~]# subscription-manager status 
+-------------------------------------------+
   System Status Details
+-------------------------------------------+
Overall Status: Current
[root@ocp34all ~]# cd containers-quickstarts
[root@ocp34all containers-quickstarts]# oc delete is jenkins-slave-base-rhel7 jenkins-slave-image-mgmt && oc delete bc jenkins-slave-image-mgmt
imagestream "jenkins-slave-base-rhel7" deleted
imagestream "jenkins-slave-image-mgmt" deleted
buildconfig "jenkins-slave-image-mgmt" deleted
[root@ocp34all containers-quickstarts]# oc process -f jenkins-slaves/templates/jenkins-slave-image-mgmt-template.json -v SOURCE_REPOSITORY_URL=https://github.com/sabre1041/containers-quickstarts.git -v SOURCE_REPOSITORY_REF=skopeo-rhel | oc apply -f-
imagestream "jenkins-slave-base-rhel7" created
imagestream "jenkins-slave-image-mgmt" created
buildconfig "jenkins-slave-image-mgmt" created
[root@ocp34all containers-quickstarts]# oc get pods
NAME                               READY     STATUS    RESTARTS   AGE
jenkins-1-o0gu2                    1/1       Running   0          5h
jenkins-slave-image-mgmt-1-build   1/1       Running   0          7s
[root@ocp34all containers-quickstarts]# oc logs -f jenkins-slave-image-mgmt-1-build
Cloning "https://github.com/sabre1041/containers-quickstarts.git" ...
	Commit:	9ef944bf18fb1fa2c476168df49fe87a9e04c6e3 (Refactor command from create to apply)
	Author:	Andrew Block <andy.block>
	Date:	Wed May 17 19:06:01 2017 -0500
Step 1 : FROM registry.access.redhat.com/openshift3/jenkins-slave-base-rhel7@sha256:37bcfe2acc95193622d819046f0cf50655f996c494b756b344bdfe1060b57426
 ---> 1efc4d59bd10
Step 2 : MAINTAINER Andrew Block <ablock>
 ---> Using cache
 ---> 53a5980c691a
Step 3 : LABEL com.redhat.component "jenkins-slave-image-mgmt" name "jenkins-slave-image-mgmt" architecture "x86_64" io.k8s.display-name "Jenkins Slave Image Management" io.k8s.description "Image management tools on top of the jenkins slave base image" io.openshift.tags "openshift,jenkins,slave,copy"
 ---> Using cache
 ---> a8230e358c65
Step 4 : USER root
 ---> Using cache
 ---> 884a7e3a3b08
Step 5 : RUN yum repolist > /dev/null &&     yum-config-manager --enable rhel-7-server-extras-rpms &&     yum clean all &&     INSTALL_PKGS="skopeo" &&     yum install -y --setopt=tsflags=nodocs $INSTALL_PKGS &&     rpm -V $INSTALL_PKGS &&     yum clean all
 ---> Running in 44fb6e01f135
https://cdn.redhat.com/content/htb/rhel/server/7/x86_64/os/repodata/repomd.xml: [Errno 14] HTTPS Error 403 - Forbidden
Trying other mirror.
To address this issue please refer to the below knowledge base article

https://access.redhat.com/solutions/69319

If above article doesn't help to resolve this issue please open a ticket with Red Hat Support.

https://cdn.redhat.com/content/htb/rhel/server/7/x86_64/os/repodata/repomd.xml: [Errno 14] HTTPS Error 403 - Forbidden
Trying other mirror.
https://cdn.redhat.com/content/beta/rhel/server/7/x86_64/rt/os/repodata/repomd.xml: [Errno 14] HTTPS Error 403 - Forbidden
Trying other mirror.
Loaded plugins: ovl, product-id
======================= repo: rhel-7-server-extras-rpms ========================
[rhel-7-server-extras-rpms]
async = True
bandwidth = 0
base_persistdir = /var/lib/yum/repos/x86_64/7Server
baseurl = https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/extras/os
cache = 0
cachedir = /var/cache/yum/x86_64/7Server/rhel-7-server-extras-rpms
check_config_file_age = True
compare_providers_priority = 80
cost = 1000
deltarpm_metadata_percentage = 100
deltarpm_percentage = 
enabled = 1
enablegroups = True
exclude = 
failovermethod = priority
ftp_disable_epsv = False
gpgcadir = /var/lib/yum/repos/x86_64/7Server/rhel-7-server-extras-rpms/gpgcadir
gpgcakey = 
gpgcheck = True
gpgdir = /var/lib/yum/repos/x86_64/7Server/rhel-7-server-extras-rpms/gpgdir
gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release
hdrdir = /var/cache/yum/x86_64/7Server/rhel-7-server-extras-rpms/headers
http_caching = all
includepkgs = 
ip_resolve = 
keepalive = True
keepcache = False
mddownloadpolicy = sqlite
mdpolicy = group:small
mediaid = 
metadata_expire = 86400
metadata_expire_filter = read-only:present
metalink = 
minrate = 0
mirrorlist = 
mirrorlist_expire = 86400
name = Red Hat Enterprise Linux 7 Server - Extras (RPMs)
old_base_cache_dir = 
password = 
persistdir = /var/lib/yum/repos/x86_64/7Server/rhel-7-server-extras-rpms
pkgdir = /var/cache/yum/x86_64/7Server/rhel-7-server-extras-rpms/packages
proxy = False
proxy_dict = 
proxy_password = 
proxy_username = 
repo_gpgcheck = False
retries = 10
skip_if_unavailable = False
ssl_check_cert_permissions = True
sslcacert = /etc/rhsm-host/ca/redhat-uep.pem
sslclientcert = /etc/pki/entitlement-host/1917833409556707654.pem
sslclientkey = /etc/pki/entitlement-host/1917833409556707654-key.pem
sslverify = True
throttle = 0
timeout = 30.0
ui_id = rhel-7-server-extras-rpms/x86_64
ui_repoid_vars = basearch
username = 

Loaded plugins: ovl, product-id, search-disabled-repos, subscription-manager
Cleaning repos: rhel-7-server-aus-rpms rhel-7-server-extras-rpms
              : rhel-7-server-htb-rpms rhel-7-server-nfv-rpms rhel-7-server-rpms
              : rhel-7-server-rt-beta-rpms rhel-7-server-rt-htb-rpms
              : rhel-7-server-rt-rpms rhel-7-server-tus-rpms
Cleaning up everything
Loaded plugins: ovl, product-id, search-disabled-repos, subscription-manager
https://cdn.redhat.com/content/htb/rhel/server/7/x86_64/os/repodata/repomd.xml: [Errno 14] HTTPS Error 403 - Forbidden
Trying other mirror.
To address this issue please refer to the below knowledge base article

https://access.redhat.com/solutions/69319

If above article doesn't help to resolve this issue please open a ticket with Red Hat Support.



 One of the configured repositories failed (Red Hat Enterprise Linux 7 Server HTB (RPMs)),
 and yum doesn't have enough cached data to continue. At this point the only
 safe thing yum can do is fail. There are a few ways to work "fix" this:

     1. Contact the upstream for the repository and get them to fix the problem.

     2. Reconfigure the baseurl/etc. for the repository, to point to a working
        upstream. This is most often useful if you are using a newer
        distribution release than is supported by the repository (and the
        packages for the previous distribution release still work).

     3. Run the command with the repository temporarily disabled
            yum --disablerepo=rhel-7-server-htb-rpms ...

     4. Disable the repository permanently, so yum won't use it by default. Yum
        will then just ignore the repository until you permanently enable it
        again or use --enablerepo for temporary usage:

            yum-config-manager --disable rhel-7-server-htb-rpms
        or
            subscription-manager repos --disable=rhel-7-server-htb-rpms

     5. Configure the failing repository to be skipped, if it is unavailable.
        Note that yum will try to contact the repo. when it runs most commands,
        so will have to try and fail each time (and thus. yum will be be much
        slower). If it is a very temporary problem though, this is often a nice
        compromise:

            yum-config-manager --save --setopt=rhel-7-server-htb-rpms.skip_if_unavailable=true

failure: repodata/repomd.xml from rhel-7-server-htb-rpms: [Errno 256] No more mirrors to try.
https://cdn.redhat.com/content/htb/rhel/server/7/x86_64/os/repodata/repomd.xml: [Errno 14] HTTPS Error 403 - Forbidden
error: build error: The command '/bin/sh -c yum repolist > /dev/null &&     yum-config-manager --enable rhel-7-server-extras-rpms &&     yum clean all &&     INSTALL_PKGS="skopeo" &&     yum install -y --setopt=tsflags=nodocs $INSTALL_PKGS &&     rpm -V $INSTALL_PKGS &&     yum clean all' returned a non-zero code: 1
```

Comment 10 Brian Cook 2018-04-25 17:48:16 UTC

RHEL 7 HTB repo is still causing problems.

failure: repodata/repomd.xml from rhel-7-server-htb-rpms: [Errno 256] No more mirrors to try.
https://cdn.redhat.com/content/htb/rhel/server/7/x86_64/os/repodata/repomd.xml: [Errno 14] HTTPS Error 403 - Forbidden
[0mRemoving intermediate container ac3de8b8ed68
error: build error: The command '/bin/sh -c yum -y update-minimal --security --sec-severity=Important --sec-severity=Critical --setopt=tsflags=nodocs &&     yum clean all && /bin/true' returned a non-zero code: 1


This is causing builds to fail in Openshift dedicated because the RHEL7 base image is configured with search-disabled-repos enabled=1.

Comment 11 Ben Parees 2018-04-25 18:48:22 UTC


*** This bug has been marked as a duplicate of bug 1570968 ***