Bug 1505744 - fail to pre-pull container engine image against an authenticated registry with openshift_docker_use_system_container enabled
Summary: fail to pre-pull container engine image against an authenticated registry wit...
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 3.7.0
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
: 3.7.z
Assignee: Giuseppe Scrivano
QA Contact: Johnny Liu
Depends On:
Blocks: 1510148
TreeView+ depends on / blocked
Reported: 2017-10-24 08:28 UTC by Johnny Liu
Modified: 2018-05-22 09:49 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: Known Issue
Doc Text:
The installer can not deploy system container based installations when the specified registry requires authentication credentials in order to pull the required system container images. The fix for this depends on an update to the atomic command which will be updated after 3.7 GA.
Clone Of:
: 1510148 (view as bug list)
Last Closed: 2018-05-22 09:49:48 UTC
Target Upstream Version:

Attachments (Terms of Use)

Comment 1 Giuseppe Scrivano 2017-10-24 16:10:27 UTC
could you please share your inventory so that it will be easier for me to reproduce?

Could you also share the credentials that I can use for registry.reg-aws.openshift.com?

Comment 2 Johnny Liu 2017-10-25 02:44:56 UTC
(In reply to Giuseppe Scrivano from comment #1)
> could you please share your inventory so that it will be easier for me to
> reproduce?
You could get inventory host file in my attachment by searching "openshift-ansible-inventory-start" keyword.
> Could you also share the credentials that I can use for
> registry.reg-aws.openshift.com?
I will set the credentials via email later.

Comment 3 Giuseppe Scrivano 2017-10-25 15:17:14 UTC
I've opened two PR:



https://github.com/openshift/openshift-ansible/pull/5880 (to address the first issue)

I've splitted them since the first one is blocked on a new feature in atomic that allows to set credentials when pulling system containers:


Comment 7 Scott Dodson 2017-11-06 18:34:11 UTC
Since this depends on a newer version of atomic that won't be available until after 3.7.0 GA we have to move this to 3.7.z.

Comment 8 Scott Dodson 2017-11-06 18:37:03 UTC
I've put in this release note request. 


Comment 9 Giuseppe Scrivano 2017-11-06 18:52:24 UTC
Closed upstream with: https://github.com/projectatomic/atomic/pull/1120

commit 1c877c5860921e8beedf75ea75964ec9d6e97b07
Author: Giuseppe Scrivano <gscrivan@redhat.com>
Date:   Wed Oct 25 09:51:05 2017 +0200

    syscontainers: support credentials for accessing the source registry
    Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1505744
    Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
    Closes: #1120
    Approved by: ashcrow

Comment 13 Johnny Liu 2017-11-17 06:11:46 UTC
In today's installation, seem like "atomic pull" is working well, but will encounter BZ#1514324.

# openshift version
openshift v3.7.9
kubernetes v1.7.6+a08f5eeb62
etcd 3.2.8

Comment 18 Giuseppe Scrivano 2018-01-02 14:17:56 UTC
Scott, yes since https://github.com/projectatomic/atomic/pull/1120

I've a WIP PR for openshift-ansible but it requires that change to hit a release:


@Johhny Liu, could you verify if that change is present with the atomic tool you are using?

Comment 20 Giuseppe Scrivano 2018-01-04 11:44:57 UTC
can you please check this again?

After the fix from: https://bugzilla.redhat.com/show_bug.cgi?id=1514324 I cannot encounter this issue anymore.

Comment 21 Johnny Liu 2018-01-08 08:32:34 UTC
Retest this bug with atomic-1.20.1-9.git436cf5d.el7.x86_64 + openshift-ansible-3.7.18-1.git.0.a01e769.el7.noarch, still reproduce.

installation log with inventory host file embedded will be attached later.

The root cause is already mentioned in comment 14, seen from openshift-ansible plabyook, "Pre-pull Container Engine System Container image" is happening prior to registry_auth.yml, that means, /root/.docker/config.json is not created yet when running "Pre-pull Container Engine System Container image" task.

Comment 23 Giuseppe Scrivano 2018-01-08 10:45:58 UTC
PR opened to address that issue:


Comment 24 Giuseppe Scrivano 2018-01-11 16:29:07 UTC
*** Bug 1528583 has been marked as a duplicate of this bug. ***

Comment 26 Johnny Liu 2018-05-22 09:48:43 UTC
Verified this bug with openshift-ansible-3.7.46-1.git.0.37f607e.el7.noarch, and PASS.

Now "Create credentials for docker cli registry auth (alternative)" task happened prior to "Pre-pull Container Engine System Container image" task, so "atomic pull" is completed successfully.

[root@ip-172-18-10-123 ~]# cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.5 (Maipo)
[root@ip-172-18-10-123 ~]# uname -r

Comment 27 Johnny Liu 2018-05-22 09:49:48 UTC
# rpm -q atomic

Note You need to log in before you can comment on or make changes to this bug.