Bug 1505744 - fail to pre-pull container engine image against an authenticated registry with openshift_docker_use_system_container enabled
Summary: fail to pre-pull container engine image against an authenticated registry wit...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 3.7.0
Hardware: Unspecified
OS: Unspecified
medium
high
Target Milestone: ---
: 3.7.z
Assignee: Giuseppe Scrivano
QA Contact: Johnny Liu
URL:
Whiteboard:
Depends On:
Blocks: 1510148
TreeView+ depends on / blocked
 
Reported: 2017-10-24 08:28 UTC by Johnny Liu
Modified: 2018-05-22 09:49 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: Known Issue
Doc Text:
The installer can not deploy system container based installations when the specified registry requires authentication credentials in order to pull the required system container images. The fix for this depends on an update to the atomic command which will be updated after 3.7 GA.
Clone Of:
: 1510148 (view as bug list)
Environment:
Last Closed: 2018-05-22 09:49:48 UTC
Target Upstream Version:


Attachments (Terms of Use)

Comment 1 Giuseppe Scrivano 2017-10-24 16:10:27 UTC
could you please share your inventory so that it will be easier for me to reproduce?

Could you also share the credentials that I can use for registry.reg-aws.openshift.com?

Comment 2 Johnny Liu 2017-10-25 02:44:56 UTC
(In reply to Giuseppe Scrivano from comment #1)
> could you please share your inventory so that it will be easier for me to
> reproduce?
You could get inventory host file in my attachment by searching "openshift-ansible-inventory-start" keyword.
> 
> Could you also share the credentials that I can use for
> registry.reg-aws.openshift.com?
I will set the credentials via email later.

Comment 3 Giuseppe Scrivano 2017-10-25 15:17:14 UTC
I've opened two PR:

https://github.com/openshift/openshift-ansible/pull/5878

and:

https://github.com/openshift/openshift-ansible/pull/5880 (to address the first issue)

I've splitted them since the first one is blocked on a new feature in atomic that allows to set credentials when pulling system containers:

https://github.com/projectatomic/atomic/pull/1120

Comment 7 Scott Dodson 2017-11-06 18:34:11 UTC
Since this depends on a newer version of atomic that won't be available until after 3.7.0 GA we have to move this to 3.7.z.

Comment 8 Scott Dodson 2017-11-06 18:37:03 UTC
I've put in this release note request. 

https://github.com/openshift/openshift-docs/issues/4906#issuecomment-342242967

Comment 9 Giuseppe Scrivano 2017-11-06 18:52:24 UTC
Closed upstream with: https://github.com/projectatomic/atomic/pull/1120

commit 1c877c5860921e8beedf75ea75964ec9d6e97b07
Author: Giuseppe Scrivano <gscrivan@redhat.com>
Date:   Wed Oct 25 09:51:05 2017 +0200

    syscontainers: support credentials for accessing the source registry
    
    Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1505744
    
    Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
    
    Closes: #1120
    Approved by: ashcrow

Comment 13 Johnny Liu 2017-11-17 06:11:46 UTC
In today's installation, seem like "atomic pull" is working well, but will encounter BZ#1514324.

openshift-ansible-3.7.9-1.git.0.60e60a0.el7.noarch
atomic-1.19.1-5.git48c224b.el7.x86_64
# openshift version
openshift v3.7.9
kubernetes v1.7.6+a08f5eeb62
etcd 3.2.8

Comment 18 Giuseppe Scrivano 2018-01-02 14:17:56 UTC
Scott, yes since https://github.com/projectatomic/atomic/pull/1120

I've a WIP PR for openshift-ansible but it requires that change to hit a release:

https://github.com/openshift/openshift-ansible/pull/5878

@Johhny Liu, could you verify if that change is present with the atomic tool you are using?

Comment 20 Giuseppe Scrivano 2018-01-04 11:44:57 UTC
can you please check this again?

After the fix from: https://bugzilla.redhat.com/show_bug.cgi?id=1514324 I cannot encounter this issue anymore.

Comment 21 Johnny Liu 2018-01-08 08:32:34 UTC
Retest this bug with atomic-1.20.1-9.git436cf5d.el7.x86_64 + openshift-ansible-3.7.18-1.git.0.a01e769.el7.noarch, still reproduce.

installation log with inventory host file embedded will be attached later.


The root cause is already mentioned in comment 14, seen from openshift-ansible plabyook, "Pre-pull Container Engine System Container image" is happening prior to registry_auth.yml, that means, /root/.docker/config.json is not created yet when running "Pre-pull Container Engine System Container image" task.

Comment 23 Giuseppe Scrivano 2018-01-08 10:45:58 UTC
PR opened to address that issue:

https://github.com/openshift/openshift-ansible/pull/6644

Comment 24 Giuseppe Scrivano 2018-01-11 16:29:07 UTC
*** Bug 1528583 has been marked as a duplicate of this bug. ***

Comment 26 Johnny Liu 2018-05-22 09:48:43 UTC
Verified this bug with openshift-ansible-3.7.46-1.git.0.37f607e.el7.noarch, and PASS.

Now "Create credentials for docker cli registry auth (alternative)" task happened prior to "Pre-pull Container Engine System Container image" task, so "atomic pull" is completed successfully.

[root@ip-172-18-10-123 ~]# cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.5 (Maipo)
[root@ip-172-18-10-123 ~]# uname -r
3.10.0-862.el7.x86_64

Comment 27 Johnny Liu 2018-05-22 09:49:48 UTC
# rpm -q atomic
atomic-1.22.1-3.git2fd0860.el7.x86_64


Note You need to log in before you can comment on or make changes to this bug.