Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1523461

Summary:	"Setup ro mount of /root/.docker for containerized hosts" task is skipped in containerized install
Product:	OpenShift Container Platform	Reporter:	Johnny Liu <jialiu>
Component:	Installer	Assignee:	Michael Gugino <mgugino>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Johnny Liu <jialiu>
Severity:	medium	Docs Contact:
Priority:	high
Version:	3.8.0	CC:	aos-bugs, gpei, jokerman, mgugino, mmccomas, smunilla
Target Milestone:	---	Keywords:	Regression
Target Release:	3.9.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	openshift-ansible-3.9.0-0.16.0	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:	1520866	Environment:
Last Closed:	2018-06-13 14:18:48 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1520866
Bug Blocks:

Description Johnny Liu 2017-12-08 03:38:50 UTC

The same issue also happened in 3.8.

openshift-ansible-3.8.11-1.git.0.4f5b128.el7.noarch

+++ This bug was initially created as a clone of Bug #1520866 +++

Description of problem:
Trigger a containerized install (3 masters + 2 nodes), "Setup ro mount of /root/.docker for containerized hosts" is skipped.


TASK [openshift_node : Check for credentials file for registry auth] ***********
Tuesday 05 December 2017  08:47:35 +0000 (0:00:00.468)       0:23:00.159 ****** 
ok: [host-8-241-23.host.centralci.eng.rdu2.redhat.com] => {"changed": false, "failed": false, "stat": {"exists": false}}
ok: [host-8-241-26.host.centralci.eng.rdu2.redhat.com] => {"changed": false, "failed": false, "stat": {"exists": false}}

TASK [openshift_node : Create credentials for registry auth] *******************
Tuesday 05 December 2017  08:47:36 +0000 (0:00:00.323)       0:23:00.483 ****** 
changed: [host-8-241-23.host.centralci.eng.rdu2.redhat.com] => {"attempts": 1, "changed": true, "cmd": ["docker", "--config=/var/lib/origin/.docker", "login", "-u", "****", "-p", "****", "registry.reg-aws.openshift.com:443"], "delta": "0:00:00.271697", "end": "2017-12-05 03:47:36.314913", "failed": false, "rc": 0, "start": "2017-12-05 03:47:36.043216", "stderr": "", "stderr_lines": [], "stdout": "Login Succeeded", "stdout_lines": ["Login Succeeded"]}
changed: [host-8-241-26.host.centralci.eng.rdu2.redhat.com] => {"attempts": 1, "changed": true, "cmd": ["docker", "--config=/var/lib/origin/.docker", "login", "-u", "****", "-p", "****", "registry.reg-aws.openshift.com:443"], "delta": "0:00:00.324764", "end": "2017-12-05 03:47:36.591846", "failed": false, "rc": 0, "start": "2017-12-05 03:47:36.267082", "stderr": "", "stderr_lines": [], "stdout": "Login Succeeded", "stdout_lines": ["Login Succeeded"]}

TASK [openshift_node : Create credentials for registry auth (alternative)] *****
Tuesday 05 December 2017  08:47:36 +0000 (0:00:00.641)       0:23:01.125 ****** 
skipping: [host-8-241-23.host.centralci.eng.rdu2.redhat.com] => {"changed": false, "skip_reason": "Conditional result was False", "skipped": true}
skipping: [host-8-241-26.host.centralci.eng.rdu2.redhat.com] => {"changed": false, "skip_reason": "Conditional result was False", "skipped": true}

TASK [openshift_node : Setup ro mount of /root/.docker for containerized hosts] ***
Tuesday 05 December 2017  08:47:36 +0000 (0:00:00.047)       0:23:01.172 ****** 
skipping: [host-8-241-23.host.centralci.eng.rdu2.redhat.com] => {"changed": false, "skip_reason": "Conditional result was False", "skipped": true}
skipping: [host-8-241-26.host.centralci.eng.rdu2.redhat.com] => {"changed": false, "skip_reason": "Conditional result was False", "skipped": true}


That would cause all the pods are pending with "ContainerCreating" state due to "Failed create pod sandbox".

Check /etc/systemd/system/atomic-openshift-node.service, found no "-v /var/lib/origin/.docker:/root/.docker:ro" is added there.

Check openshift-ansible code, the root cause is "Create credentials for registry auth (alternative)" task is skipped, "node_oreg_auth_credentials_create" is overwritten, that lead to the when judgment for "Setup ro mount of /root/.docker for containerized hosts" task is set to FALSE, so the task is skipped.


This bug is should be a regression bug introduced by "0de92e0e4c85c876436ae21f61daee31c870705e"


Version-Release number of the following components:
openshift-ansible-3.7.11-1.git.0.42a781f.el7.noarch

How reproducible:
Always

Steps to Reproduce:
1.
2.
3.

Actual results:
Please include the entire output from the last TASK line through the end of output if an error is generated

Expected results:

Additional info:
Please attach logs from ansible-playbook with the -vvv flag

--- Additional comment from Michael Gugino on 2017-12-05 12:23:56 EST ---

PR Created: https://github.com/openshift/openshift-ansible/pull/6357

--- Additional comment from Michael Gugino on 2017-12-05 14:26:48 EST ---

PR merged.

Backport PR to 3.7 merged: https://github.com/openshift/openshift-ansible/pull/6358

Comment 4 Gaoyun Pei 2018-01-04 07:09:44 UTC

Tried 3.9 docker containerized installation with openshift-ansible-3.9.0-0.16.0.git.0.9f19afc.el7.noarch.rpm.

Registry and router pod are running well after installation, sti-build test also pass.


Related ansible output:
TASK [openshift_node : Check for credentials file for registry auth] ***********
Thursday 04 January 2018  06:03:31 +0000 (0:00:00.034)       0:15:21.015 ****** 
ok: [ec2-34-207-76-25.compute-1.amazonaws.com] => {"changed": false, "stat": {"exists": false}}

TASK [openshift_node : Create credentials for registry auth] *******************
Thursday 04 January 2018  06:03:31 +0000 (0:00:00.302)       0:15:21.318 ****** 
changed: [ec2-34-207-76-25.compute-1.amazonaws.com] => {"attempts": 1, "changed": true, "cmd": ["docker", "--config=/var/lib/origin/.docker", "login", "-u", "aos-qe-pull36", "-p", "CHnetzuONoYPO1swNS2AfEdrj5G67iJVogbrzec8lv8", "registry.reg-aws.openshift.com:443"], "delta": "0:00:00.120341", "end": "2018-01-04 01:03:32.222285", "rc": 0, "start": "2018-01-04 01:03:32.101944", "stderr": "", "stderr_lines": [], "stdout": "Login Succeeded", "stdout_lines": ["Login Succeeded"]}

TASK [openshift_node : Create credentials for registry auth (alternative)] *****
Thursday 04 January 2018  06:03:32 +0000 (0:00:00.447)       0:15:21.766 ****** 
skipping: [ec2-34-207-76-25.compute-1.amazonaws.com] => {"changed": false, "skip_reason": "Conditional result was False"}

TASK [openshift_node : Setup ro mount of /root/.docker for containerized hosts] ***
Thursday 04 January 2018  06:03:32 +0000 (0:00:00.026)       0:15:21.793 ****** 
ok: [ec2-34-207-76-25.compute-1.amazonaws.com] => {"ansible_facts": {"l_bind_docker_reg_auth": true}, "changed": false}


On node:
[root@ip-172-18-15-94 ~]# grep '/root/.docker' /etc/systemd/system/atomic-openshift-node.service
   -v /var/lib/origin/.docker:/root/.docker:ro\