Bug 1520866

Summary: [3.7] "Setup ro mount of /root/.docker for containerized hosts" task is skipped in containerized install
Product: OpenShift Container Platform Reporter: Johnny Liu <jialiu>
Component: InstallerAssignee: Michael Gugino <mgugino>
Status: CLOSED ERRATA QA Contact: Johnny Liu <jialiu>
Severity: medium Docs Contact:
Priority: high    
Version: 3.7.0CC: aos-bugs, jokerman, mmccomas
Target Milestone: ---Keywords: Regression
Target Release: 3.7.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1523461 (view as bug list) Environment:
Last Closed: 2018-04-05 09:33:10 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1523461    

Description Johnny Liu 2017-12-05 10:26:28 UTC
Description of problem:
Trigger a containerized install (3 masters + 2 nodes), "Setup ro mount of /root/.docker for containerized hosts" is skipped.


TASK [openshift_node : Check for credentials file for registry auth] ***********
Tuesday 05 December 2017  08:47:35 +0000 (0:00:00.468)       0:23:00.159 ****** 
ok: [host-8-241-23.host.centralci.eng.rdu2.redhat.com] => {"changed": false, "failed": false, "stat": {"exists": false}}
ok: [host-8-241-26.host.centralci.eng.rdu2.redhat.com] => {"changed": false, "failed": false, "stat": {"exists": false}}

TASK [openshift_node : Create credentials for registry auth] *******************
Tuesday 05 December 2017  08:47:36 +0000 (0:00:00.323)       0:23:00.483 ****** 
changed: [host-8-241-23.host.centralci.eng.rdu2.redhat.com] => {"attempts": 1, "changed": true, "cmd": ["docker", "--config=/var/lib/origin/.docker", "login", "-u", "****", "-p", "****", "registry.reg-aws.openshift.com:443"], "delta": "0:00:00.271697", "end": "2017-12-05 03:47:36.314913", "failed": false, "rc": 0, "start": "2017-12-05 03:47:36.043216", "stderr": "", "stderr_lines": [], "stdout": "Login Succeeded", "stdout_lines": ["Login Succeeded"]}
changed: [host-8-241-26.host.centralci.eng.rdu2.redhat.com] => {"attempts": 1, "changed": true, "cmd": ["docker", "--config=/var/lib/origin/.docker", "login", "-u", "****", "-p", "****", "registry.reg-aws.openshift.com:443"], "delta": "0:00:00.324764", "end": "2017-12-05 03:47:36.591846", "failed": false, "rc": 0, "start": "2017-12-05 03:47:36.267082", "stderr": "", "stderr_lines": [], "stdout": "Login Succeeded", "stdout_lines": ["Login Succeeded"]}

TASK [openshift_node : Create credentials for registry auth (alternative)] *****
Tuesday 05 December 2017  08:47:36 +0000 (0:00:00.641)       0:23:01.125 ****** 
skipping: [host-8-241-23.host.centralci.eng.rdu2.redhat.com] => {"changed": false, "skip_reason": "Conditional result was False", "skipped": true}
skipping: [host-8-241-26.host.centralci.eng.rdu2.redhat.com] => {"changed": false, "skip_reason": "Conditional result was False", "skipped": true}

TASK [openshift_node : Setup ro mount of /root/.docker for containerized hosts] ***
Tuesday 05 December 2017  08:47:36 +0000 (0:00:00.047)       0:23:01.172 ****** 
skipping: [host-8-241-23.host.centralci.eng.rdu2.redhat.com] => {"changed": false, "skip_reason": "Conditional result was False", "skipped": true}
skipping: [host-8-241-26.host.centralci.eng.rdu2.redhat.com] => {"changed": false, "skip_reason": "Conditional result was False", "skipped": true}


That would cause all the pods are pending with "ContainerCreating" state due to "Failed create pod sandbox".

Check /etc/systemd/system/atomic-openshift-node.service, found no "-v /var/lib/origin/.docker:/root/.docker:ro" is added there.

Check openshift-ansible code, the root cause is "Create credentials for registry auth (alternative)" task is skipped, "node_oreg_auth_credentials_create" is overwritten, that lead to the when judgment for "Setup ro mount of /root/.docker for containerized hosts" task is set to FALSE, so the task is skipped.


This bug is should be a regression bug introduced by "0de92e0e4c85c876436ae21f61daee31c870705e"


Version-Release number of the following components:
openshift-ansible-3.7.11-1.git.0.42a781f.el7.noarch

How reproducible:
Always

Steps to Reproduce:
1.
2.
3.

Actual results:
Please include the entire output from the last TASK line through the end of output if an error is generated

Expected results:

Additional info:
Please attach logs from ansible-playbook with the -vvv flag

Comment 1 Michael Gugino 2017-12-05 17:23:56 UTC
PR Created: https://github.com/openshift/openshift-ansible/pull/6357

Comment 2 Michael Gugino 2017-12-05 19:26:48 UTC
PR merged.

Backport PR to 3.7 merged: https://github.com/openshift/openshift-ansible/pull/6358

Comment 4 Johnny Liu 2018-01-26 06:56:19 UTC
Verified this bug with openshift-ansible-3.7.26-1.git.0.f87f1af.el7.noarch, and PASS.

"-v /var/lib/origin/.docker:/root/.docker:ro" is shown in /etc/systemd/system/atomic-openshift-node.service, all the pods are running well.

Comment 8 errata-xmlrpc 2018-04-05 09:33:10 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0636