Red Hat Bugzilla – Bug 1602120
[Term based registry] Installer fails to authenticate against prod secure registry
Last modified: 2018-10-11 03:22:04 EDT
Description of problem: Installer fails when trying to use Authenticated registry with following error <ec2-52-33-135-189.us-west-2.compute.amazonaws.com> (0, '\r\n{"exception": " File \\"/tmp/ansible_PzdEUT/ansible_module_docker_creds.py\\", line 141, in validate_registry_login\\n urllib.request.urlopen(req)\\n File \\"/usr/lib64/python2.7/urllib2.py\\", line 154, in urlopen\\n return opener.open(url, data, timeout)\\n File \\"/usr/lib64/python2.7/urllib2.py\\", line 437, in open\\n response = meth(req, response)\\n File \\"/usr/lib64/python2.7/urllib2.py\\", line 550, in http_response\\n \'http\', request, response, code, msg, hdrs)\\n File \\"/usr/lib64/python2.7/urllib2.py\\", line 475, in error\\n return self._call_chain(*args)\\n File \\"/usr/lib64/python2.7/urllib2.py\\", line 409, in _call_chain\\n result = func(*args)\\n File \\"/usr/lib64/python2.7/urllib2.py\\", line 558, in http_error_default\\n raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)\\n", "changed": false, "failed": true, "state": "unknown", "invocation": {"module_args": {"username": "prod-user-name", "test_login": "True", "password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER", "registry": "registry.redhat.io", "path": "/root/.docker"}}, "msg": "HTTP Error 401: Unauthorized"}\r\n', 'Shared connection to ec2-52-33-135-189.us-west-2.compute.amazonaws.com closed.\r\n') fatal: [ec2-52-33-135-189.us-west-2.compute.amazonaws.com]: FAILED! => { "failed": true, "msg": "The conditional check 'crt_oreg_auth_credentials_create.rc == 0' failed. The error was: error while evaluating conditional (crt_oreg_auth_credentials_create.rc == 0): 'dict object' has no attribute 'rc'" } Version-Release number of the following components: openshift-ansible 0e5dd09aa1d4bc9f938d8ef30a60b42c13358fbb rpm -q ansible ansible-2.4.5.0-1.el7ae.noarch ansible --version ansible 2.4.5.0 config file = /etc/ansible/ansible.cfg configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules'] ansible python module location = /usr/lib/python2.7/site-packages/ansible executable location = /usr/bin/ansible python version = 2.7.5 (default, Jun 12 2018, 10:42:23) [GCC 4.8.5 20150623 (Red Hat 4.8.5-34)] How reproducible: Only with prod authenticated registry Actual results: Please include the entire output from the last TASK line through the end of output if an error is generated <ec2-52-33-135-189.us-west-2.compute.amazonaws.com> (0, '\r\n{"exception": " File \\"/tmp/ansible_PzdEUT/ansible_module_docker_creds.py\\", line 141, in validate_registry_login\\n urllib.request.urlopen(req)\\n File \\"/usr/lib64/python2.7/urllib2.py\\", line 154, in urlopen\\n return opener.open(url, data, timeout)\\n File \\"/usr/lib64/python2.7/urllib2.py\\", line 437, in open\\n response = meth(req, response)\\n File \\"/usr/lib64/python2.7/urllib2.py\\", line 550, in http_response\\n \'http\', request, response, code, msg, hdrs)\\n File \\"/usr/lib64/python2.7/urllib2.py\\", line 475, in error\\n return self._call_chain(*args)\\n File \\"/usr/lib64/python2.7/urllib2.py\\", line 409, in _call_chain\\n result = func(*args)\\n File \\"/usr/lib64/python2.7/urllib2.py\\", line 558, in http_error_default\\n raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)\\n", "changed": false, "failed": true, "state": "unknown", "invocation": {"module_args": {"username": "prod-user-name", "test_login": "True", "password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER", "registry": "registry.redhat.io", "path": "/root/.docker"}}, "msg": "HTTP Error 401: Unauthorized"}\r\n', 'Shared connection to ec2-52-33-135-189.us-west-2.compute.amazonaws.com closed.\r\n') fatal: [ec2-52-33-135-189.us-west-2.compute.amazonaws.com]: FAILED! => { "failed": true, "msg": "The conditional check 'crt_oreg_auth_credentials_create.rc == 0' failed. The error was: error while evaluating conditional (crt_oreg_auth_credentials_create.rc == 0): 'dict object' has no attribute 'rc'" } Expected results: Install should succeed. Additional info: Please attach logs from ansible-playbook with the -vvv flag
Aside from I think we're validating incorrect variables. https://github.com/openshift/openshift-ansible/pull/9237
Will also need this patch: https://github.com/openshift/openshift-ansible/pull/9244 Currently, oreg_host is equal to '' if not oreg_url is not explicitly set in inventory; this is the value used for which registry to create credentials to. Patch updates to account for new enterprise reg.
I also hit the same issue, openshift-ansible-3.11.0-0.7.0.git.0.6e3e78eNone.noarch + ansible-2.6. Inventory file and installation log will be attached soon. This is blocking QE's testing.
Created attachment 1464871 [details] installation log with inventory file embedded
I also tested it with the latest openshift-ansible master branch (the last commit id: ec178734d6e555a87f66e2cc9ced23d854b5c9ba), also reproduce.
PR Created: https://github.com/openshift/openshift-ansible/pull/9286
PR merged in master.
The PR is available since openshift-ansible-3.11.0-0.8.0
I need to get some patches out to resolve this. For now, workaround, set following inv var: oreg_test_login: False
PR Created: https://github.com/openshift/openshift-ansible/pull/9349
Retest with openshift-ansible-3.11.0-0.10.0.git.0.91bb588None.noarch + "registry.dev.redhat.io" registry + "oreg_test_login=false" setting in inventory file, still reproduced.
Also hit the issue on openshift-ansible-3.11.0-0.10.0.git.0.91bb588None.noarch when do upgrade test with "registry.redhat.io" registry and correct oreg_auth_user+oreg_auth_password.
PR created: https://github.com/openshift/openshift-ansible/pull/9443
Based on comment 22, found workaround, removing testblocker keyword.
https://github.com/openshift/openshift-ansible/pull/9490 follow up fixes
This issue should be fixed on openshift-ansible-3.11.0-0.13.0.git.0.16dc599None.noarch. But upgrade still failed for another blocker bug 1612144.
Should be in openshift-ansible-3.11.0-0.15.0
Verified this bug with openshift-ansible-3.11.0-0.15.0.git.0.842d3d1None + "5318290|aosqeaosqe" as oreg_auth_user, and PASS. TASK [container_runtime : Create credentials for docker cli registry auth (alternative)] *** Wednesday 15 August 2018 16:06:51 +0800 (0:00:02.721) 0:01:56.806 ****** FAILED - RETRYING: Create credentials for docker cli registry auth (alternative) (3 retries left). FAILED - RETRYING: Create credentials for docker cli registry auth (alternative) (2 retries left). changed: [host-8-252-102.host.centralci.eng.rdu2.redhat.com] => {"attempts": 3, "changed": true, "rc": 0}
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2018:2652