Bug 1393342

Summary: Specified docker registry did not take effect
Product: OpenShift Container Platform Reporter: liujia <jiajliu>
Component: Cluster Version OperatorAssignee: Scott Dodson <sdodson>
Status: CLOSED WONTFIX QA Contact: Anping Li <anli>
Severity: low Docs Contact:
Priority: medium    
Version: 3.4.0CC: aos-bugs, dgoodwin, jokerman, mmccomas, pprakash
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-08-24 20:49:06 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description liujia 2016-11-09 11:23:59 UTC
Description of problem:
When upgrade OCP3.3 to 3.4 with openshift_docker_additional_registries and openshift_docker_insecure_registries specified in installer.cfg.yml, the hosts file has been generated correctly, but the docker registry was not set into /etc/sysconfig/docker, which result that upgrade failed for not find images.
fatal: [openshift-128.x.x.x]: FAILED! => {
    "changed": true, 
    "cmd": [
        "docker", 
        "run", 
        "--rm", 
        "openshift3/ose:v3.4", 
        "version"
    ], 
    "delta": "0:00:10.722507", 
    "end": "2016-11-09 05:00:00.395162", 
    "failed": true, 
    "rc": 125, 
    "start": "2016-11-09 04:59:49.672655", 
    "warnings": []
}

STDERR:

Unable to find image 'openshift3/ose:v3.4' locally
Trying to pull repository x.x.x.x/openshift3/ose ... 
Error parsing HTTP response: invalid character 'F' looking for beginning of value: "File not found.\""
Trying to pull repository docker.io/openshift3/ose ... 
Pulling repository docker.io/openshift3/ose
Error: image openshift3/ose not found
docker: Error: image openshift3/ose not found.
See '/usr/bin/docker-current run --help'.


Version-Release number of selected component (if applicable):
atomic-openshift-utils-3.4.18-1.git.0.ed7dac0.el7.noarch

How reproducible:
always

Steps to Reproduce:
1.Install OCP3.3(container)
2.Edit installer.cfg.yml to set docker registry and upgrade OCP to 3.4 with latest 3.4 installer
<--snip-->
  master_routingconfig_subdomain: ''
  openshift_docker_additional_registries: virt-openshift-05.x.x.x:5000
  openshift_docker_insecure_registries: virt-openshift-05.x.x.x:5000

Actual results:
Upgrade failed for not found images.

Expected results:
It should set registry params and upgrade successfully.

Comment 1 Devan Goodwin 2016-11-09 20:20:36 UTC
Reconfiguring Docker to add additional registries requires a Docker daemon restart, something we have to be extremely careful with during upgrade. Currently we do this after the control plane is upgraded, and in parallel with node upgrade so we can safely evac each node, then upgrade docker.

However for your use case you quite likely need that registry configured *before* control plane upgrade so images can be pulled, which would require double node evacuation.

This is technically very difficult today, but might be more feasible once Docker 1.12 is a requirement as we could restart without affecting running pods.

Given this is largely a QE requirement, we moved to upcoming release. For a workaround, you can just re-run the config.yml playbook after applying your inventory changes to deploy the docker config and restart.