Hide Forgot
Description of problem: When upgrade OCP3.3 to 3.4 with openshift_docker_additional_registries and openshift_docker_insecure_registries specified in installer.cfg.yml, the hosts file has been generated correctly, but the docker registry was not set into /etc/sysconfig/docker, which result that upgrade failed for not find images. fatal: [openshift-128.x.x.x]: FAILED! => { "changed": true, "cmd": [ "docker", "run", "--rm", "openshift3/ose:v3.4", "version" ], "delta": "0:00:10.722507", "end": "2016-11-09 05:00:00.395162", "failed": true, "rc": 125, "start": "2016-11-09 04:59:49.672655", "warnings": [] } STDERR: Unable to find image 'openshift3/ose:v3.4' locally Trying to pull repository x.x.x.x/openshift3/ose ... Error parsing HTTP response: invalid character 'F' looking for beginning of value: "File not found.\"" Trying to pull repository docker.io/openshift3/ose ... Pulling repository docker.io/openshift3/ose Error: image openshift3/ose not found docker: Error: image openshift3/ose not found. See '/usr/bin/docker-current run --help'. Version-Release number of selected component (if applicable): atomic-openshift-utils-3.4.18-1.git.0.ed7dac0.el7.noarch How reproducible: always Steps to Reproduce: 1.Install OCP3.3(container) 2.Edit installer.cfg.yml to set docker registry and upgrade OCP to 3.4 with latest 3.4 installer <--snip--> master_routingconfig_subdomain: '' openshift_docker_additional_registries: virt-openshift-05.x.x.x:5000 openshift_docker_insecure_registries: virt-openshift-05.x.x.x:5000 Actual results: Upgrade failed for not found images. Expected results: It should set registry params and upgrade successfully.
Reconfiguring Docker to add additional registries requires a Docker daemon restart, something we have to be extremely careful with during upgrade. Currently we do this after the control plane is upgraded, and in parallel with node upgrade so we can safely evac each node, then upgrade docker. However for your use case you quite likely need that registry configured *before* control plane upgrade so images can be pulled, which would require double node evacuation. This is technically very difficult today, but might be more feasible once Docker 1.12 is a requirement as we could restart without affecting running pods. Given this is largely a QE requirement, we moved to upcoming release. For a workaround, you can just re-run the config.yml playbook after applying your inventory changes to deploy the docker config and restart.