Description of problem: During undercloud deployment, docker-registry is started in puppet. A recent docker release removed docker-registry, so the service fails to start which causes undercloud deployment to fail Version-Release number of selected component (if applicable): Ocata RDO OSP 11 OSP 10 How reproducible: Always Steps to Reproduce: 1.deploy undercloud with default options 2. 3. Actual results: undercloud fails Expected results: undercloud install succeeds Additional info: Related bug 1428606 has been filed to disable docker registry by default. Workaround: In the undercloud.conf file set the following: hieradata_override = foo.yaml in foo.yaml # turn off docker registry enable_docker_registry: false
Created attachment 1259354 [details] undercloud.log
RHEL bug that dropped docker-registry: https://bugzilla.redhat.com/show_bug.cgi?id=1388923
(In reply to Mike Burns from comment #2) > RHEL bug that dropped docker-registry: > https://bugzilla.redhat.com/show_bug.cgi?id=1388923 That bug says docker-registry is unmaintained since long time and docker-distribution should be used. So bug is in tripleo that it uses docker-registry at all. On the RPM level, docker-distribution has correct Obsoletes/Provides: http://pkgs.devel.redhat.com/cgit/rpms/docker-distribution/tree/docker-distribution.spec?h=extras-rhel-7.3#n54 but they missed to add compat docker-registry.service symlink :(
> So bug is in tripleo that it uses docker-registry at all. ok, that's already fixed https://github.com/openstack/puppet-tripleo/commit/a21f1a193a7076a335d82f47cab04340da9c93ff Why is it failing then in Ocata ?!
(In reply to Alan Pevec from comment #4) > > So bug is in tripleo that it uses docker-registry at all. > > ok, that's already fixed > https://github.com/openstack/puppet-tripleo/commit/ > a21f1a193a7076a335d82f47cab04340da9c93ff > > Why is it failing then in Ocata ?! The other half of the change was only in master: https://review.openstack.org/#/c/440842/
Linking another patch that has to be backported alongside, to prevent introducing this bug: https://bugs.launchpad.net/tripleo/+bug/1666459
Adding a note here as I hit it on two environments which needed slightly different workarounds, so may be helpful for anyone else working here today as I wasted a few runs getting them both to pass. ---- Starting with OSP10 and upgrading to OSP11, the openstack undercloud upgrade fails like: 2017-03-03 07:04:18,606 INFO: ^[[1;31mError: /Stage[main]/Main/Service[docker-registry]/ensure: change from stopped to running failed: Systemd start for docker-registry failed! To fix this you need the missing patch as per comment #5 so you can do: # https://review.openstack.org/#/c/440842/ Move Docker registry setup into its own profile curl https://review.openstack.org/changes/440842/revisions/current/patch?download | \ base64 -d | sudo patch -d /usr/share/instack-undercloud/ -p2 and then the openstack undercloud upgrade should complete. ---- Starting at OSP10 from scratch (new deployment), the openstack undercloud install fails like: 2017-03-03 07:25:28,926 INFO: 2017-03-03 07:25:28 - ^[[1;31mError: Could not start Service[docker-registry]: Execution of '/bin/systemctl start docker-registry' returned 5: Failed to start docker-registry.service: Unit not found.^[[0m Here the patch from 440842 above doesn't apply cleanly so I manually applied it at /usr/share/instack-undercloud/puppet-stack-config/puppet-stack.config.pp [root@instack puppet-stack-config]# diff puppet-stack-config.pp puppet-stack-config.pp.ORIG 435c435,444 < include ::tripleo::profile::base::docker_registry --- > package{'docker-registry': } > augeas { 'docker-registry': > context => '/files/etc/sysconfig/docker-registry', > changes => 'set REGISTRY_PORT 8787', > notify => Service['docker-registry'], > } > service { 'docker-registry': > ensure => running, > require => Package['docker-registry'], > } Here I also applied the two patches that add the docker registry into puppet-tripleo as they were missing (the profile::base::docker_registry): # https://review.openstack.org/#/c/441124 Add Docker Registry profile curl https://review.openstack.org/changes/441124/revisions/current/patch?download | \ base64 -d | sudo patch -d /usr/share/openstack-puppet/modules/tripleo/ -p1 && # https://review.openstack.org/#/c/441130/ Stop accidentally removing docker-distribution curl https://review.openstack.org/changes/441130/revisions/current/patch?download | \ base64 -d | sudo patch -d /usr/share/openstack-puppet/modules/tripleo/ -p1 Both applied cleanly for me today and then the undercloud install completes.
I have the same issue on OSP9 and here are my results of patch application failure (440842): curl https://review.openstack.org/changes/441124/revisions/current/patch?download | base64 -d | sudo patch -d /usr/share/openstack-puppet/modules/tripleo/ -p1 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 4864 0 4864 0 0 3806 0 --:--:-- 0:00:01 --:--:-- 3808 patching file manifests/profile/base/docker_registry.pp patching file templates/docker_distribution/registry_config.yml.erb curl https://review.openstack.org/changes/441130/revisions/current/patch?download | base64 -d | sudo patch -d /usr/share/openstack-puppet/modules/tripleo/ -p1 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 1740 0 1740 0 0 341 0 --:--:-- 0:00:05 --:--:-- 435 patching file manifests/profile/base/docker_registry.pp curl https://review.openstack.org/changes/440842/revisions/current/patch?download | base64 -d | sudo patch -d /usr/share/instack-undercloud/ -p2 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 2468 0 2468 0 0 1315 0 --:--:-- 0:00:01 --:--:-- 1314 patching file puppet-stack-config/puppet-stack-config.pp Hunk #1 FAILED at 515. 1 out of 1 hunk FAILED -- saving rejects to file puppet-stack-config/puppet-stack-config.pp.rej
Facing same problem with undercloud isntall on OSP9
Facing this on OSP 10 GA.
I tried the patches on an already failed OSP10 GA undercloud install and that failed as well. I then cleaned my undercloud machine (rebuilt the RHEL OS) and redeployed a newer build with the fix in it and I was able to get passed the undercloud install. I would recommend as a current workaround to deploy the latest with the fix rather than trying to patch.
Yes, this puddle worked for me 2017-03-03.1
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2017:1245