Bug 1428608

Created attachment 1259354 [details]
undercloud.log

RHEL bug that dropped docker-registry:  https://bugzilla.redhat.com/show_bug.cgi?id=1388923

(In reply to Mike Burns from comment #2)
> RHEL bug that dropped docker-registry: 
> https://bugzilla.redhat.com/show_bug.cgi?id=1388923

That bug says docker-registry is unmaintained since long time and docker-distribution should be used.
So bug is in tripleo that it uses docker-registry at all.
On the RPM level, docker-distribution has correct Obsoletes/Provides:
 http://pkgs.devel.redhat.com/cgit/rpms/docker-distribution/tree/docker-distribution.spec?h=extras-rhel-7.3#n54
but they missed to add compat docker-registry.service symlink :(

> So bug is in tripleo that it uses docker-registry at all.

ok, that's already fixed https://github.com/openstack/puppet-tripleo/commit/a21f1a193a7076a335d82f47cab04340da9c93ff

Why is it failing then in Ocata ?!

(In reply to Alan Pevec from comment #4)
> > So bug is in tripleo that it uses docker-registry at all.
> 
> ok, that's already fixed
> https://github.com/openstack/puppet-tripleo/commit/
> a21f1a193a7076a335d82f47cab04340da9c93ff
> 
> Why is it failing then in Ocata ?!

The other half of the change was only in master:

https://review.openstack.org/#/c/440842/

Linking another patch that has to be backported alongside, to prevent introducing this bug:

https://bugs.launchpad.net/tripleo/+bug/1666459

Adding a note here as I hit it on two environments which needed slightly different workarounds, so may be helpful for anyone else working here today as I wasted a few runs getting them both to pass. 

----
Starting with OSP10 and upgrading to OSP11, the openstack undercloud upgrade fails like:

    2017-03-03 07:04:18,606 INFO: ^[[1;31mError: /Stage[main]/Main/Service[docker-registry]/ensure: change from stopped to running failed: Systemd start for docker-registry failed!

To fix this you need the missing patch as per comment #5 so you can do:

    # https://review.openstack.org/#/c/440842/ Move Docker registry setup into its own profile
    curl https://review.openstack.org/changes/440842/revisions/current/patch?download | \
        base64 -d | sudo patch  -d /usr/share/instack-undercloud/ -p2

and then the openstack undercloud upgrade should complete.


----
Starting at OSP10 from scratch (new deployment), the openstack undercloud install fails like:

    2017-03-03 07:25:28,926 INFO: 2017-03-03 07:25:28 - ^[[1;31mError: Could not start Service[docker-registry]: Execution of '/bin/systemctl start docker-registry' returned 5: Failed to start docker-registry.service: Unit not found.^[[0m

Here the patch from 440842 above doesn't apply cleanly so I manually applied it at /usr/share/instack-undercloud/puppet-stack-config/puppet-stack.config.pp

    [root@instack puppet-stack-config]# diff puppet-stack-config.pp puppet-stack-config.pp.ORIG 
    435c435,444
    <   include ::tripleo::profile::base::docker_registry
    ---
    >   package{'docker-registry': }
    >   augeas { 'docker-registry':
    >     context => '/files/etc/sysconfig/docker-registry',
    >     changes => 'set REGISTRY_PORT 8787',
    >     notify  => Service['docker-registry'],
    >   }
    >   service { 'docker-registry':
    >     ensure  => running,
    >     require => Package['docker-registry'],
    >   }

Here I also applied the two patches that add the docker registry into puppet-tripleo as they were missing (the profile::base::docker_registry):

    # https://review.openstack.org/#/c/441124 Add Docker Registry profile
    curl https://review.openstack.org/changes/441124/revisions/current/patch?download | \
        base64 -d | sudo patch  -d /usr/share/openstack-puppet/modules/tripleo/ -p1
&&
    # https://review.openstack.org/#/c/441130/ Stop accidentally removing docker-distribution
    curl https://review.openstack.org/changes/441130/revisions/current/patch?download | \
        base64 -d | sudo patch  -d /usr/share/openstack-puppet/modules/tripleo/ -p1

Both applied cleanly for me today and then the undercloud install completes.

I have the same issue on OSP9 and here are my results of patch application failure (440842):

curl https://review.openstack.org/changes/441124/revisions/current/patch?download | base64 -d | sudo patch  -d /usr/share/openstack-puppet/modules/tripleo/ -p1
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  4864    0  4864    0     0   3806      0 --:--:--  0:00:01 --:--:--  3808
patching file manifests/profile/base/docker_registry.pp
patching file templates/docker_distribution/registry_config.yml.erb


curl https://review.openstack.org/changes/441130/revisions/current/patch?download |  base64 -d | sudo patch  -d /usr/share/openstack-puppet/modules/tripleo/ -p1
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1740    0  1740    0     0    341      0 --:--:--  0:00:05 --:--:--   435
patching file manifests/profile/base/docker_registry.pp


curl https://review.openstack.org/changes/440842/revisions/current/patch?download | base64 -d | sudo patch  -d /usr/share/instack-undercloud/ -p2
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  2468    0  2468    0     0   1315      0 --:--:--  0:00:01 --:--:--  1314
patching file puppet-stack-config/puppet-stack-config.pp
Hunk #1 FAILED at 515.
1 out of 1 hunk FAILED -- saving rejects to file puppet-stack-config/puppet-stack-config.pp.rej

Facing same problem with undercloud isntall on OSP9

Facing this on OSP 10 GA.

I tried the patches on an already failed OSP10 GA undercloud install and that failed as well.  I then cleaned my undercloud machine (rebuilt the RHEL OS) and redeployed a newer build with the fix in it and I was able to get passed the undercloud install.  I would recommend as a current workaround to deploy the latest with the fix rather than trying to patch.

Yes, this puddle worked for me 2017-03-03.1

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1245