Bug 1410218

Summary: [RPM] Docker-distribution is installed instead of docker-registry
Product: Red Hat Quickstart Cloud Installer Reporter: Thom Carlin <tcarlin>
Component: DistributionAssignee: Ann Marie Rubin <arubin>
Status: CLOSED ERRATA QA Contact: Tasos Papaioannou <tpapaioa>
Severity: low Docs Contact:
Priority: unspecified    
Version: 1.1CC: athomas, etaylor, fkluknav, jesusr, jmatthew, jmontleo, lsm5, mburns, qci-bugzillas, rhartman, tpapaioa
Target Milestone: ---Keywords: Triaged
Target Release: 1.1   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1428932 (view as bug list) Environment:
Last Closed: 2017-02-28 01:42:54 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1428932    

Description Thom Carlin 2017-01-04 19:32:33 UTC
Description of problem:

During RPM install following the QCI 1.1 guide, saw "Error: Could not start Service[docker-registry]: Execution of '/bin/systemctl start docker-registry' returned 5: Failed to start docker-registry.service: Unit not found."

Version-Release number of selected component (if applicable):

1.1.2-1

How reproducible:

100%

Steps to Reproduce:
1. Section 4.1.3 Steps 2 b
2. Examine /var/log/fusor-undercloud-install.log

Actual results:

Error: Could not start Service[docker-registry]: Execution of '/bin/systemctl start docker-registry' returned 5: Failed to start docker-registry.service: Unit not found.

Expected results:

Successful install

Additional info:

No errors during RPM installs, no errors in yum databases (verified using package-cleanup options), and all dependencies satisfied

Problem isolated to:
A. yum provides docker-registry
On an ISO system, returns docker-registry
On a RPM system, returns 1 docker-distribution and 2 different docker-registry rpms

Workaround:

yum --exclude=docker-distribution install qci-tripleo

Comment 2 Lokesh Mandvekar 2017-01-05 14:36:16 UTC
docker-registry has been obsoleted by docker-distribution by upstream and also in RHEL. Please let us know if you explicity need docker-registry.

Comment 3 Thom Carlin 2017-01-05 14:49:19 UTC
Adding mburns for input

Comment 4 Frantisek Kluknavsky 2017-01-05 14:50:37 UTC
It's systemd unit file has a different name, which is great if you want to avoid ambiguities, not that great if you want one installer supporting both.

Comment 5 Mike Burns 2017-01-05 15:11:54 UTC
Adding the OpenStack container folks.

Quick look -- it's possible RHEL is not doing the right thing.  

in rhel-extras, I see 

docker-registry-0.9.1-7.el7.x86_64
docker-distribution-2.5.1-1.el7.x86_64

docker-distribution has a provides/obsoletes for docker-registry, but I still managed to install both.

I'm not sure where the requires comes from or the service start comes from, though.  I'd need the yum logs/more details to see where that is happening.  (I suspect puppet, though)

Comment 6 Lokesh Mandvekar 2017-01-05 15:17:36 UTC
Not sure why the obsoletes doesn't work, Ryan do you know if there is a way to retire a package in rhel?

Comment 7 Jason Montleon 2017-01-05 15:20:09 UTC
rpm -q --whatprovides  /usr/share/instack-undercloud/puppet-stack-config/puppet-stack-config.pp
instack-undercloud-5.1.0-3.el7ost.noarch

It looks like this is getting copied to /etc/puppet/manifests/puppet-stack-config.pp

if str2bool(hiera('enable_docker_registry', true)) {
  package{'docker-registry': }
...


I don't recall explicitly enabling docker registry; not sure if it can be disabled in the undercloud install answers file, which would work find for us, I believe as we're not deploying containerized compute nodes.

WRT docker-distribution if it replaces docker-registry and you want to rename the systemd unit file maybe a unit file alias would be appropriate?

Finally, if both can be installed at the same time maybe an Obsoletes or similar is missing on the rpm package itself as well?

"Units can be aliased (have an alternative name), by creating a symlink from the new name to the existing name in one of the unit search paths. For example, systemd-networkd.service has the alias dbus-org.freedesktop.network1.service, created during installation as the symlink /usr/lib/systemd/system/dbus-org.freedesktop.network1.service. In addition, unit files may specify aliases through the Alias= directive in the [Install] section; those aliases are only effective when the unit is enabled. When the unit is enabled, symlinks will be created for those names, and removed when the unit is disabled. For example, reboot.target specifies Alias=ctrl-alt-del.target, so when enabled it will be invoked whenever CTRL+ALT+DEL is pressed. Alias names may be used in commands like enable, disable, start, stop, status, …, and in unit dependency directives Wants=, Requires=, Before=, After=, …, with the limitation that aliases specified through Alias= are only effective when the unit is enabled. Aliases cannot be used with the preset command."

Comment 8 Jason Montleon 2017-01-05 15:46:35 UTC
I think your obsoletes needs to be:
Obsoletes:      docker-registry < 0.9.1-8

rather than:
Obsoletes:      docker-registry <= 0.9.1-7

.el7 dist tag gets added to the release so it's actually greater than 7

After rebuilding the package and doing a yum update with a local repo I get:
Obsoleting Packages
docker-distribution.x86_64                                                                                              2.5.1-2.el7                                                                                              docker        
    docker-registry.x86_64                                                                                              0.9.1-7.el7                                                                                              @anaconda/10.0
[root@undercloud SPECS]# yum update
Loaded plugins: langpacks, package_upload, product-id, search-disabled-repos, subscription-manager
rhel-7-server-extras-rpms                                                                                                                                                                                               | 2.0 kB  00:00:00     
rhel-7-server-optional-rpms                                                                                                                                                                                             | 2.0 kB  00:00:00     
rhel-7-server-rpms                                                                                                                                                                                                      | 2.0 kB  00:00:00     
Resolving Dependencies
--> Running transaction check
---> Package docker-distribution.x86_64 0:2.5.1-2.el7 will be obsoleting
---> Package docker-registry.x86_64 0:0.9.1-7.el7 will be obsoleted
--> Finished Dependency Resolution

Dependencies Resolved

===============================================================================================================================================================================================================================================
 Package                                                           Arch                                                 Version                                                     Repository                                            Size
===============================================================================================================================================================================================================================================
Installing:
 docker-distribution                                               x86_64                                               2.5.1-2.el7                                                 docker                                               3.5 M
     replacing  docker-registry.x86_64 0.9.1-7.el7

Transaction Summary
===============================================================================================================================================================================================================================================
Install  1 Package

Total download size: 3.5 M
Is this ok [y/d/N]:

Comment 9 Jason Montleon 2017-01-05 16:24:22 UTC
I created  a link called docker-registry.service pointing to docker-distribution.service and added the line 'Alias=docker-registry.service' to the docker-distribution.service file.

ls -l /usr/lib/systemd/system/docker-registry.service 
lrwxrwxrwx. 1 root root 27 Jan  5 11:14 /usr/lib/systemd/system/docker-registry.service -> docker-distribution.service
[root@undercloud system]# cat /usr/lib/systemd/system/docker-registry.service
[Unit]
Description=v2 Registry server for Docker

[Service]
Type=simple
ExecStart=/usr/bin/registry serve /etc/docker-distribution/registry/config.yml
Restart=on-failure

[Install]
WantedBy=multi-user.target
Alias=docker-registry.service


I can then manage the docker-distribution service with the docker-registry name.

[root@undercloud system]# systemctl stop docker-registry
[root@undercloud system]# service docker-registry start
Redirecting to /bin/systemctl start  docker-registry.service
[root@undercloud system]# chkconfig docker-registry off
Note: Forwarding request to 'systemctl disable docker-registry.service'.
Removed symlink /etc/systemd/system/multi-user.target.wants/docker-distribution.service.
Removed symlink /etc/systemd/system/docker-registry.service.
[root@undercloud system]# systemctl enable docker-registry
Created symlink from /etc/systemd/system/docker-registry.service to /usr/lib/systemd/system/docker-distribution.service.
Created symlink from /etc/systemd/system/multi-user.target.wants/docker-distribution.service to /usr/lib/systemd/system/docker-distribution.service.
[root@undercloud system]# systemctl status docker-registry
● docker-distribution.service - v2 Registry server for Docker
   Loaded: loaded (/usr/lib/systemd/system/docker-distribution.service; enabled; vendor preset: disabled)
   Active: failed (Result: start-limit) since Thu 2017-01-05 11:17:42 EST; 21s ago
 Main PID: 11396 (code=exited, status=1/FAILURE)

Jan 05 11:17:42 undercloud.example.com systemd[1]: Unit docker-distribution.service entered failed state.
Jan 05 11:17:42 undercloud.example.com systemd[1]: docker-distribution.service failed.
Jan 05 11:17:42 undercloud.example.com systemd[1]: docker-distribution.service holdoff time over, scheduling restart.
Jan 05 11:17:42 undercloud.example.com systemd[1]: start request repeated too quickly for docker-distribution.service
Jan 05 11:17:42 undercloud.example.com systemd[1]: Failed to start v2 Registry server for Docker.
Jan 05 11:17:42 undercloud.example.com systemd[1]: Unit docker-distribution.service entered failed state.
Jan 05 11:17:42 undercloud.example.com systemd[1]: docker-distribution.service failed.
[root@undercloud system]# systemctl -t service | grep docker
● docker-distribution.service                loaded failed failed  v2 Registry server for Docker

The service is failing to start because it's trying to bind to port 5000. OSP 10 I think changes the config to point to 8787, but of course this file is moved from /etc/sysconfig/docker-registry to /etc/docker-distribution/registry/config.yml

It looks like the layout of these files has changed as well, so I don't have an answer on how to make this backward compatible without more thought. OSP 10 may need to just have a template for both and lay them down.

Comment 10 Thom Carlin 2017-01-06 19:02:13 UTC
To clarify:

fusor-tripleo (a.k.a. qci-tripleo) has a requires of docker-registry
python-tripleoclient does not

Apologies for any confusion

So to fix the problem if fusor-tripleo is already installed:
* rpm -e --nodeps docker-distribution
* yum -y --exclude=docker-distribution reinstall qci-tripleo

Comment 11 Jason Montleon 2017-01-09 16:27:12 UTC
I've been told that installing docker-registry is optional for OSP but I do not believe it is.

The key looks to be:
grep enable_docker_registry /usr/share/instack-undercloud/puppet-stack-config/puppet-stack-config.pp
if str2bool(hiera('enable_docker_registry', true)) {

If enable_docker_registry is nil we get true here which causes the undercloud installer to try to install and configure docker-registry.

As /etc/hiera.yaml and /etc/puppet/hieradata/puppet-stack-config.yaml get overwritten on every invocation of the undercloud installer there is no way to modify the values in either file.

Looking at /usr/share/instack-undercloud/puppet-stack-config/puppet-stack-config.yaml.template there is no way to set enable_docker_registry there, so it's essentially always undefined and therefore always true.

However it appears that there is a hieradata_override feature with which we can provide an additional hiera file to be copied into /etc/puppet/hieradata and to be added to the hiera.yaml file, so we have a way to work around it.

I should be able to do this with a PR to fusor-undercloud-installer.

I'll also have to remove docker-registry from the fusor-tripleo package.

Comment 14 Tasos Papaioannou 2017-01-17 21:14:55 UTC
The pull requests don't appear to have been merged yet.

Comment 17 Tasos Papaioannou 2017-01-24 18:57:16 UTC
Verified on QCIOOO-10.0-RHEL-7-20170123.t.1.


docker-registry is no longer a requirement for the fusor-tripleo rpm:

# rpm -qR fusor-tripleo | grep docker
#

And no docker* rpms are installed after running fusor-undercloud-installer from an rpm install:

# rpm -qa docker\*
#

Comment 19 errata-xmlrpc 2017-02-28 01:42:54 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:0335

Comment 20 Ed Taylor 2017-03-06 17:30:13 UTC
I found a quick workaround for this, when installing undercloud, is to link /usr/lib/systemd/system/docker-registry.service
to
/usr/lib/systemd/system/docker-distribution.service

The undercloud then installed successfully without errors.