Bug 1594367

Summary:	OSP13 minor update: all containers restarted even when Docker RPM is not updated
Product:	Red Hat OpenStack	Reporter:	Assaf Muller <amuller>
Component:	openstack-tripleo-heat-templates	Assignee:	Jiri Stransky <jstransk>
Status:	CLOSED ERRATA	QA Contact:	Raviv Bar-Tal <rbartal>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	13.0 (Queens)	CC:	amuller, asimonel, bcafarel, beagles, bhaley, ccamacho, chjones, chrisw, ekuris, jschluet, jstransk, kgiusti, lmarsh, majopela, mburns, mcornea, michele, nyechiel, sasha, sclewis, sgolovat, srevivo, tfreger
Target Milestone:	z3	Keywords:	Triaged, ZStream
Target Release:	13.0 (Queens)
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	openstack-tripleo-heat-templates-8.0.7-2.el7ost	Doc Type:	Bug Fix
Doc Text:	In prior versions, the conditions for checking whether a Docker daemon requires a restart were too strict. As a result, the Docker daemon and all containers were restarted whenever the Docker configuration changed or when the Docker RPM was updated. With this release, the conditions are relaxed to prevent unnecessary container restarts. Use the "live restore" functionality for configuration changes to make sure the Docker daemon and all containers are restarted when Docker RPM is updated, but not when the Docker configuration is changed.	Story Points:	---
Clone Of:	1589684	Environment:
Last Closed:	2018-11-13 22:26:39 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1589684
Bug Blocks:

Comment 4 Jiri Stransky 2018-06-26 12:50:31 UTC

Currently the containers are stopped if we detect either config change or a RPM update for docker.

Perhaps we could make the stops/failovers occur less frequently by either:

A) Only checking for RPM update. (Just config changes probably wouldn't cause that Docker wouldn't be able to manage previous containers after its restart. (?))

B) Going a bit further and actually trying to parse what kind of Docker RPM change are we doing during the update. Perhaps we could get away with keeping the containers running if we only change the patch number of RPM, but not the version number. But can we depend on patch-number-only RPM updates being "safe" and never causing the issue where we lose the ability to manage the containers which were left running?

C) There was also a suggestion that we'd only stop containers managed by Paunch and Pacemaker. (Also we'd have to think about software managed by external installers like ceph-ansible, so the approach would probably end up being "stop everything except the Neutron-managed containers".) I think we haven't yet completely ruled out the possiblility of the persisting containers still becoming unmanageable though.

Comment 10 Jiri Stransky 2018-09-11 15:20:16 UTC

Posted a Queens-only patch (Rocky+ has this done differently and already doesn't seem to stop containers on config change of Docker).

I tested by editing Docker config by hand and running minor update. The config got set to the state dictated by Puppet, Docker service got restarted (visible in `systemctl status docker` uptime), while containers remained up (visible in `docker ps` uptime). There didn't seem to be any duplicate services running (looked e.g. via `pgrep -a nova-compute`).

Comment 19 errata-xmlrpc 2018-11-13 22:26:39 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:3587