Bug 1537726

Summary:	redeploy_node_certificates.yaml restarts docker daemon
Product:	OpenShift Container Platform	Reporter:	Borja Aranda <farandac>
Component:	Installer	Assignee:	Scott Dodson <sdodson>
Status:	CLOSED ERRATA	QA Contact:	Gaoyun Pei <gpei>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	3.7.0	CC:	abutcher, aos-bugs, dmoessne, gpei, joboyer, jokerman, mmccomas
Target Milestone:	---
Target Release:	3.9.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:	The docker daemon was incorrectly restarted when redeploying node certificates. This is only necessary when deploying a new CA and can safely be skipped which ensures that running pods are not restarted when updating node certificates.	Story Points:	---
Clone Of:
Clones:	1542162 (view as bug list)		Environment:
Last Closed:	2018-03-28 14:21:18 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1542162

Description Borja Aranda 2018-01-23 18:34:44 UTC

Description of problem:
When running the playbook redeploy-node-certificates.yml, every node in the cluster gets its certificates regenerated and a new proper kubeconfig.

Once the task is completed, the playbook restarts docker and atomic-openshift-node, this causes an unnecessary downtime in nodes since the only component loading the kubeconfig is atomic-openshift-node.

The docker restart happens here: https://github.com/openshift/openshift-ansible/blob/9a405010c5a656f89866906d29866ba98493e91b/playbooks/openshift-node/private/restart.yml#L10

As this seems to be a task called from different playbooks, it would be great to add some kind of check or flag. If this task is called from redeploy_certs, then do not trigger the docker restart.

Actual results:
docker daemon gets restarted in playbook redeploy-node-certificates.yml

Expected results:
docker daemon not restarted. Only atomic-openshift-node restart should be needed to load the new kubeconfig and certificates.
By doing this, this playbook can run without downtime.

Comment 1 Scott Dodson 2018-01-23 18:52:16 UTC

Andrew,

Do you agree there's no need to restart docker when re-deploying node certificates? Seems like this would only be necessary when a new CA is generated so that docker picks up that change.

Comment 2 Andrew Butcher 2018-01-23 19:03:15 UTC

Scott,

I agree. Restarting docker should be skipped when we have not replaced the CA certificate.

Comment 4 Scott Dodson 2018-01-24 17:47:13 UTC

Master PR https://github.com/openshift/openshift-ansible/pull/6855

Comment 6 Gaoyun Pei 2018-02-05 06:02:04 UTC

Verify this bug with openshift-ansible-3.9.0-0.36.0.git.0.da68f13.el7.noarch


Run node cert redeployment playbook, docker was not restart during redeployment.
ansible-playbook -i host /usr/share/ansible/openshift-ansible/playbooks/openshift-node/redeploy-certificates.yml

PLAY [Restart nodes] ********************************************************************************************************************************************************

TASK [Gathering Facts] ******************************************************************************************************************************************************
ok: [ec2-34-207-99-213.compute-1.amazonaws.com]

TASK [Restart docker] *******************************************************************************************************************************************************
skipping: [ec2-34-207-99-213.compute-1.amazonaws.com] => {"changed": false, "skip_reason": "Conditional result was False"}

Comment 9 errata-xmlrpc 2018-03-28 14:21:18 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0489