Bug 1684646 - docker daemon becomes unresponsive during redeploy
Summary: docker daemon becomes unresponsive during redeploy
Keywords:
Status: CLOSED DUPLICATE of bug 1678096
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 13.0 (Queens)
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: ---
Assignee: Emilien Macchi
QA Contact: Gurenko Alex
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-03-01 18:17 UTC by Lars Kellogg-Stedman
Modified: 2019-03-04 01:57 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-03-04 01:57:36 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Lars Kellogg-Stedman 2019-03-01 18:17:57 UTC
Description of problem:

Re-running the 'openstack overcloud deploy ...' command in an OSP 13 overcloud results in the Docker daemon on all overcloud hosts becoming unresponsive.  On all the systems, we see paunch hanging on this:

    /usr/bin/python2 /usr/bin/paunch --debug apply --file /var/lib/tripleo-config/hashed-docker-container-startup-config-step_3.json --config-id tripleo_step3 --managed-by tripleo-ComputeDeployedServer

At this point, commands that do not interact with containers  ("docker info", "docker image ls") work fine, but commands that interact with containers ("docker container ls") hang indefinitely.

Restarting the daemon ("systemctl restart docker") seems to restore the expected behavior, but it's clear if that will allow the deploy operation to complete.


Version-Release number of selected component (if applicable):

OSP 13

Comment 1 Lars Kellogg-Stedman 2019-03-01 18:21:34 UTC
Restarting docker caused the deploy to fail.  This is not the first time we have seen this behavior. I'm going to re-run the deploy to see if it reproduces immediately or not.

Comment 2 Alex Schultz 2019-03-01 20:19:31 UTC
Please provide a sosreport for the system and the steps to reproduce this issue. Does it happen all the time or randomly

Comment 3 Lars Kellogg-Stedman 2019-03-04 01:57:36 UTC
It happens all the time, and appears to be a dupe of either https://bugzilla.redhat.com/show_bug.cgi?id=1671861 or https://bugzilla.redhat.com/show_bug.cgi?id=1678096.  Upgrading to docker-1.13.1-94.gitb2f74b2.el7.x86_64 seems to resolve the issue.

*** This bug has been marked as a duplicate of bug 1678096 ***


Note You need to log in before you can comment on or make changes to this bug.