Bug 1679858 - [upgrades][13->14] Overcloud upgrade getting stuck on docker restart
Summary: [upgrades][13->14] Overcloud upgrade getting stuck on docker restart
Keywords:
Status: CLOSED DUPLICATE of bug 1671861
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: python-docker
Version: 14.0 (Rocky)
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: ---
: ---
Assignee: RHOS Maint
QA Contact: RHOS Maint
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-02-22 04:30 UTC by Archit Modi
Modified: 2019-06-12 13:00 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-06-12 13:00:53 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Archit Modi 2019-02-22 04:30:18 UTC
+++ This bug is a clone of BZ #1671861 https://bugzilla.redhat.com/show_bug.cgi?id=1671861 +++

Description of problem: Upgrades from Rhos-13 to Rhos-14 are getting stuck for hours at overcloud upgrade step due to docker unable to respond/restart


Version-Release number of selected component (if applicable):
Docker version 1.13.1, build 07f3374/1.13.1

Steps to Reproduce:
1. Start upgrade from Rhos-13 to Rhos-14

Actual results:
Deployment error'd/Timed out


Expected results:
Deployment successful


Additional info:

Comment 2 Archit Modi 2019-02-22 11:20:56 UTC
Overcloud upgrade gets stuck in the below task at the docker restart step:
2019-02-20 06:33:15,247 p=3197 u=mistral |  TASK [Remove any container using the same Mariadb image] ***********************
2019-02-20 06:33:15,247 p=3197 u=mistral |  Wednesday 20 February 2019  06:33:15 -0500 (0:00:00.682)       0:24:33.361 **** 

Docker gets hung and has be aborted by Ctrl+C as shown:
[heat-admin@controller-0 ~]$ sudo docker ps
^C

Comment 4 Archit Modi 2019-03-08 15:31:07 UTC
We are facing the same issue with the latest passed_phase1 puddle:

[root@controller-1 ~]# time timeout 30 docker ps

real    0m30.004s
user    0m0.014s
sys     0m0.028s
[root@controller-2 ~]# time timeout 30 docker ps

real    0m30.004s
user    0m0.017s
sys 0m0.016s

(undercloud) [stack@undercloud-0 ~]$ ssh heat-admin.24.14
Warning: Permanently added '192.168.24.14' (ECDSA) to the list of known hosts.
Last login: Thu Mar  7 12:13:36 2019 from 192.168.24.1
[heat-admin@controller-0 ~]$ sudo docker ps

Comment 7 Jose Luis Franco 2019-06-12 13:00:53 UTC
We can close this bug now that the problematic docker package has been updated

*** This bug has been marked as a duplicate of bug 1671861 ***


Note You need to log in before you can comment on or make changes to this bug.