Bug 1571858

Summary: FFU: Deploy step wasn't started during 'openstack overcloud upgrade run --roles Controller --skip-tags validation'
Product: Red Hat OpenStack Reporter: Chris Janiszewski <cjanisze>
Component: python-tripleoclientAssignee: Lukas Bezdicka <lbezdick>
Status: CLOSED DUPLICATE QA Contact: Gurenko Alex <agurenko>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 13.0 (Queens)CC: hbrock, jslagle, jstransk, mbracho, mburns, mcornea, morazi
Target Milestone: rcKeywords: Triaged
Target Release: 13.0 (Queens)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-05-09 15:40:14 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
mistral-logs none

Description Chris Janiszewski 2018-04-25 14:33:40 UTC
Created attachment 1426722 [details]
mistral-logs

Description of problem:
Performing FFU upgrade from OSP10 -> OSP13
Executing step - 'openstack overcloud upgrade run --roles Controller --skip-tags validation' .. that completes without error, but Deploy steps hasn't been started on overcloud:

 u'TASK [include] *****************************************************************',
 u'skipping: [172.16.0.111] => {"changed": false, "skip_reason": "Conditional result was False"}',
 u'skipping: [172.16.0.105] => {"changed": false, "skip_reason": "Conditional result was False"}',
 u'skipping: [172.16.0.109] => {"changed": false, "skip_reason": "Conditional result was False"}',
 u'',
 u'PLAY RECAP *********************************************************************',
 u'172.16.0.105               : ok=446  changed=290  unreachable=0    failed=0   ',
 u'172.16.0.109               : ok=446  changed=290  unreachable=0    failed=0   ',
 u'172.16.0.111               : ok=456  changed=300  unreachable=0    failed=0   ',
 u'']
'headers'



Version-Release number of selected component (if applicable):
OSP10 -> OSP13

How reproducible:
Every time

Steps to Reproduce:
1. Deploy OSP10
2. Perform FFU step until reach 'openstack overcloud upgrade run --roles Controller --skip-tags validation'
3. Playbooks completed but no services or docker containers are running on controller nodes 

Actual results:
[root@chrisj-ffu-controller-0 ~]# docker ps
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?


Expected results:
All openstack services running on the docker containers

Additional info:
The workaround is to rerun the step, which deploys containers properly.
Attaching mistral logs from undercloud

Comment 1 Chris Janiszewski 2018-04-25 15:04:14 UTC
the same occurred when upgrading computes.

Comment 2 Jiri Stransky 2018-04-27 13:48:33 UTC
We suspect these patches might be fixing the issue but we're not sure. More investigation needs to be done:

https://review.openstack.org/#/c/562995/
https://review.openstack.org/#/c/563000/

Comment 4 Lukas Bezdicka 2018-05-09 15:40:14 UTC
Patches already downstream.

*** This bug has been marked as a duplicate of bug 1568714 ***