Bug 1563628 - [UPDATE osp13 update from core paddle 2018-03-16.1 fails with error resources.WorkflowTasks_Step2_Execution: ERROR
Summary: [UPDATE osp13 update from core paddle 2018-03-16.1 fails with error resources...
Keywords:
Status: CLOSED INSUFFICIENT_DATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: rhosp-director
Version: 13.0 (Queens)
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: ---
Assignee: Jiri Stransky
QA Contact: Raviv Bar-Tal
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-04-04 11:09 UTC by Raviv Bar-Tal
Modified: 2018-04-26 12:20 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-04-23 20:38:23 UTC
Target Upstream Version:

Attachments (Terms of Use)
/home/stack (660.00 KB, application/x-tar)
2018-04-04 11:15 UTC, Raviv Bar-Tal 		no flags Details
ceph sosreport (10.93 MB, application/x-xz)
2018-04-04 11:26 UTC, Raviv Bar-Tal 		no flags Details
compute logs (11.42 MB, application/x-xz)
2018-04-04 11:29 UTC, Raviv Bar-Tal 		no flags Details
controller logs (18.64 MB, application/x-xz)
2018-04-04 11:30 UTC, Raviv Bar-Tal 		no flags Details
Add an attachment (proposed patch, testcase, etc.)

Description Raviv Bar-Tal 2018-04-04 11:09:33 UTC 
Description of problem:
trying to update osp13 from 2018-03-16.1 to 2018-03-29.1 fails 
2018-04-04 10:39:58Z [overcloud-AllNodesDeploySteps-25fgux5nj775.WorkflowTasks_Step2_Execution]: UPDATE_COMPLETE  The Resource WorkflowTasks_Step2_Execution requires replacement.
2018-04-04 10:39:59Z [overcloud-AllNodesDeploySteps-25fgux5nj775.WorkflowTasks_Step2_Execution]: CREATE_IN_PROGRESS  state changed
2018-04-04 10:43:11Z [overcloud-AllNodesDeploySteps-25fgux5nj775.WorkflowTasks_Step2_Execution]: CREATE_FAILED  resources.WorkflowTasks_Step2_Execution: ERROR
2018-04-04 10:43:11Z [overcloud-AllNodesDeploySteps-25fgux5nj775]: UPDATE_FAILED  Resource CREATE failed: resources.WorkflowTasks_Step2_Execution: ERROR
2018-04-04 10:43:11Z [AllNodesDeploySteps]: UPDATE_FAILED  resources.AllNodesDeploySteps: Resource CREATE failed: resources.WorkflowTasks_Step2_Execution: ERROR
2018-04-04 10:43:11Z [overcloud]: UPDATE_FAILED  Resource UPDATE failed: resources.AllNodesDeploySteps: Resource CREATE failed: resources.WorkflowTasks_Step2_Execution: ERROR

 Stack overcloud UPDATE_FAILED 

overcloud.AllNodesDeploySteps.WorkflowTasks_Step2_Execution:
  resource_type: OS::Mistral::ExternalResource
  physical_resource_id: 2110b7a1-08e8-4062-b56a-99c5ad5cfde7
  status: CREATE_FAILED
  status_reason: |
    resources.WorkflowTasks_Step2_Execution: ERROR
Heat Stack update failed.


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. install osp13 puddle 2018-03-16.1
2. run the update procedure
3.

Actual results:
update failed

Expected results:


Additional info:
Comment 1 Raviv Bar-Tal 2018-04-04 11:15:00 UTC 
Created attachment 1417216 [details]
/home/stack
Comment 2 Raviv Bar-Tal 2018-04-04 11:26:02 UTC 
Created attachment 1417239 [details]
ceph sosreport
Comment 3 Raviv Bar-Tal 2018-04-04 11:29:29 UTC 
Created attachment 1417241 [details]
compute logs
Comment 4 Raviv Bar-Tal 2018-04-04 11:30:31 UTC 
Created attachment 1417243 [details]
controller logs
Comment 5 Jiri Stransky 2018-04-13 09:23:05 UTC 
Hi Raviv,

can you please provide some more info?

1) Mistral logs from undercloud would be useful (/var/log/mistral/*).

2) What were the `overcloud deploy` and `overcloud update` commands that were ran.

3) Was this an environment with Ceph? The error looks like it could be related to ceph-ansible, as WorkflowTasks_Step2_Execution is where ceph-ansible runs when it's enabled.

I understand we may not have the environment anymore but in case we hit this again, please add the info.
Comment 6 Lukas Bezdicka 2018-04-16 12:40:25 UTC 
Please add logs from /var/log/mistral on undercloud
Comment 7 Carlos Camacho 2018-04-23 13:53:58 UTC 
Hey Raviv can you add last logs Lukas asked?

Thanks!
Comment 8 John Fulton 2018-04-23 20:38:23 UTC 
Hi Raviv,

I'm just closing this for now, but I will be happy to re-open if you provide the data requested in 10 days ago in #5.

Thanks,
  John
Comment 9 Raviv Bar-Tal 2018-04-26 12:20:11 UTC 
Sure, I have mover to a newer build, which doe's not had this problem.
Note You need to log in before you can comment on or make changes to this bug.