Bug 1637013

Summary: Deployments fail on AllNodesDeploySteps.WorkflowTasks_Step2_Execution, no ceph-install-workflow.log
Product: Red Hat OpenStack Reporter: Udi Kalifon <ukalifon>
Component: rhosp-directorAssignee: RHOS Maint <rhos-maint>
Status: CLOSED DUPLICATE QA Contact: Gurenko Alex <agurenko>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 13.0 (Queens)CC: dbecker, johfulto, mburns, morazi
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-10-08 16:07:39 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Udi Kalifon 2018-10-08 13:08:34 UTC
Description of problem:
After upgrading the undercloud from OSP10 to OSP13, I am trying to deploy but consistently failing:

overcloud.AllNodesDeploySteps.WorkflowTasks_Step2_Execution:
  resource_type: OS::TripleO::WorkflowSteps
  physical_resource_id: 79ff0259-ed46-47f9-8f39-346d0b4783b0
  status: CREATE_FAILED
  status_reason: |
    resources.WorkflowTasks_Step2_Execution: ERROR

I can't find any helpful errors in the logs on the undercloud or the overcloud.


Version-Release number of selected component (if applicable):
openstack-tripleo-0.0.8-0.3.4de13b3git.el7ost.noarch
openstack-tripleo-common-containers-8.6.3-13.el7ost.noarch


How reproducible:
100%


Steps to Reproduce:
1. Install OSP10 but don't deploy an overcloud
2. Upgrade to OSP13 according to: https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/13/html/fast_forward_upgrades/assembly-preparing_for_openstack-platform_upgrade
3. Try to deploy in OSP13. I tried to deploy 3 controllers + 2 computes + 3 ceph with the command: openstack overcloud deploy --templates openstack-tripleo-heat-templates/ -e openstack-tripleo-heat-templates/environments/docker.yaml -e openstack-tripleo-heat-templates/environments/docker-ha.yaml -e templates/overcloud_images.yaml -e openstack-tripleo-heat-templates/environments/ceph-ansible/ceph-ansible.yaml -e openstack-tripleo-heat-templates/environments/network-isolation.yaml -e custom.yaml


Actual results:
Deployment fails.


Additional info:
I will attach the sosreports from the undercloud.

Comment 2 John Fulton 2018-10-08 16:07:39 UTC
unpacked your sosreport and I see you're hitting a failure on the create_admin_via_nova task [1]. 

I see your deployment command doesn't contain a line to inject the trust anchor though OSP13 UC expects a trust anchor by default unless you set generate_service_certificate=false in your undercloud.conf. So inject a trust anchor as per the docs or disable SSL. For more info see the links in the bug that I marked this a duplicate of. 


[1]

cd sosreport-undercloud-0.redhat.local-20181008083720/var/log/mistral
grep create_admin_via_nova engine.log 

  create_admin_via_nova [task_ex_id=4705c6e6-9ee6-4ca7-bc56-c1b5dc2b2e48] -> Failure caused by error in tasks: create_admin
2018-10-08 06:49:33.830 1444 INFO mistral.engine.engine_server [req-aa16f4e3-9518-4cf9-9cd2-815c3eb1cfc0 2ba9be06538349ebaa8f3218da832239 7cc61979642f43d5a0c93b0f518ccf39 - - -] Received RPC request 'on_action_complete'[action_ex_id=d6c1744b-caad-4d41-9492-0f046c506130, result=Result [data=None, error=Failure caused by error in tasks: create_admin_via_nova

*** This bug has been marked as a duplicate of bug 1565861 ***