Bug 1478452 - openstack-containers: OSP12 deployment with ceph fails, throwing "esources.WorkflowTasks_Step2_Execution: ERROR"
openstack-containers: OSP12 deployment with ceph fails, throwing "esources.Wo...
Status: CLOSED NOTABUG
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-containers (Show other bugs)
12.0 (Pike)
Unspecified Unspecified
medium Severity medium
: rc
: 12.0 (Pike)
Assigned To: John Fulton
Yogev Rabl
Andrew Burden
: Triaged
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2017-08-04 11:03 EDT by Alexander Chuzhoy
Modified: 2017-08-09 21:34 EDT (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-08-09 21:34:41 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
upstream documentation for tripleo ceph deploys with or without contianers (43.10 KB, text/html)
2017-08-04 16:47 EDT, John Fulton
no flags Details

  None (edit)
Description Alexander Chuzhoy 2017-08-04 11:03:25 EDT
openstack-containers: OSP12 deployment with ceph fails, throwing "esources.WorkflowTasks_Step2_Execution: ERROR"

Environment:
openstack-puppet-modules-10.0.0-0.20170315222135.0333c73.el7.1.noarch
openstack-tripleo-heat-templates-7.0.0-0.20170721174554.el7ost.noarch
instack-undercloud-7.1.1-0.20170714211622.el7ost.noarch


Steps to reproduce:
1. Deploy undercloud
2. Apply patch:
https://review.openstack.org/#/c/465066/
3) Attemp to deploy OC with ceph


Result:
the deployment fails:


+--------------------------------------+------------+---------------+----------------------+--------------+----------------------------------+
| 00a36745-8f80-4738-ab70-4018930b98f6 | overcloud  | CREATE_FAILED | 2017-08-04T14:31:33Z | None         | 7035a8ef3af44cba80060cd25b2267c7 |
+--------------------------------------+------------+---------------+----------------------+--------------+----------------------------------+
(undercloud) [stack@undercloud-0 ~]$ heat resource-list -n5 overcloud|grep -v COMPLE
WARNING (shell) "heat resource-list" is deprecated, please use "openstack stack resource list" instead
/usr/lib/python2.7/site-packages/requests/packages/urllib3/connection.py:303: SubjectAltNameWarning: Certificate for 192.168.24.2 has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)
  SubjectAltNameWarning
/usr/lib/python2.7/site-packages/requests/packages/urllib3/connection.py:303: SubjectAltNameWarning: Certificate for 192.168.24.2 has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)
  SubjectAltNameWarning
+----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------+-----------------+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
| resource_name                                | physical_resource_id                                                                                                                                                                 | resource_type                                                                                                                    | resource_status | updated_time         | stack_name                                                                                                                                               |
+----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------+-----------------+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
| AllNodesDeploySteps                          | cd7aae1d-975e-4e44-b838-e017ffd9ce78                                                                                                                                                 | OS::TripleO::PostDeploySteps                                                                                                     | CREATE_FAILED   | 2017-08-04T14:31:35Z | overcloud                                                                                                                                                |
| WorkflowTasks_Step2_Execution                |                                                                                                                                                                                      | OS::Mistral::ExternalResource                                                                                                    | CREATE_FAILED   | 2017-08-04T14:47:07Z | overcloud-AllNodesDeploySteps-b6zdpfhnlhoc                                                                                                               |
+----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------+-----------------+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
(undercloud) [stack@undercloud-0 ~]$ ssh ^C
(undercloud) [stack@undercloud-0 ~]$ openstack stack failures list overcloud
overcloud.AllNodesDeploySteps.WorkflowTasks_Step2_Execution:
  resource_type: OS::Mistral::ExternalResource
  physical_resource_id: 
  status: CREATE_FAILED
  status_reason: |
    resources.WorkflowTasks_Step2_Execution: ERROR
Comment 1 Martin André 2017-08-04 12:01:52 EDT
It seems to be related to an outdated tripleo-common package.

The ceph-install workflow upstream [1] takes the ansible_playbook_verbosity, ceph_ansible_extra_vars, and ceph_ansible_playbook arguments -- matching what the ceph patch merged last week provides -- while the one from installed on the undercloud (openstack-tripleo-common-7.3.1-0.20170720003002.f0ef9ac.el7ost.noarch) only takes ceph_ansible as input.

I also noticed that the ceph-ansible package was missing from the ceph nodes. Not sure what is supposed to install the rpm.

[1] https://github.com/openstack/tripleo-common/blob/master/workbooks/ceph-ansible.yaml
Comment 4 John Fulton 2017-08-04 16:04:39 EDT
root cause: ceph-ansible needs to be installed on the undercloud
Comment 5 John Fulton 2017-08-04 16:38:32 EDT
The deployment command [1] has some issues. It's just cause there are new changes and it's in an upstream doc. I will share here. 

For example, storage-environment.yaml (like puppet-ceph.yaml) will trigger puppet-ceph while environments/ceph-ansible/ceph-ansible.yaml and they should not be used together. 

How puppet-ceph or ceph-ansible may be used is covered in the doc. 

[1] 
timeout 100m openstack overcloud deploy \
--templates /usr/share/openstack-tripleo-heat-templates \
--libvirt-type kvm \
--ntp-server clock.redhat.com \
-e /usr/share/openstack-tripleo-heat-templates/environments/storage-environment.yaml \
-e /home/stack/virt/internal.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \
-e /home/stack/virt/network/network-environment.yaml \
-e /home/stack/virt/enable-tls.yaml \
-e /home/stack/virt/inject-trust-anchor.yaml \
-e /home/stack/virt/public_vip.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/tls-endpoints-public-ip.yaml \
-e /home/stack/virt/hostnames.yml \
-e /usr/share/openstack-tripleo-heat-templates/environments/docker.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/docker-ha.yaml \
-e /home/stack/virt/debug.yaml \
-e /home/stack/virt/nodes_data.yaml \
-e /home/stack/virt/workaround_params.yaml \
-e /home/stack/virt/docker-images.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/ceph-ansible/ceph-ansible.yaml --log-file overcloud_deployment_49.log
Comment 6 John Fulton 2017-08-04 16:47 EDT
Created attachment 1309180 [details]
upstream documentation for tripleo ceph deploys with or without contianers
Comment 7 Alexander Chuzhoy 2017-08-08 10:54:03 EDT
Filed a doc bug:
https://bugzilla.redhat.com/show_bug.cgi?id=1479449
Comment 8 John Fulton 2017-08-09 21:34:41 EDT
The cause of this issue was not following the new process to deploy with ceph-ansible. The new process will be addressed by the docbug https://bugzilla.redhat.com/show_bug.cgi?id=1479449.

Note You need to log in before you can comment on or make changes to this bug.