rhosp-director: HA overcloud +SSL+ceph fails: "YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function "#property#deploy_stderr", data={}]" Environment: puppet-ceph-2.3.1-0.20170805094345.868e6d6.el7ost.noarch instack-undercloud-7.2.1-0.20170729010706.el7ost.noarch ceph-ansible-3.0.0-0.1.rc3.el7cp.noarch openstack-tripleo-heat-templates-7.0.0-0.20170805163048.el7ost.noarch openstack-puppet-modules-10.0.0-0.20170315222135.0333c73.el7.1.noarch Steps to reproduce: Try to deploy overcloud with: openstack overcloud deploy --templates \ --libvirt-type kvm \ -e /usr/share/openstack-tripleo-heat-templates/environments/docker.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/docker-ha.yaml \ -e /home/stack/templates/nodes_data.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/ceph-ansible/ceph-ansible.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \ -e /home/stack/virt/network/network-environment.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/ssl/enable-tls.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/ssl/inject-trust-anchor.yaml \ -e /home/stack/virt/public_vip.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/ssl/tls-endpoints-public-ip.yaml \ -e /home/stack/rhos12.yaml Result: (undercloud) [stack@undercloud-0 ~]$ heat stack-list hWARNING (shell) "heat stack-list" is deprecated, please use "openstack stack list" instead /usr/lib/python2.7/site-packages/requests/packages/urllib3/connection.py:303: SubjectAltNameWarning: Certificate for 192.168.24.2 has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.) SubjectAltNameWarning eat /usr/lib/python2.7/site-packages/requests/packages/urllib3/connection.py:303: SubjectAltNameWarning: Certificate for 192.168.24.2 has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.) SubjectAltNameWarning r+--------------------------------------+------------+---------------+----------------------+--------------+----------------------------------+ | id | stack_name | stack_status | creation_time | updated_time | project | +--------------------------------------+------------+---------------+----------------------+--------------+----------------------------------+ | 894dd12a-9a5b-4104-bbd1-6b25290b5326 | overcloud | CREATE_FAILED | 2017-08-25T02:58:08Z | None | 60cd8093e4ad402c8a714f32d8bd83b1 | +--------------------------------------+------------+---------------+----------------------+--------------+----------------------------------+ (undercloud) [stack@undercloud-0 ~]$ heat resource-list -n5 overcloud |grep -v COMPLE +----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------+-----------------+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+ | resource_name | physical_resource_id | resource_type | resource_status | updated_time | stack_name | +----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------+-----------------+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+ | AllNodesDeploySteps | fa2eb091-2f63-4ab4-af60-51878e36423b | OS::TripleO::PostDeploySteps | CREATE_FAILED | 2017-08-25T02:58:09Z | overcloud | | WorkflowTasks_Step2_Execution | fd4a0dbc-1780-4974-95e6-992d1bebf142 | OS::Mistral::ExternalResource | CREATE_FAILED | 2017-08-25T03:12:23Z | overcloud-AllNodesDeploySteps-dv67rnzvb75e | +----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------+-----------------+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+ (undercloud) [stack@undercloud-0 ~]$ for i in `openstack workflow execution list|awk '/ERROR/ {print $2}'`; do echo $i; for j in $(openstack task execution list $i|awk '/ERROR/ {print $2}'); do openstack task execution result show $j|sed 's/\\n/\n/g'|grep -i error; done; done 7f3133a9-dc1b-4edc-ba58-6c123f9381f8 "result": "Failure caused by error in tasks: create_admin_via_nova create_admin_via_nova [task_ex_id=aec801b8-d0b3-42ae-900c-70481152f1e0] -> Failure caused by error in tasks: create_admin [wf_ex_id=00b245c4-78a3-4ed9-9102-428fc48d9a79, idx=0]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=1f5ade53-4dda-45d0-b0f0-156ef3753d80, idx=1]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=37ed61bf-8b4c-4650-aa54-d4529c225eba, idx=2]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=404c278f-fbee-4d59-89fd-31416adb7dc6, idx=3]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=80a6b760-3899-4f88-a45a-d84fb09bfb92, idx=4]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=93d0b417-ace4-4bb0-81ca-4a06a3128ba5, idx=5]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=db47ca44-4d4d-4562-93ad-d2829ecd9279, idx=6]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=f3d7bf92-0280-4801-a87c-3cf1f4b0b3f4, idx=7]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=aab83ff0-24ae-4d1a-b2fb-6240c1876a93, idx=0]: Failure caused by error in tasks: create_admin [wf_ex_id=00b245c4-78a3-4ed9-9102-428fc48d9a79, idx=0]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=1f5ade53-4dda-45d0-b0f0-156ef3753d80, idx=1]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=37ed61bf-8b4c-4650-aa54-d4529c225eba, idx=2]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=404c278f-fbee-4d59-89fd-31416adb7dc6, idx=3]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=80a6b760-3899-4f88-a45a-d84fb09bfb92, idx=4]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=93d0b417-ace4-4bb0-81ca-4a06a3128ba5, idx=5]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=db47ca44-4d4d-4562-93ad-d2829ecd9279, idx=6]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=f3d7bf92-0280-4801-a87c-3cf1f4b0b3f4, idx=7]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] fd4a0dbc-1780-4974-95e6-992d1bebf142 "result": "Failure caused by error in tasks: enable_ssh_admin enable_ssh_admin [task_ex_id=0860dece-f3e9-4419-8fdc-da201e722806] -> Failure caused by error in tasks: create_admin_via_nova create_admin_via_nova [task_ex_id=aec801b8-d0b3-42ae-900c-70481152f1e0] -> Failure caused by error in tasks: create_admin [wf_ex_id=00b245c4-78a3-4ed9-9102-428fc48d9a79, idx=0]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=1f5ade53-4dda-45d0-b0f0-156ef3753d80, idx=1]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=37ed61bf-8b4c-4650-aa54-d4529c225eba, idx=2]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=404c278f-fbee-4d59-89fd-31416adb7dc6, idx=3]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=80a6b760-3899-4f88-a45a-d84fb09bfb92, idx=4]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=93d0b417-ace4-4bb0-81ca-4a06a3128ba5, idx=5]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=db47ca44-4d4d-4562-93ad-d2829ecd9279, idx=6]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=f3d7bf92-0280-4801-a87c-3cf1f4b0b3f4, idx=7]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=aab83ff0-24ae-4d1a-b2fb-6240c1876a93, idx=0]: Failure caused by error in tasks: create_admin [wf_ex_id=00b245c4-78a3-4ed9-9102-428fc48d9a79, idx=0]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=1f5ade53-4dda-45d0-b0f0-156ef3753d80, idx=1]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=37ed61bf-8b4c-4650-aa54-d4529c225eba, idx=2]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=404c278f-fbee-4d59-89fd-31416adb7dc6, idx=3]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=80a6b760-3899-4f88-a45a-d84fb09bfb92, idx=4]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=93d0b417-ace4-4bb0-81ca-4a06a3128ba5, idx=5]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=db47ca44-4d4d-4562-93ad-d2829ecd9279, idx=6]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=f3d7bf92-0280-4801-a87c-3cf1f4b0b3f4, idx=7]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=f9d4a9ce-199e-41ca-a14d-a6aff0505344, idx=0]: Failure caused by error in tasks: create_admin_via_nova create_admin_via_nova [task_ex_id=aec801b8-d0b3-42ae-900c-70481152f1e0] -> Failure caused by error in tasks: create_admin [wf_ex_id=00b245c4-78a3-4ed9-9102-428fc48d9a79, idx=0]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=1f5ade53-4dda-45d0-b0f0-156ef3753d80, idx=1]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=37ed61bf-8b4c-4650-aa54-d4529c225eba, idx=2]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=404c278f-fbee-4d59-89fd-31416adb7dc6, idx=3]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=80a6b760-3899-4f88-a45a-d84fb09bfb92, idx=4]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=93d0b417-ace4-4bb0-81ca-4a06a3128ba5, idx=5]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=db47ca44-4d4d-4562-93ad-d2829ecd9279, idx=6]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=f3d7bf92-0280-4801-a87c-3cf1f4b0b3f4, idx=7]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=aab83ff0-24ae-4d1a-b2fb-6240c1876a93, idx=0]: Failure caused by error in tasks: create_admin [wf_ex_id=00b245c4-78a3-4ed9-9102-428fc48d9a79, idx=0]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=1f5ade53-4dda-45d0-b0f0-156ef3753d80, idx=1]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=37ed61bf-8b4c-4650-aa54-d4529c225eba, idx=2]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=404c278f-fbee-4d59-89fd-31416adb7dc6, idx=3]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=80a6b760-3899-4f88-a45a-d84fb09bfb92, idx=4]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=93d0b417-ace4-4bb0-81ca-4a06a3128ba5, idx=5]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=db47ca44-4d4d-4562-93ad-d2829ecd9279, idx=6]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=f3d7bf92-0280-4801-a87c-3cf1f4b0b3f4, idx=7]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] aab83ff0-24ae-4d1a-b2fb-6240c1876a93 "result": "Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] "result": "Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] "result": "Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] "result": "Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] "result": "Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] "result": "Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] "result": "Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] "result": "Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] f9d4a9ce-199e-41ca-a14d-a6aff0505344 "result": "Failure caused by error in tasks: create_admin [wf_ex_id=00b245c4-78a3-4ed9-9102-428fc48d9a79, idx=0]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=1f5ade53-4dda-45d0-b0f0-156ef3753d80, idx=1]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=37ed61bf-8b4c-4650-aa54-d4529c225eba, idx=2]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=404c278f-fbee-4d59-89fd-31416adb7dc6, idx=3]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=80a6b760-3899-4f88-a45a-d84fb09bfb92, idx=4]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=93d0b417-ace4-4bb0-81ca-4a06a3128ba5, idx=5]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=db47ca44-4d4d-4562-93ad-d2829ecd9279, idx=6]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}] [wf_ex_id=f3d7bf92-0280-4801-a87c-3cf1f4b0b3f4, idx=7]: Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}]
Could you please check overcloud controller node if there is "[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed" error same as here: https://bugs.launchpad.net/tripleo/+bug/1712836 If it is, fix for that issue is addressed by: https://review.openstack.org/#/c/496639/
[root@overcloud-controller-0 ~]# journalctl -u os-collect-config|grep -i CERTIFICATE_VERIFY_FAILED Aug 25 03:20:21 overcloud-controller-0 os-collect-config[3150]: requests.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:579)
The fix applies to tripleo-quickstart-extras for the other environments the solution is to provide from an environment file the CAMap parameter populated as following: parameter_defaults: CAMap: overcloud-ca: overcloud_ca_pem_contents undercloud-ca: undercloud_ca_pem_contents Setting needinfo on Juan who can help us better and confirm/amend the above.
I see the certificate validation fails while trying to reach undercloud Swift on the public endpoint: Aug 25 03:20:21 overcloud-controller-0 os-collect-config[3150]: [2017-08-25 03:20:21,363] (heat-config) [ERROR] [2017-08-25 03:20:21,312] (heat-config-notify) [DEBUG] Signaling to https://192.168.24.2:13808/v1/AUTH_60cd8093e4ad402c8a714f32d8bd83b1/create_admin-5d77912b-aaa4-4c4b-9379-fc449aab7d44/fb3187a9-e4bc-4507-a685-c8442c0e297f?temp_url_sig=bb02a6bc9a6d2a05ed8c2a106df9b1704c331f93&temp_url_expires=1503649197 via PUT I'm wondering why are we using the public endpoint and not the internal one which is http and I'd expect it to be used for inter node communication: (undercloud) [stack@undercloud-0 ~]$ openstack catalog show swift +-----------+-------------------------------------------------------------------------------+ | Field | Value | +-----------+-------------------------------------------------------------------------------+ | endpoints | regionOne | | | admin: http://192.168.24.3:8080 | | | regionOne | | | internal: http://192.168.24.3:8080/v1/AUTH_60cd8093e4ad402c8a714f32d8bd83b1 | | | regionOne | | | public: https://192.168.24.2:13808/v1/AUTH_60cd8093e4ad402c8a714f32d8bd83b1 | | | | | id | 61b8436abcfd41309ac1a205cb6a3f11 | | name | swift | | type | object-store | +-----------+-------------------------------------------------------------------------------+
Confirm that when overcloud+SSL deployed on undercloud without SSL - the issue doesn't reproduce.
I'll update the upstream documentation to include this extra step (trusting the undercloud's certificate). And post the link to the docs review here.
*** Bug 1486916 has been marked as a duplicate of this bug. ***
I think the issue reproduced on BM latest puddle : openstack stack failures list overcloud --long overcloud.AllNodesDeploySteps.WorkflowTasks_Step2_Execution: resource_type: OS::Mistral::ExternalResource physical_resource_id: 75cf38af-9d79-47f0-88a8-b4c7d6b60979 status: CREATE_FAILED status_reason: | resources.WorkflowTasks_Step2_Execution: ERROR Mistral/engine.log: -------------------- Workflow 'tripleo.deployment.v1.deploy_on_server' [RUNNING -> ERROR, msg=Failed to run task [error=Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function "#property#deploy_stderr", data={}], wf=tripleo.deployment.v1.deploy_on_server, task=send_message]: Heat-engine.log: ------------------ 2017-09-01 14:19:24.130 29148 DEBUG heat.engine.resources.openstack.mistral.external_resource [req-b78cbe81-d89b-46d0-a0d1-cc1dd04c3fd5 - admin - default default] Mistral execution 75cf38af-9d79-47f0-88a8-b4c7d6b60979 is in state ERROR _check_execution /usr/lib/python2.7/site-packages/heat/engine/resources/openstack/mistral/external_resource.py:159 2017-09-01 14:19:24.131 29148 INFO heat.engine.resource [req-b78cbe81-d89b-46d0-a0d1-cc1dd04c3fd5 - admin - default default] CREATE: MistralExternalResource "WorkflowTasks_Step2_Execution" [75cf38af-9d79-47f0-88a8-b4c7d6b60979] Stack "overcloud-AllNodesDeploySteps-f5fcphgjkrpl" [a32a5339-de2a-44ae-b5a2-b92e55c486de] 2017-09-01 14:19:24.131 29148 ERROR heat.engine.resource Traceback (most recent call last): 2017-09-01 14:19:24.131 29148 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 831, in _action_recorder 2017-09-01 14:19:24.131 29148 ERROR heat.engine.resource yield 2017-09-01 14:19:24.131 29148 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 939, in _do_action 2017-09-01 14:19:24.131 29148 ERROR heat.engine.resource yield self.action_handler_task(action, args=handler_args) 2017-09-01 14:19:24.131 29148 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/scheduler.py", line 351, in wrapper 2017-09-01 14:19:24.131 29148 ERROR heat.engine.resource step = next(subtask) 2017-09-01 14:19:24.131 29148 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 890, in action_handler_task 2017-09-01 14:19:24.131 29148 ERROR heat.engine.resource done = check(handler_data) 2017-09-01 14:19:24.131 29148 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/resources/openstack/mistral/external_resource.py", line 251, in check_create_complete 2017-09-01 14:19:24.131 29148 ERROR heat.engine.resource return self._check_action(self.CREATE, execution_id) 2017-09-01 14:19:24.131 29148 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/resources/openstack/mistral/external_resource.py", line 204, in _check_action 2017-09-01 14:19:24.131 29148 ERROR heat.engine.resource success, output = self._check_execution(action, execution_id) 2017-09-01 14:19:24.131 29148 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/resources/openstack/mistral/external_resource.py", line 171, in _check_execution 2017-09-01 14:19:24.131 29148 ERROR heat.engine.resource action=action) 2017-09-01 14:19:24.131 29148 ERROR heat.engine.resource ResourceFailure: resources.WorkflowTasks_Step2_Execution: ERROR 2017-09-01 14:19:24.131 29148 ERROR heat.engine.resource
After applying the recommended steps, I got Overcloud deploy SUCCESS
moving to documentation
Updating bug summary to more clearly capture the request for documentation update.
Chris, Guilio, Chen, Omri, et al... do we have a solution that everyone agrees works now? If so, please identify a prime that I, and our documentation writer, can work with and we can get this defined and into our schedule. Thanks!
Derek, I believe we are still discussing the right solution. I have opened another bug to track it from the product perspective: https://bugzilla.redhat.com/show_bug.cgi?id=1611807
*** Bug 1615189 has been marked as a duplicate of this bug. ***
*** Bug 1501779 has been marked as a duplicate of this bug. ***
Hi, so I got that for osp13 this is going to be documentation only. Please, make sure you're updating the Upgrade documentation as well as we got bz#1501779. For osp14 I understood it was automatic, so upgrade from osp13 to osp14 shouldn't be an issue here, right ? Thanks,
Reviewing with Dan
Looks good. Merged!
Raised BZ requesting QE validation: https://bugzilla.redhat.com/show_bug.cgi?id=1638607
QE successfully tested procedure.
Republished guide with new procedure: https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/12/html-single/director_installation_and_usage/index#sect-Configure_overcloud_nodes_to_trust_the_undercloud_CA