Created attachment 1423690 [details] stack failures list long Description of problem: Cannot deploy OSP13 with 1 controller and 13 computes and custom hostnames (undercloud) [stack@undercloud-0 ~]$ cat virt/custom_hostnames.yaml parameter_defaults: HostnameMap: controller-0: ctrl-0 compute-0: test_compute-00 compute-1: test_compute-01 compute-2: test_compute-02 compute-3: test_compute-03 compute-4: test_compute-04 compute-5: test_compute-05 compute-6: test_compute-06 compute-7: test_compute-07 compute-8: test_compute-08 compute-9: test_compute-09 compute-10: test_compute-10 compute-11: test_compute-11 compute-12: test_compute-12 (undercloud) [stack@undercloud-0 ~]$ cat overcloud_deploy.sh #!/bin/bash openstack overcloud deploy \ --timeout 100 \ --templates /usr/share/openstack-tripleo-heat-templates \ --stack overcloud \ --libvirt-type kvm \ --ntp-server clock.redhat.com \ -e /home/stack/virt/config_lvm.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \ -e /home/stack/virt/network/network-environment.yaml \ -e /home/stack/virt/inject-trust-anchor.yaml \ -e /home/stack/virt/hostnames.yml \ -e /home/stack/virt/debug.yaml \ -e /home/stack/virt/custom_hostnames.yaml \ -e /home/stack/virt/nodes_data.yaml \ -e /home/stack/virt/docker-images.yaml \ --log-file overcloud_deployment_26.log (undercloud) [stack@undercloud-0 ~]$ ironic node-list The "ironic" CLI is deprecated and will be removed in the S* release. Please use the "openstack baremetal" CLI instead. /usr/lib/python2.7/site-packages/requests/packages/urllib3/connection.py:344: SubjectAltNameWarning: Certificate for 192.168.24.2 has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.) SubjectAltNameWarning /usr/lib/python2.7/site-packages/requests/packages/urllib3/connection.py:344: SubjectAltNameWarning: Certificate for 192.168.24.2 has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.) SubjectAltNameWarning +--------------------------------------+--------------+--------------------------------------+-------------+--------------------+-------------+ | UUID | Name | Instance UUID | Power State | Provisioning State | Maintenance | +--------------------------------------+--------------+--------------------------------------+-------------+--------------------+-------------+ | 6d14a163-6188-4651-be31-a5946a39bd39 | compute-0 | 414d24be-d833-4369-a18c-299085efe499 | power on | active | False | | a1412bc1-5cc8-45ba-a48b-76b034672327 | compute-1 | fd2a35c6-eba6-4ddf-b055-c102e08cebf7 | power on | active | False | | 3645103e-2a0a-4e2d-a3c7-a8617414d248 | compute-10 | 3f9eafd0-9e9a-4740-9c93-615cad7ac879 | power on | active | False | | 243a05c1-e6be-4036-8789-f9296775b7ac | compute-11 | b7122211-68e0-48db-b250-47f9f440a87b | power on | active | False | | eb8b1db4-f3aa-4fdc-b6ae-0048ff582f27 | compute-12 | 85ae8ecc-0c2c-477b-a63d-ab9645ecfb4f | power on | active | False | | d81b743f-f620-4c72-a0b3-fdd17adc33f0 | compute-2 | 12ce919a-1903-4983-a712-a5acc955e395 | power on | active | False | | 74985c9b-7057-436e-b2f5-2011c367ed65 | compute-3 | 7e501818-daed-4771-be72-4bcfbb5e13f7 | power on | active | False | | 8ff208f8-b91e-4220-8f57-9ea86f21c483 | compute-4 | e22fec0f-bf59-43da-a14a-bd4ff53ae5e3 | power on | active | False | | 90e78973-fa17-421c-b1ee-f7be1dbd967b | compute-5 | 2e88847b-8937-4d04-847d-66f3be823c43 | power on | active | False | | a1faca21-e271-4224-97d8-68b7e3e4cc32 | compute-6 | 205e447e-465b-46aa-84ee-5cff1ab5fbd6 | power on | active | False | | b01648d8-3510-4924-b601-aec172d429b4 | compute-7 | 2e3d6656-dcd3-4f88-9166-00e8b69db99b | power on | active | False | | 5c518d12-12f6-47c7-943d-07f366e4ac1c | compute-8 | 6a084974-939e-4f21-b34b-37cb0e9760c7 | power on | active | False | | 02ced649-afcb-4acf-b7d3-4529843fd1c8 | compute-9 | 3795f5db-5c70-4749-bd75-3d58654a8bfe | power on | active | False | | 7d39ff0a-3806-491d-88d1-663cf0d6dca8 | controller-0 | ae13333a-fe51-4c80-9348-3d2c8d90ed73 | power on | active | False | +--------------------------------------+--------------+--------------------------------------+-------------+--------------------+-------------+ TASK [Debug output for task which failed: Run docker-puppet tasks (generate config) during step 1] *** fatal: [localhost]: FAILED! => { "changed": false, "failed_when_result": true, "outputs.stdout_lines|default([])|union(outputs.stderr_lines|default([]))": [ "2018-04-18 17:39:54,970 INFO: 17092 -- Running docker-puppet", "2018-04-18 17:39:54,971 INFO: 17092 -- Service compilation completed.", "2018-04-18 17:39:54,971 INFO: 17092 -- Starting multiprocess configuration steps. Using 3 processes.", "2018-04-18 17:39:55,018 INFO: 17094 -- Starting configuration of nova_libvirt using image 192.168.24.1:8787/rhosp13/openstack-nova-compute:2018-04-10.2", "2018-04-18 17:39:55,018 INFO: 17093 -- Starting configuration of ceilometer using image 192.168.24.1:8787/rhosp13/openstack-ceilometer-central:2018-04-10.2", "2018-04-18 17:39:55,019 INFO: 17095 -- Starting configuration of crond using image 192.168.24.1:8787/rhosp13/openstack-cron:2018-04-10.2", "2018-04-18 17:39:55,020 INFO: 17094 -- Removing container: docker-puppet-nova_libvirt", "2018-04-18 17:39:55,020 INFO: 17093 -- Removing container: docker-puppet-ceilometer", "2018-04-18 17:39:55,028 INFO: 17095 -- Removing container: docker-puppet-crond", "2018-04-18 17:39:55,103 INFO: 17093 -- Pulling image: 192.168.24.1:8787/rhosp13/openstack-ceilometer-central:2018-04-10.2", "2018-04-18 17:39:55,108 INFO: 17095 -- Pulling image: 192.168.24.1:8787/rhosp13/openstack-cron:2018-04-10.2", "2018-04-18 17:39:55,111 INFO: 17094 -- Pulling image: 192.168.24.1:8787/rhosp13/openstack-nova-compute:2018-04-10.2", "2018-04-18 17:40:49,869 INFO: 17095 -- Removing container: docker-puppet-crond", "2018-04-18 17:40:50,107 INFO: 17095 -- Finished processing puppet configs for crond", "2018-04-18 17:40:50,108 INFO: 17095 -- Starting configuration of neutron using image 192.168.24.1:8787/rhosp13/openstack-neutron-server:2018-04-10.2", "2018-04-18 17:40:50,125 INFO: 17095 -- Removing container: docker-puppet-neutron", "2018-04-18 17:40:50,954 INFO: 17095 -- Pulling image: 192.168.24.1:8787/rhosp13/openstack-neutron-server:2018-04-10.2", "2018-04-18 17:41:16,214 ERROR: 17093 -- Failed running docker-puppet.py for ceilometer", "2018-04-18 17:41:16,214 ERROR: 17093 -- /usr/bin/docker-current: Error response from daemon: transport is closing.", "", "2018-04-18 17:41:16,214 INFO: 17093 -- Finished processing puppet configs for ceilometer", "2018-04-18 17:41:16,215 INFO: 17093 -- Starting configuration of iscsid using image 192.168.24.1:8787/rhosp13/openstack-iscsid:2018-04-10.2", "2018-04-18 17:41:16,227 INFO: 17093 -- Removing container: docker-puppet-iscsid", "2018-04-18 17:41:16,338 INFO: 17093 -- Pulling image: 192.168.24.1:8787/rhosp13/openstack-iscsid:2018-04-10.2", "2018-04-18 17:42:24,685 INFO: 17093 -- Removing container: docker-puppet-iscsid", "2018-04-18 17:42:25,101 INFO: 17093 -- Finished processing puppet configs for iscsid", "2018-04-18 17:42:51,947 INFO: 17095 -- Removing container: docker-puppet-neutron", "2018-04-18 17:42:52,030 INFO: 17095 -- Finished processing puppet configs for neutron", "2018-04-18 17:43:21,843 INFO: 17094 -- Removing container: docker-puppet-nova_libvirt", "2018-04-18 17:43:21,915 INFO: 17094 -- Finished processing puppet configs for nova_libvirt", "2018-04-18 17:43:21,917 ERROR: 17092 -- ERROR configuring ceilometer" ] } to retry, use: --limit @/var/lib/heat-config/heat-config-ansible/64c08a67-374d-4d86-954b-2de4383623bf_playbook.retry PLAY RECAP ********************************************************************* localhost : ok=19 changed=12 unreachable=0 failed=1 deploy_stderr: | Version-Release number of selected component (if applicable): OSP13 with puddle 2018-04-10 How reproducible: ALWAYS Steps to Reproduce: 1.create custom_hostnames.yaml parameter_defaults: HostnameMap: controller-0: ctrl-0 compute-0: test_compute-00 compute-1: test_compute-01 compute-2: test_compute-02 compute-3: test_compute-03 compute-4: test_compute-04 compute-5: test_compute-05 compute-6: test_compute-06 compute-7: test_compute-07 compute-8: test_compute-08 compute-9: test_compute-09 compute-10: test_compute-10 compute-11: test_compute-11 compute-12: test_compute-12 2. Deploy OSP13 using infrared infrared tripleo-overcloud -v --introspect yes --containers yes --tagging yes --post no --deployment-files virt --version 13 --deploy yes --build passed_phase2 --custom-hostnames /home/ahre/infrared/custom_hostnames.yaml --specific-node-ids yes Actual results: Overcloud deployment were fail Expected results: Overcloud deployment were pass Additional info:
"2018-04-18 17:41:14,383 ERROR: 17076 -- /usr/bin/docker-current: Error response from daemon: transport is closing.", Not sure it's related to custom hostnames, but the logs provided don't show any other trace. Could you please run "export DEBUG=true && python /var/lib/docker-puppet/docker-puppet.py" and show the output. Thanks
Created attachment 1423706 [details] docker-puppet.py from failed nodes
Created attachment 1486717 [details] Failures
The provided debug log did not have the error in it. Closing for now, feel free to reopen with additional logs if this is still an issue.