Created attachment 1540273 [details] Full tempest test failure log Description of problem: Deployment with skip-deploy-identifier flag and performing stack update or scale out, causing tempest tests failures due fact, that nova discovery is not triggered Version-Release number of selected component (if applicable): How reproducible: Deployment with skip-deploy-identifier flag Steps to Reproduce: 1. 2. 3. Actual results: tempest test fails with error: tempest.exceptions.BuildErrorException: Server de052b96-fcfd-4a2e-b743-d565bc571c49 failed to build and is in ERROR status Details: {u'message': u"Host 'compute-3.localdomain' is not mapped to any cell", u'code': 400, u'created': u'2019-02-27T07:31:03Z'} Expected results: Additional info:
Verified by running a scale out and running the tempest test: 1. Disable a compute service ---------------------------- (overcloud) [stack@undercloud-0 ~]$ openstack compute service list +----+------------------+--------------------------+----------+---------+-------+----------------------------+ | ID | Binary | Host | Zone | Status | State | Updated At | +----+------------------+--------------------------+----------+---------+-------+----------------------------+ | 2 | nova-conductor | controller-1.localdomain | internal | enabled | up | 2019-06-20T10:20:21.000000 | | 11 | nova-conductor | controller-2.localdomain | internal | enabled | up | 2019-06-20T10:20:24.000000 | | 17 | nova-conductor | controller-0.localdomain | internal | enabled | up | 2019-06-20T10:20:22.000000 | | 23 | nova-consoleauth | controller-1.localdomain | internal | enabled | up | 2019-06-20T10:20:25.000000 | | 26 | nova-consoleauth | controller-0.localdomain | internal | enabled | up | 2019-06-20T10:20:20.000000 | | 29 | nova-consoleauth | controller-2.localdomain | internal | enabled | up | 2019-06-20T10:20:24.000000 | | 32 | nova-scheduler | controller-1.localdomain | internal | enabled | up | 2019-06-20T10:20:25.000000 | | 44 | nova-scheduler | controller-0.localdomain | internal | enabled | up | 2019-06-20T10:20:22.000000 | | 47 | nova-scheduler | controller-2.localdomain | internal | enabled | up | 2019-06-20T10:20:20.000000 | | 68 | nova-compute | compute-0.localdomain | nova | enabled | up | 2019-06-20T10:20:21.000000 | | 80 | nova-compute | compute-1.localdomain | nova | enabled | up | 2019-06-20T10:20:24.000000 | +----+------------------+--------------------------+----------+---------+-------+----------------------------+ (overcloud) [stack@undercloud-0 ~]$ openstack compute service set compute-1.localdomain nova-compute --disable (overcloud) [stack@undercloud-0 ~]$ openstack compute service list +----+------------------+--------------------------+----------+----------+-------+----------------------------+ | ID | Binary | Host | Zone | Status | State | Updated At | +----+------------------+--------------------------+----------+----------+-------+----------------------------+ | 2 | nova-conductor | controller-1.localdomain | internal | enabled | up | 2019-06-20T10:22:01.000000 | | 11 | nova-conductor | controller-2.localdomain | internal | enabled | up | 2019-06-20T10:22:04.000000 | | 17 | nova-conductor | controller-0.localdomain | internal | enabled | up | 2019-06-20T10:22:02.000000 | | 23 | nova-consoleauth | controller-1.localdomain | internal | enabled | up | 2019-06-20T10:22:05.000000 | | 26 | nova-consoleauth | controller-0.localdomain | internal | enabled | up | 2019-06-20T10:22:00.000000 | | 29 | nova-consoleauth | controller-2.localdomain | internal | enabled | up | 2019-06-20T10:22:04.000000 | | 32 | nova-scheduler | controller-1.localdomain | internal | enabled | up | 2019-06-20T10:22:05.000000 | | 44 | nova-scheduler | controller-0.localdomain | internal | enabled | up | 2019-06-20T10:22:02.000000 | | 47 | nova-scheduler | controller-2.localdomain | internal | enabled | up | 2019-06-20T10:22:00.000000 | | 68 | nova-compute | compute-0.localdomain | nova | enabled | up | 2019-06-20T10:22:01.000000 | | 80 | nova-compute | compute-1.localdomain | nova | disabled | up | 2019-06-20T10:22:04.000000 | +----+------------------+--------------------------+----------+----------+-------+----------------------------+ (undercloud) [stack@undercloud-0 ~]$ openstack stack list +--------------------------------------+------------+----------------------------------+-----------------+----------------------+----------------------+ | ID | Stack Name | Project | Stack Status | Creation Time | Updated Time | +--------------------------------------+------------+----------------------------------+-----------------+----------------------+----------------------+ | d3c72b3c-e625-468d-9d52-91c55850f9cb | overcloud | 23eb5c82d4f24460a7b297aef27c35dc | UPDATE_COMPLETE | 2019-06-17T12:12:44Z | 2019-06-18T17:41:37Z | +--------------------------------------+------------+----------------------------------+-----------------+----------------------+----------------------+ (undercloud) [stack@undercloud-0 ~]$ openstack server list +--------------------------------------+--------------+--------+------------------------+----------------+------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+--------------+--------+------------------------+----------------+------------+ | c60458ce-b589-4f31-bd0b-5b74efcc2871 | controller-0 | ACTIVE | ctlplane=192.168.24.10 | overcloud-full | controller | | bbf1bc82-249b-46c7-8f69-03e83f65b91c | compute-0 | ACTIVE | ctlplane=192.168.24.16 | overcloud-full | compute | | 524fa118-0adb-4976-a821-a879cf0d4b44 | controller-2 | ACTIVE | ctlplane=192.168.24.6 | overcloud-full | controller | | fb245279-7bc7-4e42-add5-d8ca602cda1e | compute-1 | ACTIVE | ctlplane=192.168.24.13 | overcloud-full | compute | | 14dbce2f-fb48-4977-814b-efa4cea8cdc6 | controller-1 | ACTIVE | ctlplane=192.168.24.14 | overcloud-full | controller | +--------------------------------------+--------------+--------+------------------------+----------------+------------+ 3. Delete node -------------- (undercloud) [stack@undercloud-0 ~]$ cat overcloud_node_delete.sh #!/bin/bash openstack overcloud node delete --stack d3c72b3c-e625-468d-9d52-91c55850f9cb \ --templates -e /home/stack/virt/config_lvm.yaml \ -e /home/stack/virt/config_lvm.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \ -e /home/stack/virt/network/network-environment.yaml \ -e /home/stack/virt/inject-trust-anchor.yaml \ -e /home/stack/virt/hostnames.yml \ -e /home/stack/virt/nodes_data.yaml \ -e ~/containers-prepare-parameter.yaml \ -e /home/stack/virt/extra_templates.yaml \ fb245279-7bc7-4e42-add5-d8ca602cda1e (undercloud) [stack@undercloud-0 ~]$ time bash overcloud_node_delete.sh Deleting the following nodes from stack overcloud: - fb245279-7bc7-4e42-add5-d8ca602cda1e Waiting for messages on queue 'tripleo' with no timeout. real 21m40.710s user 0m0.913s sys 0m0.218s (undercloud) [stack@undercloud-0 ~]$ openstack server list # note that compute-1 is removed. +--------------------------------------+--------------+--------+------------------------+----------------+------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+--------------+--------+------------------------+----------------+------------+ | c60458ce-b589-4f31-bd0b-5b74efcc2871 | controller-0 | ACTIVE | ctlplane=192.168.24.10 | overcloud-full | controller | | bbf1bc82-249b-46c7-8f69-03e83f65b91c | compute-0 | ACTIVE | ctlplane=192.168.24.16 | overcloud-full | compute | | 524fa118-0adb-4976-a821-a879cf0d4b44 | controller-2 | ACTIVE | ctlplane=192.168.24.6 | overcloud-full | controller | | 14dbce2f-fb48-4977-814b-efa4cea8cdc6 | controller-1 | ACTIVE | ctlplane=192.168.24.14 | overcloud-full | controller | +--------------------------------------+--------------+--------+------------------------+----------------+------------+ (overcloud) [stack@undercloud-0 ~]$ openstack compute service list +----+------------------+--------------------------+----------+----------+-------+----------------------------+ | ID | Binary | Host | Zone | Status | State | Updated At | +----+------------------+--------------------------+----------+----------+-------+----------------------------+ | 2 | nova-conductor | controller-1.localdomain | internal | enabled | up | 2019-06-20T14:54:35.000000 | | 11 | nova-conductor | controller-2.localdomain | internal | enabled | up | 2019-06-20T14:54:38.000000 | | 17 | nova-conductor | controller-0.localdomain | internal | enabled | up | 2019-06-20T14:54:36.000000 | | 23 | nova-consoleauth | controller-1.localdomain | internal | enabled | up | 2019-06-20T14:54:38.000000 | | 26 | nova-consoleauth | controller-0.localdomain | internal | enabled | up | 2019-06-20T14:54:35.000000 | | 29 | nova-consoleauth | controller-2.localdomain | internal | enabled | up | 2019-06-20T14:54:38.000000 | | 32 | nova-scheduler | controller-1.localdomain | internal | enabled | up | 2019-06-20T14:54:30.000000 | | 44 | nova-scheduler | controller-0.localdomain | internal | enabled | up | 2019-06-20T14:54:39.000000 | | 47 | nova-scheduler | controller-2.localdomain | internal | enabled | up | 2019-06-20T14:54:37.000000 | | 68 | nova-compute | compute-0.localdomain | nova | enabled | up | 2019-06-20T14:54:33.000000 | | 80 | nova-compute | compute-1.localdomain | nova | disabled | down | 2019-06-20T11:05:44.000000 | +----+------------------+--------------------------+----------+----------+-------+----------------------------+ (overcloud) [stack@undercloud-0 ~]$ openstack compute service delete 80 4. re-add the node ------------------ (overcloud) [stack@undercloud-0 ~]$ cat /home/stack/virt/nodes_data.yaml parameter_defaults: ControllerCount: 3 OvercloudControlFlavor: controller ComputeCount: 2 OvercloudComputeFlavor: compute (overcloud) [stack@undercloud-0 ~]$ cat overcloud_deploy_blacklist.sh #!/bin/bash openstack overcloud deploy \ --timeout 100 \ --templates /usr/share/openstack-tripleo-heat-templates \ --stack overcloud \ --libvirt-type kvm \ --skip-deployer-identifier \ --ntp-server clock.redhat.com \ -e /home/stack/virt/config_lvm.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \ -e /home/stack/virt/network/network-environment.yaml \ -e /home/stack/virt/inject-trust-anchor.yaml \ -e /home/stack/virt/hostnames.yml \ -e /home/stack/virt/nodes_data.yaml \ -e ~/containers-prepare-parameter.yaml \ -e /home/stack/virt/extra_templates.yaml \ -e /home/stack/blacklist.yaml \ --log-file overcloud_deployment_47.log
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:1672