Created attachment 1568631 [details] tempest-run.log Description of problem: Deploy RHos-15 with the mentioned topology and run the following tempest test which fails: tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_resize_server_confirm_from_stopped core_puddle: RHOS_TRUNK-15.0-RHEL-8-20190509.n.1 Topology: 1 Controller, 2 Computes Steps to Reproduce: 1. Deploy RHOS-15 with mentioned topology 2. Run the following tempest test: tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_resize_server_confirm_from_stopped Actual results: Response - Headers: {'date': 'Sat, 11 May 2019 21:20:54 GMT', 'server': 'Apache', 'content-length': '1701', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version,Accept-Encoding', 'x-openstack-request-id': 'req-f78f7829-029f-4971-ac82-dca85c1e04dd', 'x-compute-request-id': 'req-f78f7829-029f-4971-ac82-dca85c1e04dd', 'connection': 'close', 'content-type': 'application/json', 'status': '200', 'content-location': 'http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e'} Body: b'{"server": {"id": "fc272b73-ff3e-4448-ab85-49bcec47546e", "name": "tempest-ServerActionsTestJSON-server-1454314435", "status": "SHUTOFF", "tenant_id": "59e8e567688941f89efae0bbf6a0d3a6", "user_id": "2e0cd1029f94430aa692fbaf536afbf1", "metadata": {}, "hostId": "c3eac17a1808415e7d9c75be02baf4105d1e0e1e8221a8a29542c6e8", "image": {"id": "0e69750d-552c-4f0e-a9d6-797bc415aa7f", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/images/0e69750d-552c-4f0e-a9d6-797bc415aa7f"}]}, "flavor": {"id": "0f5c781f-d41f-46a5-a744-10f9f5c69200", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/flavors/0f5c781f-d41f-46a5-a744-10f9f5c69200"}]}, "created": "2019-05-11T21:15:10Z", "updated": "2019-05-11T21:16:12Z", "addresses": {"tempest-ServerActionsTestJSON-1252015544-network": [{"version": 4, "addr": "10.100.0.8", "OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}, {"version": 4, "addr": "10.0.0.247", "OS-EXT-IPS:type": "floating", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}]}, "accessIPv4": "", "accessIPv6": "", "links": [{"rel": "self", "href": "http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}, {"rel": "bookmark", "href": "http://10.0.0.137:8774/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}], "OS-DCF:diskConfig": "MANUAL", "OS-EXT-AZ:availability_zone": "nova", "config_drive": "", "key_name": "tempest-keypair-638249548", "OS-SRV-USG:launched_at": "2019-05-11T21:15:57.000000", "OS-SRV-USG:terminated_at": null, "security_groups": [{"name": "tempest-securitygroup--1644861636"}], "OS-EXT-STS:task_state": null, "OS-EXT-STS:vm_state": "stopped", "OS-EXT-STS:power_state": 0, "os-extended-volumes:volumes_attached": []}}' 2019-05-11 21:20:56,211 215815 INFO [tempest.lib.common.rest_client] Request (ServerActionsTestJSON:tearDown): 200 GET http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e 0.308s 2019-05-11 21:20:56,212 215815 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-OpenStack-Nova-API-Version': '2.1', 'X-Auth-Token': '<omitted>'} Body: None Response - Headers: {'date': 'Sat, 11 May 2019 21:20:55 GMT', 'server': 'Apache', 'content-length': '1701', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version,Accept-Encoding', 'x-openstack-request-id': 'req-680ed90e-6e60-4ee4-a69f-386bfd1b1fd5', 'x-compute-request-id': 'req-680ed90e-6e60-4ee4-a69f-386bfd1b1fd5', 'connection': 'close', 'content-type': 'application/json', 'status': '200', 'content-location': 'http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e'} Body: b'{"server": {"id": "fc272b73-ff3e-4448-ab85-49bcec47546e", "name": "tempest-ServerActionsTestJSON-server-1454314435", "status": "SHUTOFF", "tenant_id": "59e8e567688941f89efae0bbf6a0d3a6", "user_id": "2e0cd1029f94430aa692fbaf536afbf1", "metadata": {}, "hostId": "c3eac17a1808415e7d9c75be02baf4105d1e0e1e8221a8a29542c6e8", "image": {"id": "0e69750d-552c-4f0e-a9d6-797bc415aa7f", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/images/0e69750d-552c-4f0e-a9d6-797bc415aa7f"}]}, "flavor": {"id": "0f5c781f-d41f-46a5-a744-10f9f5c69200", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/flavors/0f5c781f-d41f-46a5-a744-10f9f5c69200"}]}, "created": "2019-05-11T21:15:10Z", "updated": "2019-05-11T21:16:12Z", "addresses": {"tempest-ServerActionsTestJSON-1252015544-network": [{"version": 4, "addr": "10.100.0.8", "OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}, {"version": 4, "addr": "10.0.0.247", "OS-EXT-IPS:type": "floating", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}]}, "accessIPv4": "", "accessIPv6": "", "links": [{"rel": "self", "href": "http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}, {"rel": "bookmark", "href": "http://10.0.0.137:8774/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}], "OS-DCF:diskConfig": "MANUAL", "OS-EXT-AZ:availability_zone": "nova", "config_drive": "", "key_name": "tempest-keypair-638249548", "OS-SRV-USG:launched_at": "2019-05-11T21:15:57.000000", "OS-SRV-USG:terminated_at": null, "security_groups": [{"name": "tempest-securitygroup--1644861636"}], "OS-EXT-STS:task_state": null, "OS-EXT-STS:vm_state": "stopped", "OS-EXT-STS:power_state": 0, "os-extended-volumes:volumes_attached": []}}' 2019-05-11 21:20:57,540 215815 INFO [tempest.lib.common.rest_client] Request (ServerActionsTestJSON:tearDown): 200 GET http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e 0.322s 2019-05-11 21:20:57,540 215815 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-OpenStack-Nova-API-Version': '2.1', 'X-Auth-Token': '<omitted>'} Body: None Response - Headers: {'date': 'Sat, 11 May 2019 21:20:57 GMT', 'server': 'Apache', 'content-length': '1701', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version,Accept-Encoding', 'x-openstack-request-id': 'req-bbc1e6b5-b558-4984-929c-a88a27ed8619', 'x-compute-request-id': 'req-bbc1e6b5-b558-4984-929c-a88a27ed8619', 'connection': 'close', 'content-type': 'application/json', 'status': '200', 'content-location': 'http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e'} Body: b'{"server": {"id": "fc272b73-ff3e-4448-ab85-49bcec47546e", "name": "tempest-ServerActionsTestJSON-server-1454314435", "status": "SHUTOFF", "tenant_id": "59e8e567688941f89efae0bbf6a0d3a6", "user_id": "2e0cd1029f94430aa692fbaf536afbf1", "metadata": {}, "hostId": "c3eac17a1808415e7d9c75be02baf4105d1e0e1e8221a8a29542c6e8", "image": {"id": "0e69750d-552c-4f0e-a9d6-797bc415aa7f", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/images/0e69750d-552c-4f0e-a9d6-797bc415aa7f"}]}, "flavor": {"id": "0f5c781f-d41f-46a5-a744-10f9f5c69200", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/flavors/0f5c781f-d41f-46a5-a744-10f9f5c69200"}]}, "created": "2019-05-11T21:15:10Z", "updated": "2019-05-11T21:16:12Z", "addresses": {"tempest-ServerActionsTestJSON-1252015544-network": [{"version": 4, "addr": "10.100.0.8", "OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}, {"version": 4, "addr": "10.0.0.247", "OS-EXT-IPS:type": "floating", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}]}, "accessIPv4": "", "accessIPv6": "", "links": [{"rel": "self", "href": "http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}, {"rel": "bookmark", "href": "http://10.0.0.137:8774/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}], "OS-DCF:diskConfig": "MANUAL", "OS-EXT-AZ:availability_zone": "nova", "config_drive": "", "key_name": "tempest-keypair-638249548", "OS-SRV-USG:launched_at": "2019-05-11T21:15:57.000000", "OS-SRV-USG:terminated_at": null, "security_groups": [{"name": "tempest-securitygroup--1644861636"}], "OS-EXT-STS:task_state": null, "OS-EXT-STS:vm_state": "stopped", "OS-EXT-STS:power_state": 0, "os-extended-volumes:volumes_attached": []}}' 2019-05-11 21:20:58,950 215815 INFO [tempest.lib.common.rest_client] Request (ServerActionsTestJSON:tearDown): 200 GET http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e 0.403s 2019-05-11 21:20:58,950 215815 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-OpenStack-Nova-API-Version': '2.1', 'X-Auth-Token': '<omitted>'} Body: None Response - Headers: {'date': 'Sat, 11 May 2019 21:20:58 GMT', 'server': 'Apache', 'content-length': '1701', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version,Accept-Encoding', 'x-openstack-request-id': 'req-c02d3133-c5a9-4725-930b-4dc14a7c3868', 'x-compute-request-id': 'req-c02d3133-c5a9-4725-930b-4dc14a7c3868', 'connection': 'close', 'content-type': 'application/json', 'status': '200', 'content-location': 'http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e'} Body: b'{"server": {"id": "fc272b73-ff3e-4448-ab85-49bcec47546e", "name": "tempest-ServerActionsTestJSON-server-1454314435", "status": "SHUTOFF", "tenant_id": "59e8e567688941f89efae0bbf6a0d3a6", "user_id": "2e0cd1029f94430aa692fbaf536afbf1", "metadata": {}, "hostId": "c3eac17a1808415e7d9c75be02baf4105d1e0e1e8221a8a29542c6e8", "image": {"id": "0e69750d-552c-4f0e-a9d6-797bc415aa7f", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/images/0e69750d-552c-4f0e-a9d6-797bc415aa7f"}]}, "flavor": {"id": "0f5c781f-d41f-46a5-a744-10f9f5c69200", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/flavors/0f5c781f-d41f-46a5-a744-10f9f5c69200"}]}, "created": "2019-05-11T21:15:10Z", "updated": "2019-05-11T21:16:12Z", "addresses": {"tempest-ServerActionsTestJSON-1252015544-network": [{"version": 4, "addr": "10.100.0.8", "OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}, {"version": 4, "addr": "10.0.0.247", "OS-EXT-IPS:type": "floating", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}]}, "accessIPv4": "", "accessIPv6": "", "links": [{"rel": "self", "href": "http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}, {"rel": "bookmark", "href": "http://10.0.0.137:8774/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}], "OS-DCF:diskConfig": "MANUAL", "OS-EXT-AZ:availability_zone": "nova", "config_drive": "", "key_name": "tempest-keypair-638249548", "OS-SRV-USG:launched_at": "2019-05-11T21:15:57.000000", "OS-SRV-USG:terminated_at": null, "security_groups": [{"name": "tempest-securitygroup--1644861636"}], "OS-EXT-STS:task_state": null, "OS-EXT-STS:vm_state": "stopped", "OS-EXT-STS:power_state": 0, "os-extended-volumes:volumes_attached": []}}' 2019-05-11 21:21:00,707 215815 INFO [tempest.lib.common.rest_client] Request (ServerActionsTestJSON:tearDown): 200 GET http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e 0.750s 2019-05-11 21:21:00,708 215815 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-OpenStack-Nova-API-Version': '2.1', 'X-Auth-Token': '<omitted>'} Body: None Response - Headers: {'date': 'Sat, 11 May 2019 21:20:59 GMT', 'server': 'Apache', 'content-length': '1701', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version,Accept-Encoding', 'x-openstack-request-id': 'req-5ec40e04-e738-4267-82e8-c0b4f5ab0b43', 'x-compute-request-id': 'req-5ec40e04-e738-4267-82e8-c0b4f5ab0b43', 'connection': 'close', 'content-type': 'application/json', 'status': '200', 'content-location': 'http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e'} Body: b'{"server": {"id": "fc272b73-ff3e-4448-ab85-49bcec47546e", "name": "tempest-ServerActionsTestJSON-server-1454314435", "status": "SHUTOFF", "tenant_id": "59e8e567688941f89efae0bbf6a0d3a6", "user_id": "2e0cd1029f94430aa692fbaf536afbf1", "metadata": {}, "hostId": "c3eac17a1808415e7d9c75be02baf4105d1e0e1e8221a8a29542c6e8", "image": {"id": "0e69750d-552c-4f0e-a9d6-797bc415aa7f", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/images/0e69750d-552c-4f0e-a9d6-797bc415aa7f"}]}, "flavor": {"id": "0f5c781f-d41f-46a5-a744-10f9f5c69200", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/flavors/0f5c781f-d41f-46a5-a744-10f9f5c69200"}]}, "created": "2019-05-11T21:15:10Z", "updated": "2019-05-11T21:16:12Z", "addresses": {"tempest-ServerActionsTestJSON-1252015544-network": [{"version": 4, "addr": "10.100.0.8", "OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}, {"version": 4, "addr": "10.0.0.247", "OS-EXT-IPS:type": "floating", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}]}, "accessIPv4": "", "accessIPv6": "", "links": [{"rel": "self", "href": "http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}, {"rel": "bookmark", "href": "http://10.0.0.137:8774/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}], "OS-DCF:diskConfig": "MANUAL", "OS-EXT-AZ:availability_zone": "nova", "config_drive": "", "key_name": "tempest-keypair-638249548", "OS-SRV-USG:launched_at": "2019-05-11T21:15:57.000000", "OS-SRV-USG:terminated_at": null, "security_groups": [{"name": "tempest-securitygroup--1644861636"}], "OS-EXT-STS:task_state": null, "OS-EXT-STS:vm_state": "stopped", "OS-EXT-STS:power_state": 0, "os-extended-volumes:volumes_attached": []}}' 2019-05-11 21:21:02,171 215815 INFO [tempest.lib.common.rest_client] Request (ServerActionsTestJSON:tearDown): 200 GET http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e 0.458s 2019-05-11 21:21:02,172 215815 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-OpenStack-Nova-API-Version': '2.1', 'X-Auth-Token': '<omitted>'} Body: None Response - Headers: {'date': 'Sat, 11 May 2019 21:21:01 GMT', 'server': 'Apache', 'content-length': '1701', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version,Accept-Encoding', 'x-openstack-request-id': 'req-2c4b758e-8dac-4b3b-99e5-a98864d9074d', 'x-compute-request-id': 'req-2c4b758e-8dac-4b3b-99e5-a98864d9074d', 'connection': 'close', 'content-type': 'application/json', 'status': '200', 'content-location': 'http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e'} Body: b'{"server": {"id": "fc272b73-ff3e-4448-ab85-49bcec47546e", "name": "tempest-ServerActionsTestJSON-server-1454314435", "status": "SHUTOFF", "tenant_id": "59e8e567688941f89efae0bbf6a0d3a6", "user_id": "2e0cd1029f94430aa692fbaf536afbf1", "metadata": {}, "hostId": "c3eac17a1808415e7d9c75be02baf4105d1e0e1e8221a8a29542c6e8", "image": {"id": "0e69750d-552c-4f0e-a9d6-797bc415aa7f", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/images/0e69750d-552c-4f0e-a9d6-797bc415aa7f"}]}, "flavor": {"id": "0f5c781f-d41f-46a5-a744-10f9f5c69200", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/flavors/0f5c781f-d41f-46a5-a744-10f9f5c69200"}]}, "created": "2019-05-11T21:15:10Z", "updated": "2019-05-11T21:16:12Z", "addresses": {"tempest-ServerActionsTestJSON-1252015544-network": [{"version": 4, "addr": "10.100.0.8", "OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}, {"version": 4, "addr": "10.0.0.247", "OS-EXT-IPS:type": "floating", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}]}, "accessIPv4": "", "accessIPv6": "", "links": [{"rel": "self", "href": "http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}, {"rel": "bookmark", "href": "http://10.0.0.137:8774/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}], "OS-DCF:diskConfig": "MANUAL", "OS-EXT-AZ:availability_zone": "nova", "config_drive": "", "key_name": "tempest-keypair-638249548", "OS-SRV-USG:launched_at": "2019-05-11T21:15:57.000000", "OS-SRV-USG:terminated_at": null, "security_groups": [{"name": "tempest-securitygroup--1644861636"}], "OS-EXT-STS:task_state": null, "OS-EXT-STS:vm_state": "stopped", "OS-EXT-STS:power_state": 0, "os-extended-volumes:volumes_attached": []}}' 2019-05-11 21:21:03,520 215815 INFO [tempest.lib.common.rest_client] Request (ServerActionsTestJSON:tearDown): 200 GET http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e 0.342s 2019-05-11 21:21:03,520 215815 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-OpenStack-Nova-API-Version': '2.1', 'X-Auth-Token': '<omitted>'} Body: None Response - Headers: {'date': 'Sat, 11 May 2019 21:21:03 GMT', 'server': 'Apache', 'content-length': '1701', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version,Accept-Encoding', 'x-openstack-request-id': 'req-bf61ffcc-bc83-4320-b3bd-6c5f07833005', 'x-compute-request-id': 'req-bf61ffcc-bc83-4320-b3bd-6c5f07833005', 'connection': 'close', 'content-type': 'application/json', 'status': '200', 'content-location': 'http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e'} Body: b'{"server": {"id": "fc272b73-ff3e-4448-ab85-49bcec47546e", "name": "tempest-ServerActionsTestJSON-server-1454314435", "status": "SHUTOFF", "tenant_id": "59e8e567688941f89efae0bbf6a0d3a6", "user_id": "2e0cd1029f94430aa692fbaf536afbf1", "metadata": {}, "hostId": "c3eac17a1808415e7d9c75be02baf4105d1e0e1e8221a8a29542c6e8", "image": {"id": "0e69750d-552c-4f0e-a9d6-797bc415aa7f", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/images/0e69750d-552c-4f0e-a9d6-797bc415aa7f"}]}, "flavor": {"id": "0f5c781f-d41f-46a5-a744-10f9f5c69200", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/flavors/0f5c781f-d41f-46a5-a744-10f9f5c69200"}]}, "created": "2019-05-11T21:15:10Z", "updated": "2019-05-11T21:16:12Z", "addresses": {"tempest-ServerActionsTestJSON-1252015544-network": [{"version": 4, "addr": "10.100.0.8", "OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}, {"version": 4, "addr": "10.0.0.247", "OS-EXT-IPS:type": "floating", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}]}, "accessIPv4": "", "accessIPv6": "", "links": [{"rel": "self", "href": "http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}, {"rel": "bookmark", "href": "http://10.0.0.137:8774/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}], "OS-DCF:diskConfig": "MANUAL", "OS-EXT-AZ:availability_zone": "nova", "config_drive": "", "key_name": "tempest-keypair-638249548", "OS-SRV-USG:launched_at": "2019-05-11T21:15:57.000000", "OS-SRV-USG:terminated_at": null, "security_groups": [{"name": "tempest-securitygroup--1644861636"}], "OS-EXT-STS:task_state": null, "OS-EXT-STS:vm_state": "stopped", "OS-EXT-STS:power_state": 0, "os-extended-volumes:volumes_attached": []}}' 2019-05-11 21:21:04,963 215815 INFO [tempest.lib.common.rest_client] Request (ServerActionsTestJSON:tearDown): 200 GET http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e 0.437s 2019-05-11 21:21:04,964 215815 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-OpenStack-Nova-API-Version': '2.1', 'X-Auth-Token': '<omitted>'} Body: None Response - Headers: {'date': 'Sat, 11 May 2019 21:21:04 GMT', 'server': 'Apache', 'content-length': '1701', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version,Accept-Encoding', 'x-openstack-request-id': 'req-42d4d792-318e-45b0-9d37-273d739bb6b6', 'x-compute-request-id': 'req-42d4d792-318e-45b0-9d37-273d739bb6b6', 'connection': 'close', 'content-type': 'application/json', 'status': '200', 'content-location': 'http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e'} Body: b'{"server": {"id": "fc272b73-ff3e-4448-ab85-49bcec47546e", "name": "tempest-ServerActionsTestJSON-server-1454314435", "status": "SHUTOFF", "tenant_id": "59e8e567688941f89efae0bbf6a0d3a6", "user_id": "2e0cd1029f94430aa692fbaf536afbf1", "metadata": {}, "hostId": "c3eac17a1808415e7d9c75be02baf4105d1e0e1e8221a8a29542c6e8", "image": {"id": "0e69750d-552c-4f0e-a9d6-797bc415aa7f", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/images/0e69750d-552c-4f0e-a9d6-797bc415aa7f"}]}, "flavor": {"id": "0f5c781f-d41f-46a5-a744-10f9f5c69200", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/flavors/0f5c781f-d41f-46a5-a744-10f9f5c69200"}]}, "created": "2019-05-11T21:15:10Z", "updated": "2019-05-11T21:16:12Z", "addresses": {"tempest-ServerActionsTestJSON-1252015544-network": [{"version": 4, "addr": "10.100.0.8", "OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}, {"version": 4, "addr": "10.0.0.247", "OS-EXT-IPS:type": "floating", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}]}, "accessIPv4": "", "accessIPv6": "", "links": [{"rel": "self", "href": "http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}, {"rel": "bookmark", "href": "http://10.0.0.137:8774/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}], "OS-DCF:diskConfig": "MANUAL", "OS-EXT-AZ:availability_zone": "nova", "config_drive": "", "key_name": "tempest-keypair-638249548", "OS-SRV-USG:launched_at": "2019-05-11T21:15:57.000000", "OS-SRV-USG:terminated_at": null, "security_groups": [{"name": "tempest-securitygroup--1644861636"}], "OS-EXT-STS:task_state": null, "OS-EXT-STS:vm_state": "stopped", "OS-EXT-STS:power_state": 0, "os-extended-volumes:volumes_attached": []}}' 2019-05-11 21:21:06,502 215815 INFO [tempest.lib.common.rest_client] Request (ServerActionsTestJSON:tearDown): 200 GET http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e 0.532s 2019-05-11 21:21:06,503 215815 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-OpenStack-Nova-API-Version': '2.1', 'X-Auth-Token': '<omitted>'} Body: None Response - Headers: {'date': 'Sat, 11 May 2019 21:21:05 GMT', 'server': 'Apache', 'content-length': '1701', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version,Accept-Encoding', 'x-openstack-request-id': 'req-69b1efec-7c76-4891-84d2-1e5bb2c42ceb', 'x-compute-request-id': 'req-69b1efec-7c76-4891-84d2-1e5bb2c42ceb', 'connection': 'close', 'content-type': 'application/json', 'status': '200', 'content-location': 'http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e'} Body: b'{"server": {"id": "fc272b73-ff3e-4448-ab85-49bcec47546e", "name": "tempest-ServerActionsTestJSON-server-1454314435", "status": "SHUTOFF", "tenant_id": "59e8e567688941f89efae0bbf6a0d3a6", "user_id": "2e0cd1029f94430aa692fbaf536afbf1", "metadata": {}, "hostId": "c3eac17a1808415e7d9c75be02baf4105d1e0e1e8221a8a29542c6e8", "image": {"id": "0e69750d-552c-4f0e-a9d6-797bc415aa7f", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/images/0e69750d-552c-4f0e-a9d6-797bc415aa7f"}]}, "flavor": {"id": "0f5c781f-d41f-46a5-a744-10f9f5c69200", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/flavors/0f5c781f-d41f-46a5-a744-10f9f5c69200"}]}, "created": "2019-05-11T21:15:10Z", "updated": "2019-05-11T21:16:12Z", "addresses": {"tempest-ServerActionsTestJSON-1252015544-network": [{"version": 4, "addr": "10.100.0.8", "OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}, {"version": 4, "addr": "10.0.0.247", "OS-EXT-IPS:type": "floating", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}]}, "accessIPv4": "", "accessIPv6": "", "links": [{"rel": "self", "href": "http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}, {"rel": "bookmark", "href": "http://10.0.0.137:8774/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}], "OS-DCF:diskConfig": "MANUAL", "OS-EXT-AZ:availability_zone": "nova", "config_drive": "", "key_name": "tempest-keypair-638249548", "OS-SRV-USG:launched_at": "2019-05-11T21:15:57.000000", "OS-SRV-USG:terminated_at": null, "security_groups": [{"name": "tempest-securitygroup--1644861636"}], "OS-EXT-STS:task_state": null, "OS-EXT-STS:vm_state": "stopped", "OS-EXT-STS:power_state": 0, "os-extended-volumes:volumes_attached": []}}' 2019-05-11 21:21:06,508 215815 ERROR [tempest.api.compute.base] Request timed out Details: (ServerActionsTestJSON:tearDown) Server fc272b73-ff3e-4448-ab85-49bcec47546e failed to reach ACTIVE status and task state "None" within the required time (300 s). Current status: SHUTOFF. Current task state: None. 2019-05-11 21:21:06.508 215815 ERROR tempest.api.compute.base Traceback (most recent call last): 2019-05-11 21:21:06.508 215815 ERROR tempest.api.compute.base File "/usr/lib/python3.6/site-packages/tempest/api/compute/base.py", line 201, in server_check_teardown 2019-05-11 21:21:06.508 215815 ERROR tempest.api.compute.base cls.server_id, 'ACTIVE') 2019-05-11 21:21:06.508 215815 ERROR tempest.api.compute.base File "/usr/lib/python3.6/site-packages/tempest/common/waiters.py", line 96, in wait_for_server_status 2019-05-11 21:21:06.508 215815 ERROR tempest.api.compute.base raise lib_exc.TimeoutException(message) 2019-05-11 21:21:06.508 215815 ERROR tempest.api.compute.base tempest.lib.exceptions.TimeoutException: Request timed out 2019-05-11 21:21:06.508 215815 ERROR tempest.api.compute.base Details: (ServerActionsTestJSON:tearDown) Server fc272b73-ff3e-4448-ab85-49bcec47546e failed to reach ACTIVE status and task state "None" within the required time (300 s). Current status: SHUTOFF. Current task state: None. 2019-05-11 21:21:06.508 215815 ERROR tempest.api.compute.base 2019-05-11 21:21:06,733 215815 INFO [tempest.lib.common.rest_client] Request (ServerActionsTestJSON:tearDown): 204 DELETE http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e 0.224s 2019-05-11 21:21:06,733 215815 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-OpenStack-Nova-API-Version': '2.1', 'X-Auth-Token': '<omitted>'} Body: None Response - Headers: {'date': 'Sat, 11 May 2019 21:21:06 GMT', 'server': 'Apache', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version', 'x-openstack-request-id': 'req-05079baa-4eb5-4a41-8820-00f3fb231c34', 'x-compute-request-id': 'req-05079baa-4eb5-4a41-8820-00f3fb231c34', 'connection': 'close', 'status': '204', 'content-location': 'http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e'} Body: b'' 2019-05-11 21:21:07,705 215815 INFO [tempest.lib.common.rest_client] Request (ServerActionsTestJSON:tearDown): 200 GET http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e 0.971s 2019-05-11 21:21:07,705 215815 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-OpenStack-Nova-API-Version': '2.1', 'X-Auth-Token': '<omitted>'} Body: None Response - Headers: {'date': 'Sat, 11 May 2019 21:21:06 GMT', 'server': 'Apache', 'content-length': '1707', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version,Accept-Encoding', 'x-openstack-request-id': 'req-0d4610bc-0216-426e-a72e-a17442ccca9b', 'x-compute-request-id': 'req-0d4610bc-0216-426e-a72e-a17442ccca9b', 'connection': 'close', 'content-type': 'application/json', 'status': '200', 'content-location': 'http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e'} Body: b'{"server": {"id": "fc272b73-ff3e-4448-ab85-49bcec47546e", "name": "tempest-ServerActionsTestJSON-server-1454314435", "status": "SHUTOFF", "tenant_id": "59e8e567688941f89efae0bbf6a0d3a6", "user_id": "2e0cd1029f94430aa692fbaf536afbf1", "metadata": {}, "hostId": "c3eac17a1808415e7d9c75be02baf4105d1e0e1e8221a8a29542c6e8", "image": {"id": "0e69750d-552c-4f0e-a9d6-797bc415aa7f", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/images/0e69750d-552c-4f0e-a9d6-797bc415aa7f"}]}, "flavor": {"id": "0f5c781f-d41f-46a5-a744-10f9f5c69200", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/flavors/0f5c781f-d41f-46a5-a744-10f9f5c69200"}]}, "created": "2019-05-11T21:15:10Z", "updated": "2019-05-11T21:21:06Z", "addresses": {"tempest-ServerActionsTestJSON-1252015544-network": [{"version": 4, "addr": "10.100.0.8", "OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}, {"version": 4, "addr": "10.0.0.247", "OS-EXT-IPS:type": "floating", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}]}, "accessIPv4": "", "accessIPv6": "", "links": [{"rel": "self", "href": "http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}, {"rel": "bookmark", "href": "http://10.0.0.137:8774/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}], "OS-DCF:diskConfig": "MANUAL", "OS-EXT-AZ:availability_zone": "nova", "config_drive": "", "key_name": "tempest-keypair-638249548", "OS-SRV-USG:launched_at": "2019-05-11T21:15:57.000000", "OS-SRV-USG:terminated_at": null, "security_groups": [{"name": "tempest-securitygroup--1644861636"}], "OS-EXT-STS:task_state": "deleting", "OS-EXT-STS:vm_state": "stopped", "OS-EXT-STS:power_state": 0, "os-extended-volumes:volumes_attached": []}}' 2019-05-11 21:21:09,085 215815 INFO [tempest.lib.common.rest_client] Request (ServerActionsTestJSON:tearDown): 200 GET http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e 0.373s 2019-05-11 21:21:09,085 215815 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-OpenStack-Nova-API-Version': '2.1', 'X-Auth-Token': '<omitted>'} Body: None Response - Headers: {'date': 'Sat, 11 May 2019 21:21:08 GMT', 'server': 'Apache', 'content-length': '1707', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version,Accept-Encoding', 'x-openstack-request-id': 'req-78692737-83db-4caa-851d-194ffec7e0b7', 'x-compute-request-id': 'req-78692737-83db-4caa-851d-194ffec7e0b7', 'connection': 'close', 'content-type': 'application/json', 'status': '200', 'content-location': 'http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e'} Body: b'{"server": {"id": "fc272b73-ff3e-4448-ab85-49bcec47546e", "name": "tempest-ServerActionsTestJSON-server-1454314435", "status": "SHUTOFF", "tenant_id": "59e8e567688941f89efae0bbf6a0d3a6", "user_id": "2e0cd1029f94430aa692fbaf536afbf1", "metadata": {}, "hostId": "c3eac17a1808415e7d9c75be02baf4105d1e0e1e8221a8a29542c6e8", "image": {"id": "0e69750d-552c-4f0e-a9d6-797bc415aa7f", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/images/0e69750d-552c-4f0e-a9d6-797bc415aa7f"}]}, "flavor": {"id": "0f5c781f-d41f-46a5-a744-10f9f5c69200", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/flavors/0f5c781f-d41f-46a5-a744-10f9f5c69200"}]}, "created": "2019-05-11T21:15:10Z", "updated": "2019-05-11T21:21:06Z", "addresses": {"tempest-ServerActionsTestJSON-1252015544-network": [{"version": 4, "addr": "10.100.0.8", "OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}, {"version": 4, "addr": "10.0.0.247", "OS-EXT-IPS:type": "floating", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}]}, "accessIPv4": "", "accessIPv6": "", "links": [{"rel": "self", "href": "http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}, {"rel": "bookmark", "href": "http://10.0.0.137:8774/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}], "OS-DCF:diskConfig": "MANUAL", "OS-EXT-AZ:availability_zone": "nova", "config_drive": "", "key_name": "tempest-keypair-638249548", "OS-SRV-USG:launched_at": "2019-05-11T21:15:57.000000", "OS-SRV-USG:terminated_at": null, "security_groups": [{"name": "tempest-securitygroup--1644861636"}], "OS-EXT-STS:task_state": "deleting", "OS-EXT-STS:vm_state": "stopped", "OS-EXT-STS:power_state": 0, "os-extended-volumes:volumes_attached": []}}' 2019-05-11 21:21:10,288 215815 INFO [tempest.lib.common.rest_client] Request (ServerActionsTestJSON:tearDown): 200 GET http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e 0.196s 2019-05-11 21:21:10,289 215815 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-OpenStack-Nova-API-Version': '2.1', 'X-Auth-Token': '<omitted>'} Body: None Response - Headers: {'date': 'Sat, 11 May 2019 21:21:10 GMT', 'server': 'Apache', 'content-length': '1583', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version,Accept-Encoding', 'x-openstack-request-id': 'req-9abf1275-1cbf-401d-99f5-7c512510012e', 'x-compute-request-id': 'req-9abf1275-1cbf-401d-99f5-7c512510012e', 'connection': 'close', 'content-type': 'application/json', 'status': '200', 'content-location': 'http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e'} Body: b'{"server": {"id": "fc272b73-ff3e-4448-ab85-49bcec47546e", "name": "tempest-ServerActionsTestJSON-server-1454314435", "status": "DELETED", "tenant_id": "59e8e567688941f89efae0bbf6a0d3a6", "user_id": "2e0cd1029f94430aa692fbaf536afbf1", "metadata": {}, "hostId": "c3eac17a1808415e7d9c75be02baf4105d1e0e1e8221a8a29542c6e8", "image": {"id": "0e69750d-552c-4f0e-a9d6-797bc415aa7f", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/images/0e69750d-552c-4f0e-a9d6-797bc415aa7f"}]}, "flavor": {"id": "0f5c781f-d41f-46a5-a744-10f9f5c69200", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/flavors/0f5c781f-d41f-46a5-a744-10f9f5c69200"}]}, "created": "2019-05-11T21:15:10Z", "updated": "2019-05-11T21:21:09Z", "addresses": {}, "accessIPv4": "", "accessIPv6": "", "links": [{"rel": "self", "href": "http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}, {"rel": "bookmark", "href": "http://10.0.0.137:8774/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}], "OS-DCF:diskConfig": "MANUAL", "fault": {"code": 500, "created": "2019-05-11T21:16:08Z", "message": "Cannot access storage file \'/var/lib/nova/instances/fc272b73-ff3e-4448-ab85-49bcec47546e/disk\' (as uid:107, gid:107): Permission denied"}, "OS-EXT-AZ:availability_zone": "nova", "config_drive": "", "key_name": "tempest-keypair-638249548", "OS-SRV-USG:launched_at": "2019-05-11T21:15:57.000000", "OS-SRV-USG:terminated_at": "2019-05-11T21:21:09.000000", "OS-EXT-STS:task_state": null, "OS-EXT-STS:vm_state": "deleted", "OS-EXT-STS:power_state": 0, "os-extended-volumes:volumes_attached": []}}' 2019-05-11 21:21:10,294 215815 INFO [tempest.common.waiters] State transition "SHUTOFF/deleting" ==> "DELETED/None" after 3 second wait 2019-05-11 21:21:11,358 215815 INFO [tempest.lib.common.rest_client] Request (ServerActionsTestJSON:tearDown): 404 GET http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e 0.063s 2019-05-11 21:21:11,359 215815 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-OpenStack-Nova-API-Version': '2.1', 'X-Auth-Token': '<omitted>'} Body: None Response - Headers: {'date': 'Sat, 11 May 2019 21:21:11 GMT', 'server': 'Apache', 'content-length': '111', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version', 'x-openstack-request-id': 'req-dd453706-d75e-4d85-a373-576bc4dbcdc0', 'x-compute-request-id': 'req-dd453706-d75e-4d85-a373-576bc4dbcdc0', 'connection': 'close', 'content-type': 'application/json; charset=UTF-8', 'status': '404', 'content-location': 'http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e'} Body: b'{"itemNotFound": {"code": 404, "message": "Instance fc272b73-ff3e-4448-ab85-49bcec47546e could not be found."}}' 2019-05-11 21:21:11,431 215815 INFO [tempest.lib.common.rest_client] Request (ServerActionsTestJSON:_run_cleanups): 404 DELETE http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e 0.072s 2019-05-11 21:21:11,432 215815 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-OpenStack-Nova-API-Version': '2.1', 'X-Auth-Token': '<omitted>'} Body: None Response - Headers: {'date': 'Sat, 11 May 2019 21:21:11 GMT', 'server': 'Apache', 'content-length': '111', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version', 'x-openstack-request-id': 'req-2dd3cc26-e04a-4a54-863a-f4bbc7eb49e7', 'x-compute-request-id': 'req-2dd3cc26-e04a-4a54-863a-f4bbc7eb49e7', 'connection': 'close', 'content-type': 'application/json; charset=UTF-8', 'status': '404', 'content-location': 'http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e'} Body: b'{"itemNotFound": {"code": 404, "message": "Instance fc272b73-ff3e-4448-ab85-49bcec47546e could not be found."}}' 2019-05-11 21:21:11,432 215815 ERROR [tempest.api.compute.base] Failed to delete server fc272b73-ff3e-4448-ab85-49bcec47546e 2019-05-11 21:21:11.432 215815 ERROR tempest.api.compute.base Traceback (most recent call last): 2019-05-11 21:21:11.432 215815 ERROR tempest.api.compute.base File "/usr/lib/python3.6/site-packages/tempest/api/compute/base.py", line 414, in delete_server 2019-05-11 21:21:11.432 215815 ERROR tempest.api.compute.base cls.servers_client.delete_server(server_id) 2019-05-11 21:21:11.432 215815 ERROR tempest.api.compute.base File "/usr/lib/python3.6/site-packages/tempest/lib/services/compute/servers_client.py", line 152, in delete_server 2019-05-11 21:21:11.432 215815 ERROR tempest.api.compute.base resp, body = self.delete("servers/%s" % server_id) 2019-05-11 21:21:11.432 215815 ERROR tempest.api.compute.base File "/usr/lib/python3.6/site-packages/tempest/lib/common/rest_client.py", line 314, in delete 2019-05-11 21:21:11.432 215815 ERROR tempest.api.compute.base return self.request('DELETE', url, extra_headers, headers, body) 2019-05-11 21:21:11.432 215815 ERROR tempest.api.compute.base File "/usr/lib/python3.6/site-packages/tempest/lib/services/compute/base_compute_client.py", line 48, in request 2019-05-11 21:21:11.432 215815 ERROR tempest.api.compute.base method, url, extra_headers, headers, body, chunked) 2019-05-11 21:21:11.432 215815 ERROR tempest.api.compute.base File "/usr/lib/python3.6/site-packages/tempest/lib/common/rest_client.py", line 679, in request 2019-05-11 21:21:11.432 215815 ERROR tempest.api.compute.base self._error_checker(resp, resp_body) 2019-05-11 21:21:11.432 215815 ERROR tempest.api.compute.base File "/usr/lib/python3.6/site-packages/tempest/lib/common/rest_client.py", line 785, in _error_checker 2019-05-11 21:21:11.432 215815 ERROR tempest.api.compute.base raise exceptions.NotFound(resp_body, resp=resp) 2019-05-11 21:21:11.432 215815 ERROR tempest.api.compute.base tempest.lib.exceptions.NotFound: Object not found 2019-05-11 21:21:11.432 215815 ERROR tempest.api.compute.base Details: {'code': 404, 'message': 'Instance fc272b73-ff3e-4448-ab85-49bcec47546e could not be found.'} 2019-05-11 21:21:11.432 215815 ERROR tempest.api.compute.base }}} Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/tempest/api/compute/servers/test_server_actions.py", line 66, in tearDown self.server_check_teardown() File "/usr/lib/python3.6/site-packages/tempest/api/compute/base.py", line 201, in server_check_teardown cls.server_id, 'ACTIVE') File "/usr/lib/python3.6/site-packages/tempest/common/waiters.py", line 96, in wait_for_server_status raise lib_exc.TimeoutException(message) tempest.lib.exceptions.TimeoutException: Request timed out Details: (ServerActionsTestJSON:tearDown) Server fc272b73-ff3e-4448-ab85-49bcec47546e failed to reach ACTIVE status and task state "None" within the required time (300 s). Current status: SHUTOFF. Current task state: None.
This is weird in tempest, but here's how the flow goes: 1. test_resize_server_confirm_from_stopped() [https://github.com/openstack/tempest/blob/20.0.0/tempest/api/compute/servers/test_server_actions.py#L357] 2. _test_resize_server_confirm(... stop=True) [https://github.com/openstack/tempest/blob/20.0.0/tempest/api/compute/servers/test_server_actions.py#L292] 3. <start the server after the test> [https://github.com/openstack/tempest/blob/20.0.0/tempest/api/compute/servers/test_server_actions.py#L318] 4. tearDown() [https://github.com/openstack/tempest/blob/20.0.0/tempest/api/compute/servers/test_server_actions.py#L61] 5. <wait for server ACTIVE from 3> [https://github.com/openstack/tempest/blob/20.0.0/tempest/api/compute/base.py#L200] 5 is the part that failed. 3's request ID to start the server is req-93ccce31-1d79-47a7-933a-c9fadde37460. In the Nova logs we see: 2019-05-11 21:16:04.648 [./controller-0/var/log/containers/nova/nova-api.log] 21 DEBUG nova.api.openstack.wsgi [req-93ccce31-1d79-47a7-933a-c9fadde37460 2e0cd1029f94430aa692fbaf536afbf1 59e8e567688941f89efae0bbf6a0d3a6 - default default] Action: 'action', calling method: <bound method ServersController._start_server of <nova.api.openstack.compute.servers.ServersController object at 0x7ff527d14c18>>, body: {"os-start": {}} _process_stack /usr/lib/python3.6/site-packages/nova/api/openstack/wsgi.py:520 2019-05-11 21:16:08.546 [./compute-0/var/log/containers/nova/nova-compute.log] 8 ERROR nova.virt.libvirt.driver [req-93ccce31-1d79-47a7-933a-c9fadde37460 2e0cd1029f94430aa692fbaf536afbf1 59e8e567688941f89efae0bbf6a0d3a6 - default default] [instance: fc272b73-ff3e-4448-ab85-49bcec47546e] Failed to start libvirt guest: libvirt.libvirtError: Cannot access storage file '/var/lib/nova/instances/fc272b73-ff3e-4448-ab85-49bcec47546e/disk' (as uid:107, gid:107): Permission denied 2019-05-11 21:16:08.764 [./compute-0/var/log/containers/nova/nova-compute.log] 8 ERROR oslo_messaging.rpc.server [req-93ccce31-1d79-47a7-933a-c9fadde37460 2e0cd1029f94430aa692fbaf536afbf1 59e8e567688941f89efae0bbf6a0d3a6 - default default] Exception during message handling: libvirt.libvirtError: Cannot access storage file '/var/lib/nova/instances/fc272b73-ff3e-4448-ab85-49bcec47546e/disk' (as uid:107, gid:107): Permission denied So to me this smells deployment-y...
*** This bug has been marked as a duplicate of bug 1710034 ***