Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1710049

Summary:

[RHOS 15] Tempest test failure: test_resize_server_confirm_from_stopped

Product:

Red Hat OpenStack

Reporter:

Archit Modi <amodi>

Component:

openstack-nova

Assignee:

Artom Lifshitz <alifshit>

Status:

CLOSED DUPLICATE

QA Contact:

OSP DFG:Compute <osp-dfg-compute>

Severity:

unspecified

Docs Contact:

Priority:

unspecified

Version:

15.0 (Stein)

CC:

dasmith, eglynn, jhakimra, kchamart, mbooth, sbauza, sgordon, vromanso

Target Milestone:

---

Target Release:

---

Hardware:

Unspecified

OS:

Unspecified

Whiteboard:

Fixed In Version:

Doc Type:

If docs needed, set a value

Doc Text:

Story Points:

---

Clone Of:

Environment:

Last Closed:

2019-05-17 12:46:03 UTC

Type:

Bug

Regression:

---

Mount Type:

---

Documentation:

---

CRM:

Verified Versions:

Category:

---

oVirt Team:

---

RHEL 7.3 requirements from Atomic Host:

Cloudforms Team:

---

Target Upstream Version:

Embargoed:

Attachments:

Description	Flags
tempest-run.log	none

Description Archit Modi 2019-05-14 19:46:00 UTC

Created attachment 1568631 [details]
tempest-run.log

Description of problem: Deploy RHos-15 with the mentioned topology and run the following tempest test which fails:
tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_resize_server_confirm_from_stopped

core_puddle: RHOS_TRUNK-15.0-RHEL-8-20190509.n.1

Topology: 1 Controller, 2 Computes

Steps to Reproduce:
1. Deploy RHOS-15 with mentioned topology
2. Run the following tempest test: tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_resize_server_confirm_from_stopped

Actual results:
    Response - Headers: {'date': 'Sat, 11 May 2019 21:20:54 GMT', 'server': 'Apache', 'content-length': '1701', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version,Accept-Encoding', 'x-openstack-request-id': 'req-f78f7829-029f-4971-ac82-dca85c1e04dd', 'x-compute-request-id': 'req-f78f7829-029f-4971-ac82-dca85c1e04dd', 'connection': 'close', 'content-type': 'application/json', 'status': '200', 'content-location': 'http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e'}
        Body: b'{"server": {"id": "fc272b73-ff3e-4448-ab85-49bcec47546e", "name": "tempest-ServerActionsTestJSON-server-1454314435", "status": "SHUTOFF", "tenant_id": "59e8e567688941f89efae0bbf6a0d3a6", "user_id": "2e0cd1029f94430aa692fbaf536afbf1", "metadata": {}, "hostId": "c3eac17a1808415e7d9c75be02baf4105d1e0e1e8221a8a29542c6e8", "image": {"id": "0e69750d-552c-4f0e-a9d6-797bc415aa7f", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/images/0e69750d-552c-4f0e-a9d6-797bc415aa7f"}]}, "flavor": {"id": "0f5c781f-d41f-46a5-a744-10f9f5c69200", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/flavors/0f5c781f-d41f-46a5-a744-10f9f5c69200"}]}, "created": "2019-05-11T21:15:10Z", "updated": "2019-05-11T21:16:12Z", "addresses": {"tempest-ServerActionsTestJSON-1252015544-network": [{"version": 4, "addr": "10.100.0.8", "OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}, {"version": 4, "addr": "10.0.0.247", "OS-EXT-IPS:type": "floating", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}]}, "accessIPv4": "", "accessIPv6": "", "links": [{"rel": "self", "href": "http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}, {"rel": "bookmark", "href": "http://10.0.0.137:8774/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}], "OS-DCF:diskConfig": "MANUAL", "OS-EXT-AZ:availability_zone": "nova", "config_drive": "", "key_name": "tempest-keypair-638249548", "OS-SRV-USG:launched_at": "2019-05-11T21:15:57.000000", "OS-SRV-USG:terminated_at": null, "security_groups": [{"name": "tempest-securitygroup--1644861636"}], "OS-EXT-STS:task_state": null, "OS-EXT-STS:vm_state": "stopped", "OS-EXT-STS:power_state": 0, "os-extended-volumes:volumes_attached": []}}'
2019-05-11 21:20:56,211 215815 INFO     [tempest.lib.common.rest_client] Request (ServerActionsTestJSON:tearDown): 200 GET http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e 0.308s
2019-05-11 21:20:56,212 215815 DEBUG    [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-OpenStack-Nova-API-Version': '2.1', 'X-Auth-Token': '<omitted>'}
        Body: None
    Response - Headers: {'date': 'Sat, 11 May 2019 21:20:55 GMT', 'server': 'Apache', 'content-length': '1701', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version,Accept-Encoding', 'x-openstack-request-id': 'req-680ed90e-6e60-4ee4-a69f-386bfd1b1fd5', 'x-compute-request-id': 'req-680ed90e-6e60-4ee4-a69f-386bfd1b1fd5', 'connection': 'close', 'content-type': 'application/json', 'status': '200', 'content-location': 'http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e'}
        Body: b'{"server": {"id": "fc272b73-ff3e-4448-ab85-49bcec47546e", "name": "tempest-ServerActionsTestJSON-server-1454314435", "status": "SHUTOFF", "tenant_id": "59e8e567688941f89efae0bbf6a0d3a6", "user_id": "2e0cd1029f94430aa692fbaf536afbf1", "metadata": {}, "hostId": "c3eac17a1808415e7d9c75be02baf4105d1e0e1e8221a8a29542c6e8", "image": {"id": "0e69750d-552c-4f0e-a9d6-797bc415aa7f", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/images/0e69750d-552c-4f0e-a9d6-797bc415aa7f"}]}, "flavor": {"id": "0f5c781f-d41f-46a5-a744-10f9f5c69200", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/flavors/0f5c781f-d41f-46a5-a744-10f9f5c69200"}]}, "created": "2019-05-11T21:15:10Z", "updated": "2019-05-11T21:16:12Z", "addresses": {"tempest-ServerActionsTestJSON-1252015544-network": [{"version": 4, "addr": "10.100.0.8", "OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}, {"version": 4, "addr": "10.0.0.247", "OS-EXT-IPS:type": "floating", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}]}, "accessIPv4": "", "accessIPv6": "", "links": [{"rel": "self", "href": "http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}, {"rel": "bookmark", "href": "http://10.0.0.137:8774/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}], "OS-DCF:diskConfig": "MANUAL", "OS-EXT-AZ:availability_zone": "nova", "config_drive": "", "key_name": "tempest-keypair-638249548", "OS-SRV-USG:launched_at": "2019-05-11T21:15:57.000000", "OS-SRV-USG:terminated_at": null, "security_groups": [{"name": "tempest-securitygroup--1644861636"}], "OS-EXT-STS:task_state": null, "OS-EXT-STS:vm_state": "stopped", "OS-EXT-STS:power_state": 0, "os-extended-volumes:volumes_attached": []}}'
2019-05-11 21:20:57,540 215815 INFO     [tempest.lib.common.rest_client] Request (ServerActionsTestJSON:tearDown): 200 GET http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e 0.322s
2019-05-11 21:20:57,540 215815 DEBUG    [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-OpenStack-Nova-API-Version': '2.1', 'X-Auth-Token': '<omitted>'}
        Body: None
    Response - Headers: {'date': 'Sat, 11 May 2019 21:20:57 GMT', 'server': 'Apache', 'content-length': '1701', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version,Accept-Encoding', 'x-openstack-request-id': 'req-bbc1e6b5-b558-4984-929c-a88a27ed8619', 'x-compute-request-id': 'req-bbc1e6b5-b558-4984-929c-a88a27ed8619', 'connection': 'close', 'content-type': 'application/json', 'status': '200', 'content-location': 'http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e'}
        Body: b'{"server": {"id": "fc272b73-ff3e-4448-ab85-49bcec47546e", "name": "tempest-ServerActionsTestJSON-server-1454314435", "status": "SHUTOFF", "tenant_id": "59e8e567688941f89efae0bbf6a0d3a6", "user_id": "2e0cd1029f94430aa692fbaf536afbf1", "metadata": {}, "hostId": "c3eac17a1808415e7d9c75be02baf4105d1e0e1e8221a8a29542c6e8", "image": {"id": "0e69750d-552c-4f0e-a9d6-797bc415aa7f", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/images/0e69750d-552c-4f0e-a9d6-797bc415aa7f"}]}, "flavor": {"id": "0f5c781f-d41f-46a5-a744-10f9f5c69200", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/flavors/0f5c781f-d41f-46a5-a744-10f9f5c69200"}]}, "created": "2019-05-11T21:15:10Z", "updated": "2019-05-11T21:16:12Z", "addresses": {"tempest-ServerActionsTestJSON-1252015544-network": [{"version": 4, "addr": "10.100.0.8", "OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}, {"version": 4, "addr": "10.0.0.247", "OS-EXT-IPS:type": "floating", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}]}, "accessIPv4": "", "accessIPv6": "", "links": [{"rel": "self", "href": "http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}, {"rel": "bookmark", "href": "http://10.0.0.137:8774/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}], "OS-DCF:diskConfig": "MANUAL", "OS-EXT-AZ:availability_zone": "nova", "config_drive": "", "key_name": "tempest-keypair-638249548", "OS-SRV-USG:launched_at": "2019-05-11T21:15:57.000000", "OS-SRV-USG:terminated_at": null, "security_groups": [{"name": "tempest-securitygroup--1644861636"}], "OS-EXT-STS:task_state": null, "OS-EXT-STS:vm_state": "stopped", "OS-EXT-STS:power_state": 0, "os-extended-volumes:volumes_attached": []}}'
2019-05-11 21:20:58,950 215815 INFO     [tempest.lib.common.rest_client] Request (ServerActionsTestJSON:tearDown): 200 GET http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e 0.403s
2019-05-11 21:20:58,950 215815 DEBUG    [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-OpenStack-Nova-API-Version': '2.1', 'X-Auth-Token': '<omitted>'}
        Body: None
    Response - Headers: {'date': 'Sat, 11 May 2019 21:20:58 GMT', 'server': 'Apache', 'content-length': '1701', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version,Accept-Encoding', 'x-openstack-request-id': 'req-c02d3133-c5a9-4725-930b-4dc14a7c3868', 'x-compute-request-id': 'req-c02d3133-c5a9-4725-930b-4dc14a7c3868', 'connection': 'close', 'content-type': 'application/json', 'status': '200', 'content-location': 'http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e'}
        Body: b'{"server": {"id": "fc272b73-ff3e-4448-ab85-49bcec47546e", "name": "tempest-ServerActionsTestJSON-server-1454314435", "status": "SHUTOFF", "tenant_id": "59e8e567688941f89efae0bbf6a0d3a6", "user_id": "2e0cd1029f94430aa692fbaf536afbf1", "metadata": {}, "hostId": "c3eac17a1808415e7d9c75be02baf4105d1e0e1e8221a8a29542c6e8", "image": {"id": "0e69750d-552c-4f0e-a9d6-797bc415aa7f", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/images/0e69750d-552c-4f0e-a9d6-797bc415aa7f"}]}, "flavor": {"id": "0f5c781f-d41f-46a5-a744-10f9f5c69200", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/flavors/0f5c781f-d41f-46a5-a744-10f9f5c69200"}]}, "created": "2019-05-11T21:15:10Z", "updated": "2019-05-11T21:16:12Z", "addresses": {"tempest-ServerActionsTestJSON-1252015544-network": [{"version": 4, "addr": "10.100.0.8", "OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}, {"version": 4, "addr": "10.0.0.247", "OS-EXT-IPS:type": "floating", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}]}, "accessIPv4": "", "accessIPv6": "", "links": [{"rel": "self", "href": "http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}, {"rel": "bookmark", "href": "http://10.0.0.137:8774/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}], "OS-DCF:diskConfig": "MANUAL", "OS-EXT-AZ:availability_zone": "nova", "config_drive": "", "key_name": "tempest-keypair-638249548", "OS-SRV-USG:launched_at": "2019-05-11T21:15:57.000000", "OS-SRV-USG:terminated_at": null, "security_groups": [{"name": "tempest-securitygroup--1644861636"}], "OS-EXT-STS:task_state": null, "OS-EXT-STS:vm_state": "stopped", "OS-EXT-STS:power_state": 0, "os-extended-volumes:volumes_attached": []}}'
2019-05-11 21:21:00,707 215815 INFO     [tempest.lib.common.rest_client] Request (ServerActionsTestJSON:tearDown): 200 GET http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e 0.750s
2019-05-11 21:21:00,708 215815 DEBUG    [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-OpenStack-Nova-API-Version': '2.1', 'X-Auth-Token': '<omitted>'}
        Body: None
    Response - Headers: {'date': 'Sat, 11 May 2019 21:20:59 GMT', 'server': 'Apache', 'content-length': '1701', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version,Accept-Encoding', 'x-openstack-request-id': 'req-5ec40e04-e738-4267-82e8-c0b4f5ab0b43', 'x-compute-request-id': 'req-5ec40e04-e738-4267-82e8-c0b4f5ab0b43', 'connection': 'close', 'content-type': 'application/json', 'status': '200', 'content-location': 'http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e'}
        Body: b'{"server": {"id": "fc272b73-ff3e-4448-ab85-49bcec47546e", "name": "tempest-ServerActionsTestJSON-server-1454314435", "status": "SHUTOFF", "tenant_id": "59e8e567688941f89efae0bbf6a0d3a6", "user_id": "2e0cd1029f94430aa692fbaf536afbf1", "metadata": {}, "hostId": "c3eac17a1808415e7d9c75be02baf4105d1e0e1e8221a8a29542c6e8", "image": {"id": "0e69750d-552c-4f0e-a9d6-797bc415aa7f", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/images/0e69750d-552c-4f0e-a9d6-797bc415aa7f"}]}, "flavor": {"id": "0f5c781f-d41f-46a5-a744-10f9f5c69200", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/flavors/0f5c781f-d41f-46a5-a744-10f9f5c69200"}]}, "created": "2019-05-11T21:15:10Z", "updated": "2019-05-11T21:16:12Z", "addresses": {"tempest-ServerActionsTestJSON-1252015544-network": [{"version": 4, "addr": "10.100.0.8", "OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}, {"version": 4, "addr": "10.0.0.247", "OS-EXT-IPS:type": "floating", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}]}, "accessIPv4": "", "accessIPv6": "", "links": [{"rel": "self", "href": "http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}, {"rel": "bookmark", "href": "http://10.0.0.137:8774/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}], "OS-DCF:diskConfig": "MANUAL", "OS-EXT-AZ:availability_zone": "nova", "config_drive": "", "key_name": "tempest-keypair-638249548", "OS-SRV-USG:launched_at": "2019-05-11T21:15:57.000000", "OS-SRV-USG:terminated_at": null, "security_groups": [{"name": "tempest-securitygroup--1644861636"}], "OS-EXT-STS:task_state": null, "OS-EXT-STS:vm_state": "stopped", "OS-EXT-STS:power_state": 0, "os-extended-volumes:volumes_attached": []}}'
2019-05-11 21:21:02,171 215815 INFO     [tempest.lib.common.rest_client] Request (ServerActionsTestJSON:tearDown): 200 GET http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e 0.458s
2019-05-11 21:21:02,172 215815 DEBUG    [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-OpenStack-Nova-API-Version': '2.1', 'X-Auth-Token': '<omitted>'}
        Body: None
    Response - Headers: {'date': 'Sat, 11 May 2019 21:21:01 GMT', 'server': 'Apache', 'content-length': '1701', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version,Accept-Encoding', 'x-openstack-request-id': 'req-2c4b758e-8dac-4b3b-99e5-a98864d9074d', 'x-compute-request-id': 'req-2c4b758e-8dac-4b3b-99e5-a98864d9074d', 'connection': 'close', 'content-type': 'application/json', 'status': '200', 'content-location': 'http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e'}
        Body: b'{"server": {"id": "fc272b73-ff3e-4448-ab85-49bcec47546e", "name": "tempest-ServerActionsTestJSON-server-1454314435", "status": "SHUTOFF", "tenant_id": "59e8e567688941f89efae0bbf6a0d3a6", "user_id": "2e0cd1029f94430aa692fbaf536afbf1", "metadata": {}, "hostId": "c3eac17a1808415e7d9c75be02baf4105d1e0e1e8221a8a29542c6e8", "image": {"id": "0e69750d-552c-4f0e-a9d6-797bc415aa7f", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/images/0e69750d-552c-4f0e-a9d6-797bc415aa7f"}]}, "flavor": {"id": "0f5c781f-d41f-46a5-a744-10f9f5c69200", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/flavors/0f5c781f-d41f-46a5-a744-10f9f5c69200"}]}, "created": "2019-05-11T21:15:10Z", "updated": "2019-05-11T21:16:12Z", "addresses": {"tempest-ServerActionsTestJSON-1252015544-network": [{"version": 4, "addr": "10.100.0.8", "OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}, {"version": 4, "addr": "10.0.0.247", "OS-EXT-IPS:type": "floating", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}]}, "accessIPv4": "", "accessIPv6": "", "links": [{"rel": "self", "href": "http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}, {"rel": "bookmark", "href": "http://10.0.0.137:8774/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}], "OS-DCF:diskConfig": "MANUAL", "OS-EXT-AZ:availability_zone": "nova", "config_drive": "", "key_name": "tempest-keypair-638249548", "OS-SRV-USG:launched_at": "2019-05-11T21:15:57.000000", "OS-SRV-USG:terminated_at": null, "security_groups": [{"name": "tempest-securitygroup--1644861636"}], "OS-EXT-STS:task_state": null, "OS-EXT-STS:vm_state": "stopped", "OS-EXT-STS:power_state": 0, "os-extended-volumes:volumes_attached": []}}'
2019-05-11 21:21:03,520 215815 INFO     [tempest.lib.common.rest_client] Request (ServerActionsTestJSON:tearDown): 200 GET http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e 0.342s
2019-05-11 21:21:03,520 215815 DEBUG    [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-OpenStack-Nova-API-Version': '2.1', 'X-Auth-Token': '<omitted>'}
        Body: None
    Response - Headers: {'date': 'Sat, 11 May 2019 21:21:03 GMT', 'server': 'Apache', 'content-length': '1701', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version,Accept-Encoding', 'x-openstack-request-id': 'req-bf61ffcc-bc83-4320-b3bd-6c5f07833005', 'x-compute-request-id': 'req-bf61ffcc-bc83-4320-b3bd-6c5f07833005', 'connection': 'close', 'content-type': 'application/json', 'status': '200', 'content-location': 'http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e'}
        Body: b'{"server": {"id": "fc272b73-ff3e-4448-ab85-49bcec47546e", "name": "tempest-ServerActionsTestJSON-server-1454314435", "status": "SHUTOFF", "tenant_id": "59e8e567688941f89efae0bbf6a0d3a6", "user_id": "2e0cd1029f94430aa692fbaf536afbf1", "metadata": {}, "hostId": "c3eac17a1808415e7d9c75be02baf4105d1e0e1e8221a8a29542c6e8", "image": {"id": "0e69750d-552c-4f0e-a9d6-797bc415aa7f", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/images/0e69750d-552c-4f0e-a9d6-797bc415aa7f"}]}, "flavor": {"id": "0f5c781f-d41f-46a5-a744-10f9f5c69200", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/flavors/0f5c781f-d41f-46a5-a744-10f9f5c69200"}]}, "created": "2019-05-11T21:15:10Z", "updated": "2019-05-11T21:16:12Z", "addresses": {"tempest-ServerActionsTestJSON-1252015544-network": [{"version": 4, "addr": "10.100.0.8", "OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}, {"version": 4, "addr": "10.0.0.247", "OS-EXT-IPS:type": "floating", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}]}, "accessIPv4": "", "accessIPv6": "", "links": [{"rel": "self", "href": "http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}, {"rel": "bookmark", "href": "http://10.0.0.137:8774/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}], "OS-DCF:diskConfig": "MANUAL", "OS-EXT-AZ:availability_zone": "nova", "config_drive": "", "key_name": "tempest-keypair-638249548", "OS-SRV-USG:launched_at": "2019-05-11T21:15:57.000000", "OS-SRV-USG:terminated_at": null, "security_groups": [{"name": "tempest-securitygroup--1644861636"}], "OS-EXT-STS:task_state": null, "OS-EXT-STS:vm_state": "stopped", "OS-EXT-STS:power_state": 0, "os-extended-volumes:volumes_attached": []}}'
2019-05-11 21:21:04,963 215815 INFO     [tempest.lib.common.rest_client] Request (ServerActionsTestJSON:tearDown): 200 GET http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e 0.437s
2019-05-11 21:21:04,964 215815 DEBUG    [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-OpenStack-Nova-API-Version': '2.1', 'X-Auth-Token': '<omitted>'}
        Body: None
    Response - Headers: {'date': 'Sat, 11 May 2019 21:21:04 GMT', 'server': 'Apache', 'content-length': '1701', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version,Accept-Encoding', 'x-openstack-request-id': 'req-42d4d792-318e-45b0-9d37-273d739bb6b6', 'x-compute-request-id': 'req-42d4d792-318e-45b0-9d37-273d739bb6b6', 'connection': 'close', 'content-type': 'application/json', 'status': '200', 'content-location': 'http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e'}
        Body: b'{"server": {"id": "fc272b73-ff3e-4448-ab85-49bcec47546e", "name": "tempest-ServerActionsTestJSON-server-1454314435", "status": "SHUTOFF", "tenant_id": "59e8e567688941f89efae0bbf6a0d3a6", "user_id": "2e0cd1029f94430aa692fbaf536afbf1", "metadata": {}, "hostId": "c3eac17a1808415e7d9c75be02baf4105d1e0e1e8221a8a29542c6e8", "image": {"id": "0e69750d-552c-4f0e-a9d6-797bc415aa7f", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/images/0e69750d-552c-4f0e-a9d6-797bc415aa7f"}]}, "flavor": {"id": "0f5c781f-d41f-46a5-a744-10f9f5c69200", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/flavors/0f5c781f-d41f-46a5-a744-10f9f5c69200"}]}, "created": "2019-05-11T21:15:10Z", "updated": "2019-05-11T21:16:12Z", "addresses": {"tempest-ServerActionsTestJSON-1252015544-network": [{"version": 4, "addr": "10.100.0.8", "OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}, {"version": 4, "addr": "10.0.0.247", "OS-EXT-IPS:type": "floating", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}]}, "accessIPv4": "", "accessIPv6": "", "links": [{"rel": "self", "href": "http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}, {"rel": "bookmark", "href": "http://10.0.0.137:8774/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}], "OS-DCF:diskConfig": "MANUAL", "OS-EXT-AZ:availability_zone": "nova", "config_drive": "", "key_name": "tempest-keypair-638249548", "OS-SRV-USG:launched_at": "2019-05-11T21:15:57.000000", "OS-SRV-USG:terminated_at": null, "security_groups": [{"name": "tempest-securitygroup--1644861636"}], "OS-EXT-STS:task_state": null, "OS-EXT-STS:vm_state": "stopped", "OS-EXT-STS:power_state": 0, "os-extended-volumes:volumes_attached": []}}'
2019-05-11 21:21:06,502 215815 INFO     [tempest.lib.common.rest_client] Request (ServerActionsTestJSON:tearDown): 200 GET http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e 0.532s
2019-05-11 21:21:06,503 215815 DEBUG    [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-OpenStack-Nova-API-Version': '2.1', 'X-Auth-Token': '<omitted>'}
        Body: None
    Response - Headers: {'date': 'Sat, 11 May 2019 21:21:05 GMT', 'server': 'Apache', 'content-length': '1701', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version,Accept-Encoding', 'x-openstack-request-id': 'req-69b1efec-7c76-4891-84d2-1e5bb2c42ceb', 'x-compute-request-id': 'req-69b1efec-7c76-4891-84d2-1e5bb2c42ceb', 'connection': 'close', 'content-type': 'application/json', 'status': '200', 'content-location': 'http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e'}
        Body: b'{"server": {"id": "fc272b73-ff3e-4448-ab85-49bcec47546e", "name": "tempest-ServerActionsTestJSON-server-1454314435", "status": "SHUTOFF", "tenant_id": "59e8e567688941f89efae0bbf6a0d3a6", "user_id": "2e0cd1029f94430aa692fbaf536afbf1", "metadata": {}, "hostId": "c3eac17a1808415e7d9c75be02baf4105d1e0e1e8221a8a29542c6e8", "image": {"id": "0e69750d-552c-4f0e-a9d6-797bc415aa7f", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/images/0e69750d-552c-4f0e-a9d6-797bc415aa7f"}]}, "flavor": {"id": "0f5c781f-d41f-46a5-a744-10f9f5c69200", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/flavors/0f5c781f-d41f-46a5-a744-10f9f5c69200"}]}, "created": "2019-05-11T21:15:10Z", "updated": "2019-05-11T21:16:12Z", "addresses": {"tempest-ServerActionsTestJSON-1252015544-network": [{"version": 4, "addr": "10.100.0.8", "OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}, {"version": 4, "addr": "10.0.0.247", "OS-EXT-IPS:type": "floating", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}]}, "accessIPv4": "", "accessIPv6": "", "links": [{"rel": "self", "href": "http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}, {"rel": "bookmark", "href": "http://10.0.0.137:8774/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}], "OS-DCF:diskConfig": "MANUAL", "OS-EXT-AZ:availability_zone": "nova", "config_drive": "", "key_name": "tempest-keypair-638249548", "OS-SRV-USG:launched_at": "2019-05-11T21:15:57.000000", "OS-SRV-USG:terminated_at": null, "security_groups": [{"name": "tempest-securitygroup--1644861636"}], "OS-EXT-STS:task_state": null, "OS-EXT-STS:vm_state": "stopped", "OS-EXT-STS:power_state": 0, "os-extended-volumes:volumes_attached": []}}'
2019-05-11 21:21:06,508 215815 ERROR    [tempest.api.compute.base] Request timed out
Details: (ServerActionsTestJSON:tearDown) Server fc272b73-ff3e-4448-ab85-49bcec47546e failed to reach ACTIVE status and task state "None" within the required time (300 s). Current status: SHUTOFF. Current task state: None.
2019-05-11 21:21:06.508 215815 ERROR tempest.api.compute.base Traceback (most recent call last):
2019-05-11 21:21:06.508 215815 ERROR tempest.api.compute.base   File "/usr/lib/python3.6/site-packages/tempest/api/compute/base.py", line 201, in server_check_teardown
2019-05-11 21:21:06.508 215815 ERROR tempest.api.compute.base     cls.server_id, 'ACTIVE')
2019-05-11 21:21:06.508 215815 ERROR tempest.api.compute.base   File "/usr/lib/python3.6/site-packages/tempest/common/waiters.py", line 96, in wait_for_server_status
2019-05-11 21:21:06.508 215815 ERROR tempest.api.compute.base     raise lib_exc.TimeoutException(message)
2019-05-11 21:21:06.508 215815 ERROR tempest.api.compute.base tempest.lib.exceptions.TimeoutException: Request timed out
2019-05-11 21:21:06.508 215815 ERROR tempest.api.compute.base Details: (ServerActionsTestJSON:tearDown) Server fc272b73-ff3e-4448-ab85-49bcec47546e failed to reach ACTIVE status and task state "None" within the required time (300 s). Current status: SHUTOFF. Current task state: None.
2019-05-11 21:21:06.508 215815 ERROR tempest.api.compute.base 
2019-05-11 21:21:06,733 215815 INFO     [tempest.lib.common.rest_client] Request (ServerActionsTestJSON:tearDown): 204 DELETE http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e 0.224s
2019-05-11 21:21:06,733 215815 DEBUG    [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-OpenStack-Nova-API-Version': '2.1', 'X-Auth-Token': '<omitted>'}
        Body: None
    Response - Headers: {'date': 'Sat, 11 May 2019 21:21:06 GMT', 'server': 'Apache', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version', 'x-openstack-request-id': 'req-05079baa-4eb5-4a41-8820-00f3fb231c34', 'x-compute-request-id': 'req-05079baa-4eb5-4a41-8820-00f3fb231c34', 'connection': 'close', 'status': '204', 'content-location': 'http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e'}
        Body: b''
2019-05-11 21:21:07,705 215815 INFO     [tempest.lib.common.rest_client] Request (ServerActionsTestJSON:tearDown): 200 GET http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e 0.971s
2019-05-11 21:21:07,705 215815 DEBUG    [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-OpenStack-Nova-API-Version': '2.1', 'X-Auth-Token': '<omitted>'}
        Body: None
    Response - Headers: {'date': 'Sat, 11 May 2019 21:21:06 GMT', 'server': 'Apache', 'content-length': '1707', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version,Accept-Encoding', 'x-openstack-request-id': 'req-0d4610bc-0216-426e-a72e-a17442ccca9b', 'x-compute-request-id': 'req-0d4610bc-0216-426e-a72e-a17442ccca9b', 'connection': 'close', 'content-type': 'application/json', 'status': '200', 'content-location': 'http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e'}
        Body: b'{"server": {"id": "fc272b73-ff3e-4448-ab85-49bcec47546e", "name": "tempest-ServerActionsTestJSON-server-1454314435", "status": "SHUTOFF", "tenant_id": "59e8e567688941f89efae0bbf6a0d3a6", "user_id": "2e0cd1029f94430aa692fbaf536afbf1", "metadata": {}, "hostId": "c3eac17a1808415e7d9c75be02baf4105d1e0e1e8221a8a29542c6e8", "image": {"id": "0e69750d-552c-4f0e-a9d6-797bc415aa7f", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/images/0e69750d-552c-4f0e-a9d6-797bc415aa7f"}]}, "flavor": {"id": "0f5c781f-d41f-46a5-a744-10f9f5c69200", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/flavors/0f5c781f-d41f-46a5-a744-10f9f5c69200"}]}, "created": "2019-05-11T21:15:10Z", "updated": "2019-05-11T21:21:06Z", "addresses": {"tempest-ServerActionsTestJSON-1252015544-network": [{"version": 4, "addr": "10.100.0.8", "OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}, {"version": 4, "addr": "10.0.0.247", "OS-EXT-IPS:type": "floating", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}]}, "accessIPv4": "", "accessIPv6": "", "links": [{"rel": "self", "href": "http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}, {"rel": "bookmark", "href": "http://10.0.0.137:8774/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}], "OS-DCF:diskConfig": "MANUAL", "OS-EXT-AZ:availability_zone": "nova", "config_drive": "", "key_name": "tempest-keypair-638249548", "OS-SRV-USG:launched_at": "2019-05-11T21:15:57.000000", "OS-SRV-USG:terminated_at": null, "security_groups": [{"name": "tempest-securitygroup--1644861636"}], "OS-EXT-STS:task_state": "deleting", "OS-EXT-STS:vm_state": "stopped", "OS-EXT-STS:power_state": 0, "os-extended-volumes:volumes_attached": []}}'
2019-05-11 21:21:09,085 215815 INFO     [tempest.lib.common.rest_client] Request (ServerActionsTestJSON:tearDown): 200 GET http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e 0.373s
2019-05-11 21:21:09,085 215815 DEBUG    [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-OpenStack-Nova-API-Version': '2.1', 'X-Auth-Token': '<omitted>'}
        Body: None
    Response - Headers: {'date': 'Sat, 11 May 2019 21:21:08 GMT', 'server': 'Apache', 'content-length': '1707', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version,Accept-Encoding', 'x-openstack-request-id': 'req-78692737-83db-4caa-851d-194ffec7e0b7', 'x-compute-request-id': 'req-78692737-83db-4caa-851d-194ffec7e0b7', 'connection': 'close', 'content-type': 'application/json', 'status': '200', 'content-location': 'http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e'}
        Body: b'{"server": {"id": "fc272b73-ff3e-4448-ab85-49bcec47546e", "name": "tempest-ServerActionsTestJSON-server-1454314435", "status": "SHUTOFF", "tenant_id": "59e8e567688941f89efae0bbf6a0d3a6", "user_id": "2e0cd1029f94430aa692fbaf536afbf1", "metadata": {}, "hostId": "c3eac17a1808415e7d9c75be02baf4105d1e0e1e8221a8a29542c6e8", "image": {"id": "0e69750d-552c-4f0e-a9d6-797bc415aa7f", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/images/0e69750d-552c-4f0e-a9d6-797bc415aa7f"}]}, "flavor": {"id": "0f5c781f-d41f-46a5-a744-10f9f5c69200", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/flavors/0f5c781f-d41f-46a5-a744-10f9f5c69200"}]}, "created": "2019-05-11T21:15:10Z", "updated": "2019-05-11T21:21:06Z", "addresses": {"tempest-ServerActionsTestJSON-1252015544-network": [{"version": 4, "addr": "10.100.0.8", "OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}, {"version": 4, "addr": "10.0.0.247", "OS-EXT-IPS:type": "floating", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:7b:23:5d"}]}, "accessIPv4": "", "accessIPv6": "", "links": [{"rel": "self", "href": "http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}, {"rel": "bookmark", "href": "http://10.0.0.137:8774/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}], "OS-DCF:diskConfig": "MANUAL", "OS-EXT-AZ:availability_zone": "nova", "config_drive": "", "key_name": "tempest-keypair-638249548", "OS-SRV-USG:launched_at": "2019-05-11T21:15:57.000000", "OS-SRV-USG:terminated_at": null, "security_groups": [{"name": "tempest-securitygroup--1644861636"}], "OS-EXT-STS:task_state": "deleting", "OS-EXT-STS:vm_state": "stopped", "OS-EXT-STS:power_state": 0, "os-extended-volumes:volumes_attached": []}}'
2019-05-11 21:21:10,288 215815 INFO     [tempest.lib.common.rest_client] Request (ServerActionsTestJSON:tearDown): 200 GET http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e 0.196s
2019-05-11 21:21:10,289 215815 DEBUG    [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-OpenStack-Nova-API-Version': '2.1', 'X-Auth-Token': '<omitted>'}
        Body: None
    Response - Headers: {'date': 'Sat, 11 May 2019 21:21:10 GMT', 'server': 'Apache', 'content-length': '1583', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version,Accept-Encoding', 'x-openstack-request-id': 'req-9abf1275-1cbf-401d-99f5-7c512510012e', 'x-compute-request-id': 'req-9abf1275-1cbf-401d-99f5-7c512510012e', 'connection': 'close', 'content-type': 'application/json', 'status': '200', 'content-location': 'http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e'}
        Body: b'{"server": {"id": "fc272b73-ff3e-4448-ab85-49bcec47546e", "name": "tempest-ServerActionsTestJSON-server-1454314435", "status": "DELETED", "tenant_id": "59e8e567688941f89efae0bbf6a0d3a6", "user_id": "2e0cd1029f94430aa692fbaf536afbf1", "metadata": {}, "hostId": "c3eac17a1808415e7d9c75be02baf4105d1e0e1e8221a8a29542c6e8", "image": {"id": "0e69750d-552c-4f0e-a9d6-797bc415aa7f", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/images/0e69750d-552c-4f0e-a9d6-797bc415aa7f"}]}, "flavor": {"id": "0f5c781f-d41f-46a5-a744-10f9f5c69200", "links": [{"rel": "bookmark", "href": "http://10.0.0.137:8774/flavors/0f5c781f-d41f-46a5-a744-10f9f5c69200"}]}, "created": "2019-05-11T21:15:10Z", "updated": "2019-05-11T21:21:09Z", "addresses": {}, "accessIPv4": "", "accessIPv6": "", "links": [{"rel": "self", "href": "http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}, {"rel": "bookmark", "href": "http://10.0.0.137:8774/servers/fc272b73-ff3e-4448-ab85-49bcec47546e"}], "OS-DCF:diskConfig": "MANUAL", "fault": {"code": 500, "created": "2019-05-11T21:16:08Z", "message": "Cannot access storage file \'/var/lib/nova/instances/fc272b73-ff3e-4448-ab85-49bcec47546e/disk\' (as uid:107, gid:107): Permission denied"}, "OS-EXT-AZ:availability_zone": "nova", "config_drive": "", "key_name": "tempest-keypair-638249548", "OS-SRV-USG:launched_at": "2019-05-11T21:15:57.000000", "OS-SRV-USG:terminated_at": "2019-05-11T21:21:09.000000", "OS-EXT-STS:task_state": null, "OS-EXT-STS:vm_state": "deleted", "OS-EXT-STS:power_state": 0, "os-extended-volumes:volumes_attached": []}}'
2019-05-11 21:21:10,294 215815 INFO     [tempest.common.waiters] State transition "SHUTOFF/deleting" ==> "DELETED/None" after 3 second wait
2019-05-11 21:21:11,358 215815 INFO     [tempest.lib.common.rest_client] Request (ServerActionsTestJSON:tearDown): 404 GET http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e 0.063s
2019-05-11 21:21:11,359 215815 DEBUG    [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-OpenStack-Nova-API-Version': '2.1', 'X-Auth-Token': '<omitted>'}
        Body: None
    Response - Headers: {'date': 'Sat, 11 May 2019 21:21:11 GMT', 'server': 'Apache', 'content-length': '111', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version', 'x-openstack-request-id': 'req-dd453706-d75e-4d85-a373-576bc4dbcdc0', 'x-compute-request-id': 'req-dd453706-d75e-4d85-a373-576bc4dbcdc0', 'connection': 'close', 'content-type': 'application/json; charset=UTF-8', 'status': '404', 'content-location': 'http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e'}
        Body: b'{"itemNotFound": {"code": 404, "message": "Instance fc272b73-ff3e-4448-ab85-49bcec47546e could not be found."}}'
2019-05-11 21:21:11,431 215815 INFO     [tempest.lib.common.rest_client] Request (ServerActionsTestJSON:_run_cleanups): 404 DELETE http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e 0.072s
2019-05-11 21:21:11,432 215815 DEBUG    [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-OpenStack-Nova-API-Version': '2.1', 'X-Auth-Token': '<omitted>'}
        Body: None
    Response - Headers: {'date': 'Sat, 11 May 2019 21:21:11 GMT', 'server': 'Apache', 'content-length': '111', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version', 'x-openstack-request-id': 'req-2dd3cc26-e04a-4a54-863a-f4bbc7eb49e7', 'x-compute-request-id': 'req-2dd3cc26-e04a-4a54-863a-f4bbc7eb49e7', 'connection': 'close', 'content-type': 'application/json; charset=UTF-8', 'status': '404', 'content-location': 'http://10.0.0.137:8774/v2.1/servers/fc272b73-ff3e-4448-ab85-49bcec47546e'}
        Body: b'{"itemNotFound": {"code": 404, "message": "Instance fc272b73-ff3e-4448-ab85-49bcec47546e could not be found."}}'
2019-05-11 21:21:11,432 215815 ERROR    [tempest.api.compute.base] Failed to delete server fc272b73-ff3e-4448-ab85-49bcec47546e
2019-05-11 21:21:11.432 215815 ERROR tempest.api.compute.base Traceback (most recent call last):
2019-05-11 21:21:11.432 215815 ERROR tempest.api.compute.base   File "/usr/lib/python3.6/site-packages/tempest/api/compute/base.py", line 414, in delete_server
2019-05-11 21:21:11.432 215815 ERROR tempest.api.compute.base     cls.servers_client.delete_server(server_id)
2019-05-11 21:21:11.432 215815 ERROR tempest.api.compute.base   File "/usr/lib/python3.6/site-packages/tempest/lib/services/compute/servers_client.py", line 152, in delete_server
2019-05-11 21:21:11.432 215815 ERROR tempest.api.compute.base     resp, body = self.delete("servers/%s" % server_id)
2019-05-11 21:21:11.432 215815 ERROR tempest.api.compute.base   File "/usr/lib/python3.6/site-packages/tempest/lib/common/rest_client.py", line 314, in delete
2019-05-11 21:21:11.432 215815 ERROR tempest.api.compute.base     return self.request('DELETE', url, extra_headers, headers, body)
2019-05-11 21:21:11.432 215815 ERROR tempest.api.compute.base   File "/usr/lib/python3.6/site-packages/tempest/lib/services/compute/base_compute_client.py", line 48, in request
2019-05-11 21:21:11.432 215815 ERROR tempest.api.compute.base     method, url, extra_headers, headers, body, chunked)
2019-05-11 21:21:11.432 215815 ERROR tempest.api.compute.base   File "/usr/lib/python3.6/site-packages/tempest/lib/common/rest_client.py", line 679, in request
2019-05-11 21:21:11.432 215815 ERROR tempest.api.compute.base     self._error_checker(resp, resp_body)
2019-05-11 21:21:11.432 215815 ERROR tempest.api.compute.base   File "/usr/lib/python3.6/site-packages/tempest/lib/common/rest_client.py", line 785, in _error_checker
2019-05-11 21:21:11.432 215815 ERROR tempest.api.compute.base     raise exceptions.NotFound(resp_body, resp=resp)
2019-05-11 21:21:11.432 215815 ERROR tempest.api.compute.base tempest.lib.exceptions.NotFound: Object not found
2019-05-11 21:21:11.432 215815 ERROR tempest.api.compute.base Details: {'code': 404, 'message': 'Instance fc272b73-ff3e-4448-ab85-49bcec47546e could not be found.'}
2019-05-11 21:21:11.432 215815 ERROR tempest.api.compute.base
}}}

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/tempest/api/compute/servers/test_server_actions.py", line 66, in tearDown
    self.server_check_teardown()
  File "/usr/lib/python3.6/site-packages/tempest/api/compute/base.py", line 201, in server_check_teardown
    cls.server_id, 'ACTIVE')
  File "/usr/lib/python3.6/site-packages/tempest/common/waiters.py", line 96, in wait_for_server_status
    raise lib_exc.TimeoutException(message)
tempest.lib.exceptions.TimeoutException: Request timed out
Details: (ServerActionsTestJSON:tearDown) Server fc272b73-ff3e-4448-ab85-49bcec47546e failed to reach ACTIVE status and task state "None" within the required time (300 s). Current status: SHUTOFF. Current task state: None.

Comment 2 Artom Lifshitz 2019-05-16 19:23:38 UTC

This is weird in tempest, but here's how the flow goes:

1. test_resize_server_confirm_from_stopped()  [https://github.com/openstack/tempest/blob/20.0.0/tempest/api/compute/servers/test_server_actions.py#L357]
2. _test_resize_server_confirm(... stop=True) [https://github.com/openstack/tempest/blob/20.0.0/tempest/api/compute/servers/test_server_actions.py#L292]
3. <start the server after the test>          [https://github.com/openstack/tempest/blob/20.0.0/tempest/api/compute/servers/test_server_actions.py#L318]
4. tearDown()                                 [https://github.com/openstack/tempest/blob/20.0.0/tempest/api/compute/servers/test_server_actions.py#L61]
5. <wait for server ACTIVE from 3>            [https://github.com/openstack/tempest/blob/20.0.0/tempest/api/compute/base.py#L200]

5 is the part that failed. 3's request ID to start the server is req-93ccce31-1d79-47a7-933a-c9fadde37460. In the Nova logs we see:

2019-05-11 21:16:04.648 [./controller-0/var/log/containers/nova/nova-api.log] 21 DEBUG nova.api.openstack.wsgi [req-93ccce31-1d79-47a7-933a-c9fadde37460 2e0cd1029f94430aa692fbaf536afbf1 59e8e567688941f89efae0bbf6a0d3a6 - default default] Action: 'action', calling method: <bound method ServersController._start_server of <nova.api.openstack.compute.servers.ServersController object at 0x7ff527d14c18>>, body: {"os-start": {}} _process_stack /usr/lib/python3.6/site-packages/nova/api/openstack/wsgi.py:520

2019-05-11 21:16:08.546 [./compute-0/var/log/containers/nova/nova-compute.log] 8 ERROR nova.virt.libvirt.driver [req-93ccce31-1d79-47a7-933a-c9fadde37460 2e0cd1029f94430aa692fbaf536afbf1 59e8e567688941f89efae0bbf6a0d3a6 - default default] [instance: fc272b73-ff3e-4448-ab85-49bcec47546e] Failed to start libvirt guest: libvirt.libvirtError: Cannot access storage file '/var/lib/nova/instances/fc272b73-ff3e-4448-ab85-49bcec47546e/disk' (as uid:107, gid:107): Permission denied

2019-05-11 21:16:08.764 [./compute-0/var/log/containers/nova/nova-compute.log] 8 ERROR oslo_messaging.rpc.server [req-93ccce31-1d79-47a7-933a-c9fadde37460 2e0cd1029f94430aa692fbaf536afbf1 59e8e567688941f89efae0bbf6a0d3a6 - default default] Exception during message handling: libvirt.libvirtError: Cannot access storage file '/var/lib/nova/instances/fc272b73-ff3e-4448-ab85-49bcec47546e/disk' (as uid:107, gid:107): Permission denied

So to me this smells deployment-y...

Comment 3 Matthew Booth 2019-05-17 12:46:03 UTC


*** This bug has been marked as a duplicate of bug 1710034 ***