Bug 1362682 - Instance was deleted successfully without detaching its volume, if nova-compute was killed during running "nova delete"
Summary: Instance was deleted successfully without detaching its volume, if nova-compu...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-cinder
Version: 8.0 (Liberty)
Hardware: x86_64
OS: Linux
high
medium
Target Milestone: async
: 8.0 (Liberty)
Assignee: Eric Harney
QA Contact: lkuchlan
URL:
Whiteboard:
: 1344177 (view as bug list)
Depends On: 1316791 1363625
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-08-02 20:50 UTC by Tom Barron
Modified: 2020-03-11 15:11 UTC (History)
27 users (show)

Fixed In Version: openstack-cinder-7.0.2-3.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1316791
Environment:
Last Closed: 2016-11-17 19:56:12 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Comment 2 Elise Gafford 2016-08-08 16:02:46 UTC
*** Bug 1344177 has been marked as a duplicate of this bug. ***

Comment 4 lkuchlan 2016-08-22 09:06:55 UTC
Tested using:
python-cinderclient-1.5.0-1.el7ost.noarch
openstack-cinder-7.0.2-3.el7ost.noarch
python-cinder-7.0.2-3.el7ost.noarch

Verification flow:

[root@dhcp163-102 ~(keystone_admin)]# cinder create 1 --image-id 81f7d67a-daf7-4b88-88a0-9afb5f2c2ca4
+---------------------------------------+--------------------------------------+
|                Property               |                Value                 |
+---------------------------------------+--------------------------------------+
|              attachments              |                  []                  |
|           availability_zone           |                 nova                 |
|                bootable               |                false                 |
|          consistencygroup_id          |                 None                 |
|               created_at              |      2016-08-22T08:43:16.000000      |
|              description              |                 None                 |
|               encrypted               |                False                 |
|                   id                  | badeba7c-8289-4ed2-9336-3a4db2c3600d |
|                metadata               |                  {}                  |
|            migration_status           |                 None                 |
|              multiattach              |                False                 |
|                  name                 |                 None                 |
|         os-vol-host-attr:host         |                 None                 |
|     os-vol-mig-status-attr:migstat    |                 None                 |
|     os-vol-mig-status-attr:name_id    |                 None                 |
|      os-vol-tenant-attr:tenant_id     |   8f59ec3f88f04c01bc31c252eea2e462   |
|   os-volume-replication:driver_data   |                 None                 |
| os-volume-replication:extended_status |                 None                 |
|           replication_status          |               disabled               |
|                  size                 |                  1                   |
|              snapshot_id              |                 None                 |
|              source_volid             |                 None                 |
|                 status                |               creating               |
|                user_id                |   ce9b248de50349b4b46d9975c3952e5c   |
|              volume_type              |                 None                 |
+---------------------------------------+--------------------------------------+

[root@dhcp163-102 ~(keystone_admin)]# cinder list
+--------------------------------------+-----------+------------------+------+------+-------------+----------+-------------+-------------+
|                  ID                  |   Status  | Migration Status | Name | Size | Volume Type | Bootable | Multiattach | Attached to |
+--------------------------------------+-----------+------------------+------+------+-------------+----------+-------------+-------------+
| badeba7c-8289-4ed2-9336-3a4db2c3600d | available |        -         |  -   |  1   |      -      |   true   |    False    |             |
+--------------------------------------+-----------+------------------+------+------+-------------+----------+-------------+-------------+

[root@dhcp163-102 ~(keystone_admin)]# nova boot --boot-volume badeba7c-8289-4ed2-9336-3a4db2c3600d --flavor 1 --nic net-id=3484458c-a479-488b-8f1a-e265549e7ed7 vm
+--------------------------------------+--------------------------------------------------+
| Property                             | Value                                            |
+--------------------------------------+--------------------------------------------------+
| OS-DCF:diskConfig                    | MANUAL                                           |
| OS-EXT-AZ:availability_zone          |                                                  |
| OS-EXT-SRV-ATTR:host                 | -                                                |
| OS-EXT-SRV-ATTR:hypervisor_hostname  | -                                                |
| OS-EXT-SRV-ATTR:instance_name        | instance-00000001                                |
| OS-EXT-STS:power_state               | 0                                                |
| OS-EXT-STS:task_state                | scheduling                                       |
| OS-EXT-STS:vm_state                  | building                                         |
| OS-SRV-USG:launched_at               | -                                                |
| OS-SRV-USG:terminated_at             | -                                                |
| accessIPv4                           |                                                  |
| accessIPv6                           |                                                  |
| adminPass                            | RpBZZXhHt2ar                                     |
| config_drive                         |                                                  |
| created                              | 2016-08-22T08:44:48Z                             |
| flavor                               | m1.tiny (1)                                      |
| hostId                               |                                                  |
| id                                   | 8d2ed590-1ac9-45e6-87f4-eafa17288955             |
| image                                | Attempt to boot from volume - no image supplied  |
| key_name                             | -                                                |
| metadata                             | {}                                               |
| name                                 | vm                                               |
| os-extended-volumes:volumes_attached | [{"id": "badeba7c-8289-4ed2-9336-3a4db2c3600d"}] |
| progress                             | 0                                                |
| security_groups                      | default                                          |
| status                               | BUILD                                            |
| tenant_id                            | 8f59ec3f88f04c01bc31c252eea2e462                 |
| updated                              | 2016-08-22T08:44:49Z                             |
| user_id                              | ce9b248de50349b4b46d9975c3952e5c                 |
+--------------------------------------+--------------------------------------------------+

[root@dhcp163-102 ~(keystone_admin)]# nova list
+--------------------------------------+------+--------+------------+-------------+------------------+
| ID                                   | Name | Status | Task State | Power State | Networks         |
+--------------------------------------+------+--------+------------+-------------+------------------+
| 8d2ed590-1ac9-45e6-87f4-eafa17288955 | vm   | ACTIVE | -          | Running     | private=10.0.0.3 |
+--------------------------------------+------+--------+------------+-------------+------------------+

[root@dhcp163-102 ~(keystone_admin)]# openstack-service status nova-compute
MainPID=24886 Id=openstack-nova-compute.service ActiveState=active

[root@dhcp163-102 ~(keystone_admin)]# openstack-service stop nova-compute

[root@dhcp163-102 ~(keystone_admin)]# openstack-service status nova-compute
MainPID=0 Id=openstack-nova-compute.service ActiveState=inactive

[root@dhcp163-102 ~(keystone_admin)]# nova delete 8d2ed590-1ac9-45e6-87f4-eafa17288955
Request to delete server 8d2ed590-1ac9-45e6-87f4-eafa17288955 has been accepted.

[root@dhcp163-102 ~(keystone_admin)]# openstack-service start nova-compute

[root@dhcp163-102 ~(keystone_admin)]# openstack-service status nova-compute
MainPID=24886 Id=openstack-nova-compute.service ActiveState=active

[root@dhcp163-102 ~(keystone_admin)]# nova list
+----+------+--------+------------+-------------+----------+
| ID | Name | Status | Task State | Power State | Networks |
+----+------+--------+------------+-------------+----------+
+----+------+--------+------------+-------------+----------+

[root@dhcp163-102 ~(keystone_admin)]# cinder list
+--------------------------------------+--------+------------------+------+------+-------------+----------+-------------+--------------------------------------+
|                  ID                  | Status | Migration Status | Name | Size | Volume Type | Bootable | Multiattach |             Attached to              |
+--------------------------------------+--------+------------------+------+------+-------------+----------+-------------+--------------------------------------+
| badeba7c-8289-4ed2-9336-3a4db2c3600d | in-use |        -         |  -   |  1   |      -      |   true   |    False    | 8d2ed590-1ac9-45e6-87f4-eafa17288955 |
+--------------------------------------+--------+------------------+------+------+-------------+----------+-------------+--------------------------------------+

Comment 5 Gorka Eguileor 2016-08-23 09:03:58 UTC
(In reply to lkuchlan from comment #4)
> Tested using:
> [root@dhcp163-102 ~(keystone_admin)]# cinder list
> +--------------------------------------+--------+------------------+------+--
> ----+-------------+----------+-------------+---------------------------------
> -----+
> |                  ID                  | Status | Migration Status | Name |
> Size | Volume Type | Bootable | Multiattach |             Attached to       
> |
> +--------------------------------------+--------+------------------+------+--
> ----+-------------+----------+-------------+---------------------------------
> -----+
> | badeba7c-8289-4ed2-9336-3a4db2c3600d | in-use |        -         |  -   | 
> 1   |      -      |   true   |    False    |
> 8d2ed590-1ac9-45e6-87f4-eafa17288955 |
> +--------------------------------------+--------+------------------+------+--
> ----+-------------+----------+-------------+---------------------------------
> -----+

Verification flow is missing the last 2 steps where the fix can be tested (the fix is the rese-state of in-use volumes):

$ cinder reset-state --state available --attach-status detached badeba7c-8289-4ed2-9336-3a4db2c3600d 

$ nova boot --boot-volume badeba7c-8289-4ed2-9336-3a4db2c3600d --flavor 1 --nic net-id=3484458c-a479-488b-8f1a-e265549e7ed7 vm

But this cannot be tested yet because this fix requires a new cinderclient build that includes fix for https://bugzilla.redhat.com/show_bug.cgi?id=1363625

Comment 11 lkuchlan 2016-10-31 14:31:48 UTC
Tested using:
openstack-cinder-7.0.2-4.el7ost.noarch

Verification flow:

[root@dhcp163-21 ~(keystone_admin)]# cinder create 1 --image-id 580a2858-0155-42ed-a42b-6434b3813ebc
+---------------------------------------+--------------------------------------+
|                Property               |                Value                 |
+---------------------------------------+--------------------------------------+
|              attachments              |                  []                  |
|           availability_zone           |                 nova                 |
|                bootable               |                false                 |
|          consistencygroup_id          |                 None                 |
|               created_at              |      2016-10-31T14:13:27.000000      |
|              description              |                 None                 |
|               encrypted               |                False                 |
|                   id                  | 329eff69-6407-4b23-b09e-a1383ffead68 |
|                metadata               |                  {}                  |
|            migration_status           |                 None                 |
|              multiattach              |                False                 |
|                  name                 |                 None                 |
|         os-vol-host-attr:host         |                 None                 |
|     os-vol-mig-status-attr:migstat    |                 None                 |
|     os-vol-mig-status-attr:name_id    |                 None                 |
|      os-vol-tenant-attr:tenant_id     |   8aa0b26a0237412097a3f58e9d3c79ff   |
|   os-volume-replication:driver_data   |                 None                 |                                                                                                                                   
| os-volume-replication:extended_status |                 None                 |                                                                                                                                   
|           replication_status          |               disabled               |                                                                                                                                   
|                  size                 |                  1                   |                                                                                                                                   
|              snapshot_id              |                 None                 |                                                                                                                                   
|              source_volid             |                 None                 |                                                                                                                                   
|                 status                |               creating               |                                                                                                                                   
|                user_id                |   5fc876721fe3436a9043f0fbb79ec3c5   |                                                                                                                                   
|              volume_type              |                 None                 |                                                                                                                                   
+---------------------------------------+--------------------------------------+                                                                                                                                   
[root@dhcp163-21 ~(keystone_admin)]# cinder list                                                                                                                                                                   
+--------------------------------------+-------------+------------------+------+------+-------------+----------+-------------+-------------+                                                                       
|                  ID                  |    Status   | Migration Status | Name | Size | Volume Type | Bootable | Multiattach | Attached to |                                                                       
+--------------------------------------+-------------+------------------+------+------+-------------+----------+-------------+-------------+                                                                       
| 329eff69-6407-4b23-b09e-a1383ffead68 | downloading |        -         |  -   |  1   |      -      |  false   |    False    |             |                                                                       
+--------------------------------------+-------------+------------------+------+------+-------------+----------+-------------+-------------+                                                                       
[root@dhcp163-21 ~(keystone_admin)]# cinder list                                                                                                                                                                   
+--------------------------------------+-----------+------------------+------+------+-------------+----------+-------------+-------------+
|                  ID                  |   Status  | Migration Status | Name | Size | Volume Type | Bootable | Multiattach | Attached to |
+--------------------------------------+-----------+------------------+------+------+-------------+----------+-------------+-------------+
| 329eff69-6407-4b23-b09e-a1383ffead68 | available |        -         |  -   |  1   |      -      |   true   |    False    |             |
+--------------------------------------+-----------+------------------+------+------+-------------+----------+-------------+-------------+

[root@dhcp163-21 ~(keystone_admin)]# nova boot --boot-volume 329eff69-6407-4b23-b09e-a1383ffead68 --flavor 1 --nic net-id=ac477bb1-2a0c-4e51-b011-b177c63420d4 vm
+--------------------------------------+--------------------------------------------------+
| Property                             | Value                                            |
+--------------------------------------+--------------------------------------------------+
| OS-DCF:diskConfig                    | MANUAL                                           |
| OS-EXT-AZ:availability_zone          |                                                  |
| OS-EXT-SRV-ATTR:host                 | -                                                |
| OS-EXT-SRV-ATTR:hypervisor_hostname  | -                                                |
| OS-EXT-SRV-ATTR:instance_name        | instance-00000001                                |
| OS-EXT-STS:power_state               | 0                                                |
| OS-EXT-STS:task_state                | scheduling                                       |
| OS-EXT-STS:vm_state                  | building                                         |
| OS-SRV-USG:launched_at               | -                                                |
| OS-SRV-USG:terminated_at             | -                                                |
| accessIPv4                           |                                                  |
| accessIPv6                           |                                                  |
| adminPass                            | oQ2yRZpq2cdU                                     |
| config_drive                         |                                                  |
| created                              | 2016-10-31T14:19:37Z                             |
| flavor                               | m1.tiny (1)                                      |
| hostId                               |                                                  |
| id                                   | d19e38b0-0025-45c7-bf3e-538389b77014             |
| image                                | Attempt to boot from volume - no image supplied  |
| key_name                             | -                                                |
| metadata                             | {}                                               |
| name                                 | vm                                               |
| os-extended-volumes:volumes_attached | [{"id": "329eff69-6407-4b23-b09e-a1383ffead68"}] |
| progress                             | 0                                                |
| security_groups                      | default                                          |
| status                               | BUILD                                            |
| tenant_id                            | 8aa0b26a0237412097a3f58e9d3c79ff                 |
| updated                              | 2016-10-31T14:19:38Z                             |
| user_id                              | 5fc876721fe3436a9043f0fbb79ec3c5                 |
+--------------------------------------+--------------------------------------------------+
[root@dhcp163-21 ~(keystone_admin)]# nova list
+--------------------------------------+------+--------+----------------------+-------------+------------------+
| ID                                   | Name | Status | Task State           | Power State | Networks         |
+--------------------------------------+------+--------+----------------------+-------------+------------------+
| d19e38b0-0025-45c7-bf3e-538389b77014 | vm   | BUILD  | block_device_mapping | NOSTATE     | private=10.0.0.3 |
+--------------------------------------+------+--------+----------------------+-------------+------------------+

[root@dhcp163-21 ~(keystone_admin)]# nova list
+--------------------------------------+------+--------+------------+-------------+------------------+
| ID                                   | Name | Status | Task State | Power State | Networks         |
+--------------------------------------+------+--------+------------+-------------+------------------+
| d19e38b0-0025-45c7-bf3e-538389b77014 | vm   | ACTIVE | -          | Running     | private=10.0.0.3 |
+--------------------------------------+------+--------+------------+-------------+------------------+

[root@dhcp163-21 ~(keystone_admin)]# cinder list
+--------------------------------------+--------+------------------+------+------+-------------+----------+-------------+--------------------------------------+
|                  ID                  | Status | Migration Status | Name | Size | Volume Type | Bootable | Multiattach |             Attached to              |
+--------------------------------------+--------+------------------+------+------+-------------+----------+-------------+--------------------------------------+
| 329eff69-6407-4b23-b09e-a1383ffead68 | in-use |        -         |  -   |  1   |      -      |   true   |    False    | d19e38b0-0025-45c7-bf3e-538389b77014 |
+--------------------------------------+--------+------------------+------+------+-------------+----------+-------------+--------------------------------------+
[root@dhcp163-21 ~(keystone_admin)]# openstack-service status nova-compute
MainPID=32665 Id=openstack-nova-compute.service ActiveState=active
[root@dhcp163-21 ~(keystone_admin)]# openstack-service stop nova-compute
[root@dhcp163-21 ~(keystone_admin)]# openstack-service status nova-compute
MainPID=0 Id=openstack-nova-compute.service ActiveState=inactive

[root@dhcp163-21 ~(keystone_admin)]# nova delete d19e38b0-0025-45c7-bf3e-538389b77014
Request to delete server d19e38b0-0025-45c7-bf3e-538389b77014 has been accepted.

[root@dhcp163-21 ~(keystone_admin)]# openstack-service start nova-compute
[root@dhcp163-21 ~(keystone_admin)]# openstack-service status nova-compute
MainPID=9419 Id=openstack-nova-compute.service ActiveState=active

[root@dhcp163-21 ~(keystone_admin)]# nova list
+----+------+--------+------------+-------------+----------+
| ID | Name | Status | Task State | Power State | Networks |
+----+------+--------+------------+-------------+----------+
+----+------+--------+------------+-------------+----------+

[root@dhcp163-21 ~(keystone_admin)]# cinder list 
+--------------------------------------+--------+------------------+------+------+-------------+----------+-------------+--------------------------------------+
|                  ID                  | Status | Migration Status | Name | Size | Volume Type | Bootable | Multiattach |             Attached to              |
+--------------------------------------+--------+------------------+------+------+-------------+----------+-------------+--------------------------------------+
| 329eff69-6407-4b23-b09e-a1383ffead68 | in-use |        -         |  -   |  1   |      -      |   true   |    False    | d19e38b0-0025-45c7-bf3e-538389b77014 |
+--------------------------------------+--------+------------------+------+------+-------------+----------+-------------+--------------------------------------+

[root@dhcp163-21 ~(keystone_admin)]# cinder reset-state --state available --attach-status detached 329eff69-6407-4b23-b09e-a1383ffead68

[root@dhcp163-21 ~(keystone_admin)]# cinder list
+--------------------------------------+-----------+------------------+------+------+-------------+----------+-------------+-------------+
|                  ID                  |   Status  | Migration Status | Name | Size | Volume Type | Bootable | Multiattach | Attached to |
+--------------------------------------+-----------+------------------+------+------+-------------+----------+-------------+-------------+
| 329eff69-6407-4b23-b09e-a1383ffead68 | available |        -         |  -   |  1   |      -      |   true   |    False    |             |
+--------------------------------------+-----------+------------------+------+------+-------------+----------+-------------+-------------+

[root@dhcp163-21 ~(keystone_admin)]# nova boot --boot-volume 329eff69-6407-4b23-b09e-a1383ffead68 --flavor 1 --nic net-id=ac477bb1-2a0c-4e51-b011-b177c63420d4 vm2
+--------------------------------------+--------------------------------------------------+
| Property                             | Value                                            |
+--------------------------------------+--------------------------------------------------+
| OS-DCF:diskConfig                    | MANUAL                                           |
| OS-EXT-AZ:availability_zone          |                                                  |
| OS-EXT-SRV-ATTR:host                 | -                                                |
| OS-EXT-SRV-ATTR:hypervisor_hostname  | -                                                |
| OS-EXT-SRV-ATTR:instance_name        | instance-00000002                                |
| OS-EXT-STS:power_state               | 0                                                |
| OS-EXT-STS:task_state                | scheduling                                       |
| OS-EXT-STS:vm_state                  | building                                         |
| OS-SRV-USG:launched_at               | -                                                |
| OS-SRV-USG:terminated_at             | -                                                |
| accessIPv4                           |                                                  |
| accessIPv6                           |                                                  |
| adminPass                            | E3DLTgTt2zLt                                     |
| config_drive                         |                                                  |
| created                              | 2016-10-31T14:28:09Z                             |
| flavor                               | m1.tiny (1)                                      |
| hostId                               |                                                  |
| id                                   | 8a8a3573-7fc1-4d59-af67-470352178741             |
| image                                | Attempt to boot from volume - no image supplied  |
| key_name                             | -                                                |
| metadata                             | {}                                               |
| name                                 | vm2                                              |
| os-extended-volumes:volumes_attached | [{"id": "329eff69-6407-4b23-b09e-a1383ffead68"}] |
| progress                             | 0                                                |
| security_groups                      | default                                          |
| status                               | BUILD                                            |
| tenant_id                            | 8aa0b26a0237412097a3f58e9d3c79ff                 |
| updated                              | 2016-10-31T14:28:10Z                             |
| user_id                              | 5fc876721fe3436a9043f0fbb79ec3c5                 |
+--------------------------------------+--------------------------------------------------+

[root@dhcp163-21 ~(keystone_admin)]# nova list
+--------------------------------------+------+--------+------------+-------------+------------------+
| ID                                   | Name | Status | Task State | Power State | Networks         |
+--------------------------------------+------+--------+------------+-------------+------------------+
| 8a8a3573-7fc1-4d59-af67-470352178741 | vm2  | ACTIVE | -          | Running     | private=10.0.0.4 |
+--------------------------------------+------+--------+------------+-------------+------------------+

[root@dhcp163-21 ~(keystone_admin)]# cinder list
+--------------------------------------+--------+------------------+------+------+-------------+----------+-------------+--------------------------------------+
|                  ID                  | Status | Migration Status | Name | Size | Volume Type | Bootable | Multiattach |             Attached to              |
+--------------------------------------+--------+------------------+------+------+-------------+----------+-------------+--------------------------------------+
| 329eff69-6407-4b23-b09e-a1383ffead68 | in-use |        -         |  -   |  1   |      -      |   true   |    False    | 8a8a3573-7fc1-4d59-af67-470352178741 |
+--------------------------------------+--------+------------------+------+------+-------------+----------+-------------+--------------------------------------+

Comment 12 Lon Hohberger 2016-11-03 18:14:13 UTC
According to our records, this should be resolved by openstack-cinder-7.0.2-4.el7ost.  This build is available now.

Comment 13 lkuchlan 2016-11-06 06:59:52 UTC
According to Comment 11


Note You need to log in before you can comment on or make changes to this bug.