Previously, rebooting a host caused all its instances to stop and change to the SHUTDOWN power state. The unpause action was only allowed on instances with PAUSED power state.
As a consequence, it was impossible to unpause/start/reboot an instance that had been paused before the reboot.
As it makes no sense to unpause an instance once it has been stopped, a check has been added in a periodic task that will fix the state and stop instances that are in SHUTDOWN/CRASHED power state but are still in the PAUSED state.
As a result users can now start those instances again, preventing them from being stuck in an inconsistent state.
Created attachment 844520[details]
compute.log try to unpuase after reboot
Description of problem:
Unpauseing an instance fails if host has rebooted.
Version-Release number of selected component (if applicable):
RHEL: release 6.5 (Santiago)
RHOS: Havana
openstack-nova-api-2013.2.1-1.el6ost.noarch
openstack-nova-compute-2013.2.1-1.el6ost.noarch
openstack-nova-scheduler-2013.2.1-1.el6ost.noarch
openstack-nova-common-2013.2.1-1.el6ost.noarch
openstack-nova-console-2013.2.1-1.el6ost.noarch
openstack-nova-conductor-2013.2.1-1.el6ost.noarch
openstack-nova-novncproxy-2013.2.1-1.el6ost.noarch
openstack-nova-cert-2013.2.1-1.el6ost.noarch
How reproducible:
Every time
Steps to Reproduce:
1. Boot an instance
2. Pause that instance
3. Reboot host
4. Unpause instance
Actual results:
can't unpause instance stuck in status paused, power state - shutdown
Expected results:
Instance should unpause, return to running state
Additional info:
virsh list -all --managed-save
ID is missing from paused instance "-" (pausecirros), state -> shut off.
[root@orange-vdse ~(keystone_admin)]# virsh list --all --managed-save
Id Name State
----------------------------------------------------
1 instance-00000003 running
2 instance-00000002 running
- instance-00000001 shut off
[root@orange-vdse ~(keystone_admin)]# nova list (notice nova status paused)
+--------------------------------------+---------------+--------+------------+-------------+-----------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+---------------+--------+------------+-------------+-----------------+
| ebe310c2-d715-45e5-83b6-32717af1ac90 | cirros | ACTIVE | None | Running | net=192.168.1.4 |
| 3ef89feb-414f-4524-b806-f14044efdb14 | pausecirros | PAUSED | None | Shutdown | net=192.168.1.5 |
| 8bcae041-2f92-4ae2-a2c2-ee59b067ac76 | suspendcirros | ACTIVE | None | Running | net=192.168.1.2 |
+--------------------------------------+---------------+--------+------------+-------------+-----------------+
Testing without rebooting host, ID/state ("1"/paused) instance (cirros) are ok and it unpauses ok.
[root@orange-vdse ~(keystone_admin)]# virsh list --all --managed-save
Id Name State
----------------------------------------------------
1 instance-00000003 paused
2 instance-00000002 running
- instance-00000001 shut off
+--------------------------------------+---------------+--------+------------+-------------+-----------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+---------------+--------+------------+-------------+-----------------+
| ebe310c2-d715-45e5-83b6-32717af1ac90 | cirros | PAUSED | None | Paused | net=192.168.1.4 |
| 3ef89feb-414f-4524-b806-f14044efdb14 | pausecirros | PAUSED | None | Shutdown | net=192.168.1.5 |
| 8bcae041-2f92-4ae2-a2c2-ee59b067ac76 | suspendcirros | ACTIVE | None | Running | net=192.168.1.2 |
+--------------------------------------+---------------+--------+------------+-------------+-----------------+
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
http://rhn.redhat.com/errata/RHSA-2014-0231.html
Created attachment 844520 [details] compute.log try to unpuase after reboot Description of problem: Unpauseing an instance fails if host has rebooted. Version-Release number of selected component (if applicable): RHEL: release 6.5 (Santiago) RHOS: Havana openstack-nova-api-2013.2.1-1.el6ost.noarch openstack-nova-compute-2013.2.1-1.el6ost.noarch openstack-nova-scheduler-2013.2.1-1.el6ost.noarch openstack-nova-common-2013.2.1-1.el6ost.noarch openstack-nova-console-2013.2.1-1.el6ost.noarch openstack-nova-conductor-2013.2.1-1.el6ost.noarch openstack-nova-novncproxy-2013.2.1-1.el6ost.noarch openstack-nova-cert-2013.2.1-1.el6ost.noarch How reproducible: Every time Steps to Reproduce: 1. Boot an instance 2. Pause that instance 3. Reboot host 4. Unpause instance Actual results: can't unpause instance stuck in status paused, power state - shutdown Expected results: Instance should unpause, return to running state Additional info: virsh list -all --managed-save ID is missing from paused instance "-" (pausecirros), state -> shut off. [root@orange-vdse ~(keystone_admin)]# virsh list --all --managed-save Id Name State ---------------------------------------------------- 1 instance-00000003 running 2 instance-00000002 running - instance-00000001 shut off [root@orange-vdse ~(keystone_admin)]# nova list (notice nova status paused) +--------------------------------------+---------------+--------+------------+-------------+-----------------+ | ID | Name | Status | Task State | Power State | Networks | +--------------------------------------+---------------+--------+------------+-------------+-----------------+ | ebe310c2-d715-45e5-83b6-32717af1ac90 | cirros | ACTIVE | None | Running | net=192.168.1.4 | | 3ef89feb-414f-4524-b806-f14044efdb14 | pausecirros | PAUSED | None | Shutdown | net=192.168.1.5 | | 8bcae041-2f92-4ae2-a2c2-ee59b067ac76 | suspendcirros | ACTIVE | None | Running | net=192.168.1.2 | +--------------------------------------+---------------+--------+------------+-------------+-----------------+ Testing without rebooting host, ID/state ("1"/paused) instance (cirros) are ok and it unpauses ok. [root@orange-vdse ~(keystone_admin)]# virsh list --all --managed-save Id Name State ---------------------------------------------------- 1 instance-00000003 paused 2 instance-00000002 running - instance-00000001 shut off +--------------------------------------+---------------+--------+------------+-------------+-----------------+ | ID | Name | Status | Task State | Power State | Networks | +--------------------------------------+---------------+--------+------------+-------------+-----------------+ | ebe310c2-d715-45e5-83b6-32717af1ac90 | cirros | PAUSED | None | Paused | net=192.168.1.4 | | 3ef89feb-414f-4524-b806-f14044efdb14 | pausecirros | PAUSED | None | Shutdown | net=192.168.1.5 | | 8bcae041-2f92-4ae2-a2c2-ee59b067ac76 | suspendcirros | ACTIVE | None | Running | net=192.168.1.2 | +--------------------------------------+---------------+--------+------------+-------------+-----------------+