If someone attempts to boot an instance in Icehouse after an upgrade from Havana and they have not yet configured /etc/neutron/neutron.conf with the appropriate keystone credentials for sending notifications to Nova, the instance will fail to spawn. Additionally, attempts to delete the instance will result in: 2014-05-03 00:50:45.757 7433 ERROR oslo.messaging.rpc.dispatcher [-] Exception during message handling: Error during following call to agent: ['ovs-vsctl', '--timeout=120', 'del-port', 'br-int', u'qvob7f77a1e -c5'] An administrator can work around this problem by running: nova reset-state <instance> After this, the delete will proceed successfully.
Hi Lars, I think the appropriate approach is to: a) Document the workaround in the upgrade notes. b) File an upstream bug to fix the fact that you can't delete it. Thoughts?
[Ping. Bug triaging here, any updates here? No comments since about 9 months. Is this still reproducible with during an IceHouse -> Juno upgrade?.]
This bug still exists in Juno. That is, if you do not have Neutron configured appropriately and you start an instance, it will end up in the ERROR state. If you try to delete that instance, you will see in the Nova logs: 2015-02-20 10:34:44.672 2214 AUDIT nova.compute.manager [req-a4b30d0b-e6d3-429f-8f7a-b7788b79c86c None] [instance: 1541a197-9f80-4ee5-a7d6-08e591aa83fd] Terminating instance 2015-02-20 10:34:44.680 2214 WARNING nova.virt.libvirt.driver [-] [instance: 1541a197-9f80-4ee5-a7d6-08e591aa83fd] During wait destroy, instance disappeared. 2015-02-20 10:34:44.904 2214 INFO nova.virt.libvirt.driver [req-a4b30d0b-e6d3-429f-8f7a-b7788b79c86c None] [instance: 1541a197-9f80-4ee5-a7d6-08e591aa83fd] Deletion of /var/lib/nova/instances/1541a197-9f80-4ee5-a7d6-08e591aa83fd_del complete 2015-02-20 10:34:45.079 2214 INFO nova.compute.manager [req-a4b30d0b-e6d3-429f-8f7a-b7788b79c86c None] [instance: 1541a197-9f80-4ee5-a7d6-08e591aa83fd] Instance disappeared during terminate And "nova list" will show: | 1541a197-9f80-4ee5-a7d6-08e591aa83fd | test0 | ERROR | deleting | NOSTATE | | It looks like running "nova reset-state <instance>" does *not* work around this problem, and at this point even getting Neutron configured correctly will not permit you to delete the instance. The only way I've found to recover from this is this SQL script: http://blog.oddbit.com/assets/2015/01/17/delete-deleting-instances.sql And that's not pretty...