Hide Forgot
Description of problem: I noticed this failure in the beakerd logs with some of my recipes which ran on OpenStack. Version-Release number of selected component (if applicable): unreleased develop ade77d7 How reproducible: not sure Actual results: bkr.server.model.scheduler DEBUG Releasing vm bfc35fa6-3a44-468e-b4e3-1bc3546cd835 for recipe 16747 bkr.server.model.scheduler ERROR Failed to destroy vm bfc35fa6-3a44-468e-b4e3-1bc3546cd835, leaked! Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/bkr/server/model/scheduler.py", line 4143, in release manager.destroy_vm(self) File "/usr/lib/python2.6/site-packages/bkr/server/dynamic_virt.py", line 143, in destroy_vm self.neutronclient.delete_network(str(vm.network_id)) [...] Conflict: Conflict (HTTP 409) (Request-ID: req-5527689c-6f31-475e-bb32-265137692352)
I noticed that the leftover network still had some ports attached -- not sure if that's expected, but it seems from the API docs that we actually need to find all remaining ports on the network and delete those, before we can delete the network. Based on that, my theory about steps to reproduce (not sure about this): 1. Submit a job which will run on OpenStack (x86_64, single host, etc) 2. Wait for the instance to be created 3. When the instance is booting into iPXE and trying to download the kernel and initrd, hit Ctrl-Alt-Delete in the Console tab of Horizon (note: in my case, iPXE was getting stuck downloading the kernel, that seems to be a separate issue but it probably helps to reproduce this) 4. The network will have a number of ports which say Attached Device is "network:dhcp", these will remain even after the instance itself is deleted.
(In reply to Dan Callaghan from comment #1) > I noticed that the leftover network still had some ports attached -- not > sure if that's expected, but it seems from the API docs that we actually > need to find all remaining ports on the network and delete those, before we > can delete the network. > Based on that, my theory about steps to reproduce (not sure about this): > 1. Submit a job which will run on OpenStack (x86_64, single host, etc) > 2. Wait for the instance to be created > 3. When the instance is booting into iPXE and trying to download the kernel > and initrd, hit Ctrl-Alt-Delete in the Console tab of Horizon > (note: in my case, iPXE was getting stuck downloading the kernel, that seems > to be a separate issue but it probably helps to reproduce this) > 4. The network will have a number of ports which say Attached Device is > "network:dhcp", these will remain even after the instance itself is deleted. There should be 5 ports created when an instance is built. One is for the instance, one is for the interface of the router and the other three ports for the private network. The ports in the leftover networks seem okay for me as I can see the port of the instance and the port of the interface have been deleted. It seems like the problem here is the port of the instance was not released yet when deleting the network, so I think we should wait until the instance is deleted and the port is released before deleting the network.
On Gerrit: https://gerrit.beaker-project.org/#/c/5391/
Beaker 24.0 has been released.