Sometimes, using 'virsh destroy' or the 'Force off' button in virt-manager will cause multiple running VMs to be destroyed. Here's an example: [wwoods@metroid ~]$ sudo virsh list --all Id Name State ---------------------------------- 1 Ubuntu_Jaunty running 4 Fedora_10_clone running 5 F10_2 running 6 F10 running [wwoods@metroid ~]$ sudo virsh destroy F10_2 Domain F10_2 destroyed [wwoods@metroid ~]$ sudo virsh list --all Id Name State ---------------------------------- 1 Ubuntu_Jaunty running 4 Fedora_10_clone running - F10 shut off - F10_2 shut off Note that 'F10' is now also shut off, even though I didn't destroy it. This doesn't seem to happen every time, and it doesn't seem to be related to the names of the hosts being similar: [wwoods@metroid ~]$ sudo virsh list --all Id Name State ---------------------------------- 1 Ubuntu_Jaunty running 4 Fedora_10_clone running 8 F10_2 running - F10 shut off [wwoods@metroid ~]$ sudo virsh start F10 Domain F10 started [wwoods@metroid ~]$ sudo virsh destroy Fedora_10_clone Domain Fedora_10_clone destroyed [wwoods@metroid ~]$ sudo virsh list --all Id Name State ---------------------------------- 1 Ubuntu_Jaunty running 10 F10 running - F10_2 shut off - Fedora_10_clone shut off There are no relevant messages in syslog, other than the expected ones (e.g. "kernel: virbr0: port 3(vnet2) entering disabled state" as the host comes down).
Can you run 'strace -f -p $PID-OF-LIBVIRTD -s 1000 -ff -o' and then try and reproduce the destroy problem. Also, can you turn on full debug logging of libvirtd & capture the results http://libvirt.org/logging.html
Nasty. I've asked wwoods for a libvirtd log ala: https://fedoraproject.org/wiki/Reporting_virtualization_bugs#libvirt
Created attachment 342908 [details] very verbose log from libvirtd This is the full log from libvirtd with log_level set to 1. It follows these basic steps: service libvirtd restart virsh list --all Id Name State ---------------------------------- 1 Ubuntu_Jaunty running 13 F10 running - F10_2 shut off - F10_RAID shut off - F9 shut off - Fedora_10_clone shut off - Rawhide shut off virsh start F10_2 virsh start Fedora_10_clone virsh list --all virsh destroy F10_2 virsh list --all virsh start F10_2 virsh destroy F10 virsh list --all virsh start F10 virsh list --all virsh list --all virsh start F10_2 virsh list --all # problem is triggered here - F10 dies as well virsh destroy F10_2 virsh list --all Id Name State ---------------------------------- 1 Ubuntu_Jaunty running 15 Fedora_10_clone running - F10 shut off - F10_2 shut off - F10_RAID shut off - F9 shut off - Rawhide shut off service libvirtd stop Hope you can make some sense of it - it's 4MB uncompressed.
Interesting, you only issued a destroy for F10_2, but yet: 14:12:43.927: debug : virDomainDestroy:1750 : domain=0x7fb270001270 14:12:43.927: debug : qemudShutdownVMDaemon:1518 : Shutting down VM 'F10_2' 14:12:43.927: debug : virEventRemoveHandleImpl:165 : Remove handle 20 14:12:43.927: debug : virEventRemoveHandleImpl:172 : mark delete 11 24 14:12:43.983: debug : virEventRunOnce:544 : Poll got 1 event 14:12:43.983: debug : virEventDispatchHandles:416 : Skip deleted 24 14:12:43.983: debug : virEventDispatchHandles:425 : Dispatch 24 32 0x13f7c00 14:12:44.036: info : Setting SELinux context on '/var/lib/libvirt/images/F10_2.img' to 'system_u:object_r:virt_image_t:s0' 14:12:44.036: debug : virEventUpdateTimeoutImpl:233 : Updating timer 0 timeout with 0 ms freq 14:12:44.036: debug : qemudShutdownVMDaemon:1518 : Shutting down VM 'F10'
This is an event loop dispatcher bug. The destroy call is killing the QEMU process, so we then get a HANGUP event on the FD associated with the guest monitor. The callback for this is already marked as deleted in the event loop though, so it gets skiped, and then we mistakenly dispatch the next callback in the loop, causing us to think another VM has died, and trigger cleanup of that guest
*** Bug 499788 has been marked as a duplicate of this bug. ***
Created attachment 343109 [details] Fix event loop handling of deletes & test functionality THis patch fixes the event loop handling of deletes and adds a test case which validates the various important scenarios actually work
*** Bug 500089 has been marked as a duplicate of this bug. ***
libvirt-0.6.2-9.fc11 has been submitted as an update for Fedora 11. http://admin.fedoraproject.org/updates/libvirt-0.6.2-9.fc11
I've pushed libvirt-0.6.2-9.fc11 to updates-testing with Dan's fix. Please test and update the update's karma using the link above * Thu May 21 2009 Mark McLoughlin <markmc> - 0.6.2-9.fc11 - Fix qemu argv detection with latest qemu (bug #501923) - Fix XML attribute escaping (bug #499791) - Fix serious event handling issues causing guests to be destroyed (bug #499698)
libvirt-0.6.2-9.fc11 has been pushed to the Fedora 11 testing repository. If problems still persist, please make note of it in this bug report. If you want to test the update, you can install it with su -c 'yum --enablerepo=updates-testing update libvirt'. You can provide feedback for this update here: http://admin.fedoraproject.org/updates/F11/FEDORA-2009-5311
libvirt-0.6.2-10.fc11 has been submitted as an update for Fedora 11. http://admin.fedoraproject.org/updates/libvirt-0.6.2-10.fc11
libvirt-0.6.2-10.fc11 has been pushed to the Fedora 11 testing repository. If problems still persist, please make note of it in this bug report. If you want to test the update, you can install it with su -c 'yum --enablerepo=updates-testing update libvirt'. You can provide feedback for this update here: http://admin.fedoraproject.org/updates/F11/FEDORA-2009-5441
libvirt-0.6.2-11.fc11 has been pushed to the Fedora 11 testing repository. If problems still persist, please make note of it in this bug report. If you want to test the update, you can install it with su -c 'yum --enablerepo=updates-testing update libvirt'. You can provide feedback for this update here: http://admin.fedoraproject.org/updates/F11/FEDORA-2009-5515
libvirt-0.6.2-11.fc11 has been pushed to the Fedora 11 stable repository. If problems still persist, please make note of it in this bug report.
libvirt-0.6.2-12.fc11 has been submitted as an update for Fedora 11. http://admin.fedoraproject.org/updates/libvirt-0.6.2-12.fc11