Red Hat Bugzilla – Bug 921387
libvirtd crashes when deleting logical storage pool
Last modified: 2013-11-21 03:50:34 EST
Created attachment 709876 [details] Output from when it crashes when trying to remove storage pool. Description of problem: libvirt lvm storage pool usage seems to be broken. virt-install can't find the lvm path and libvirtd also seems to crash when you try to undefine it. Version-Release number of selected component (if applicable): 0.10.2-18.el6.x86_64 How reproducible: 95% Steps to Reproduce: (we have an auto-install that uses /home for a majority of space) Setup: 1. cd /tmp 2. yes | sudo lvreduce --size $SIZE --resizefs --force /dev/sys/home 3. sudo lvcreate sys -n vg001 -l 100%FREE 4. sudo vgcreate vg001 /dev/sys/vg001; 5. sudo service libvirtd start 6. sudo /usr/bin/virsh pool-define-as --name vg001 --type logical --target /dev/vg001 7. sudo /usr/bin/virsh pool-start vg001 8. sudo /usr/bin/virsh pool-autostart vg001 9. sudo /usr/bin/virt-install --name testhost --ram 4096 --disk pool=vg001,size=60 --boot network,hd,menu=on --graphics none --network bridge=br0,mac=52:54:00:34:51:4A --vcpus 4 --os-variant=rhel6 Remove pool: 1. sudo /usr/bin/virsh pool-destroy vg001 2. sudo /usr/bin/virsh pool-delete vg001 3 <this is where libvirtd seems to crash> Actual results: Attempt to create virtual machine (virt-install): 2013-03-13 22:22:30.017+0000: 6988: error : virCommandWait:2345 : internal error Child process (/sbin/lvcreate --name testhost.img -L 0K --virtualsize 62914560K vg001) unexpected exit status 5: Unable to create new logical volume with no extents 2013-03-13 22:22:30.039+0000: 6988: error : virCommandWait:2345 : internal error Child process (/sbin/lvremove -f vg001/testhost.img) unexpected exit status 5: One or more specified logical volume(s) not found. 2013-03-13 22:22:30.058+0000: 6988: error : virCommandWait:2345 : internal error Child process (/sbin/lvchange -aln vg001/testhost.img) unexpected exit status 5: One or more specified logical volume(s) not found. -- Deleting Volume pool: <see attached debug log file> Expected results: It should just work as it has with previous versions. Additional info: This maybe similar to: https://bugzilla.redhat.com/show_bug.cgi?id=872162 However, if you follow command line from: https://access.redhat.com/knowledge/docs/en-US/Red_Hat_Enterprise_Linux/6/html/Virtualization_Administration_Guide/chap-Virtualization_Administration_Guide-Storage_Pools-Storage_Pools.html#sect-Virtualization-Storage_Pools-Creating-LVM this is still the result. Had to downgrade back to 0.9.10 to make this work correctly Also dies if defining pool this way: sudo virsh pool-define-as vg001 logical - - /dev/mapper/sys-vg001 vg001 /dev/vg001 sudo virsh pool-start vg001 (didn't figure it'd make a difference, but tried anyway) Here's notes of what was seen. Starting install... ERROR internal error Child process (/sbin/lvchange -aln vg001/testhost.img) unexpected exit status 5: One or more specified logical volume(s) not found. XML dump: <pool type='logical'> <name>vg001</name> <uuid>820d51e9-42ea-cda8-ba3b-d5b08ee29082</uuid> <capacity unit='bytes'>512460062720</capacity> <allocation unit='bytes'>0</allocation> <available unit='bytes'>512460062720</available> <source> <device path='/dev/mapper/sys-vg001'/> <name>vg001</name> <format type='lvm2'/> </source> <target> <path>/dev/vg001</path> <permissions> <mode>0755</mode> <owner>-1</owner> <group>-1</group> </permissions> </target> </pool>
There are two issues here. First one is that virt-install tries to create a sparse logical volume (which is not a good idea since it won't work out of the box) and I'll clone this bug to virt-install to solve it. The other issue is a libvirt bug causing libvirtd to crash when you try to delete the pool.
Hello, step 4 doesn't make sense for me. Are you trying to create a VG on an LV and missed 'pvcreate /dev/sys/vg001' command between steps 3 and 4? Thanks for the clarification.
With OS install, we have it automated to allocate X amount to each slice (/, /var, /home, etc...) on install. /home is done last and the remaineder is done then. The script I have, resizes /home down to $SIZE (50 gigs) and leaves the rest for allocation to a LVM slice for VM Guest usage. No pvcreate is done since /home was already part of a LVM and space was just given back and can be used for whatever from what I understand. (another slice, snapshots, etc). This procedure does work with 0.9.* and below. If you have an alternate suggestion I would be happy to try it.
Could you try this with libvirt-0.10.2-18.el6_4.2 ? Libvirt shouldn't fail, however, the volume will be only one extent in size. If this is true, the fix is on its way. But if it still fails, there is another problem. Thanks for checking that.
Created attachment 915683 [details] Comment (This comment was longer than 65,535 characters and has been moved to an attachment by Red Hat Bugzilla).
Moving to POST: commit 2d73f2120f2073bff9e311626f6b6febcfc1e290 Author: Martin Kletzander <mkletzan@redhat.com> Date: Thu Mar 28 16:36:50 2013 +0100 storage: Avoid double virCommandFree in virStorageBackendLogicalDeletePool
According to comment1, try to verify the issue: "libvirt bug causing libvirtd to crash when you try to delete the pool." For the issue: "virt-install tries to create a sparse logical volume (which is not a good idea since it won't work out of the box)", will be verified in the cloned bug ? Reproduce with libvirt-0.10.2-18.el6.x86_64. Verified with the packages: libvirt-0.10.2-19.el6.x86_64 Test steps: 1. cd /tmp 2. #yes| sudo lvreduce --size 30G --resizefs --force /dev/mapper/vg_intelq8200novt86-lv_home 3. #lvcreate vg_intelq8200novt86 -n vg001 -l 100%FREE Logical volume "vg001" created 4. #vgcreate vg001 /dev/vg_intelq8200novt86/vg001 Volume group "vg001" successfully created 5. #service libvirtd restart 6. #virsh pool-define-as --name vg001 --type logical --target /dev/vg001 Pool vg001 defined 7. #virsh pool-start vg001 Pool vg001 started 8. #virsh pool-autostart vg001 Pool vg001 marked as autostarted # virsh pool-list --all Name State Autostart ----------------------------------------- default active yes vg001 active yes 9. #virt-install --name testhost --ram 1024 --disk pool=vg001,size=10 --boot network,hd,menu=on --graphics none --network network=default,mac=52:54:00:15:15:4A --vcpus 2 --os-variant=rhel6 --location http://rhel6-url ---------------------------------------------------------------------- Starting install... Retrieving file .treeinfo... | 3.5 kB 00:00 ... Retrieving file file vmlinuz... | 7.7 MB 00:00 ... Retrieving file initrd.img... | 60 MB 00:00 ... Allocating 'testhost1.img' | 10 GB 00:00 Creating domain... | 0 B 00:00 Connected to domain testhost1 Escape character is ^] Didn't met error below: --------------------------------------------------------------------- internal error Child process (/sbin/lvchange -aln vg001/testhost.img) unexpected exit status 5: One or more specified logical volume(s) not found. -------------------------------------------------------------------- Then stop the installation programe. Remove pool: 1. # virsh pool-destroy vg001 Pool vg001 destroyed 2. # virsh pool-delete vg001 Pool vg001 deleted 3. # service libvirtd status libvirtd (pid 9430) is running... 4. check the log:/var/log/libvirt/libvirtd.log, no libvirtd segfaults. 5. # virsh pool-undefine vg001 Pool vg001 has been undefined Test results: After running pool-delete, libvirtd is still running, without crash. Command work well.
Yes, the virt-install part will be verified in the virt-install bug.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHBA-2013-1581.html