RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 921387 - libvirtd crashes when deleting logical storage pool
Summary: libvirtd crashes when deleting logical storage pool
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: libvirt
Version: 6.4
Hardware: x86_64
OS: Unspecified
unspecified
high
Target Milestone: rc
: ---
Assignee: Martin Kletzander
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks: 921480
TreeView+ depends on / blocked
 
Reported: 2013-03-14 05:44 UTC by Brian Kruger
Modified: 2013-11-21 08:50 UTC (History)
10 users (show)

Fixed In Version: libvirt-0.10.2-19.el6
Doc Type: Bug Fix
Doc Text:
Cause: There was invalid use-after-free in logical storage backed. Consequence: Sometimes, when deleting logical storage pool, the daemon crashed. Fix: The pointer was cleared after free'd. Result: Deleting logical volumes works correctly.
Clone Of:
: 921480 (view as bug list)
Environment:
Last Closed: 2013-11-21 08:50:34 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)
Output from when it crashes when trying to remove storage pool. (64.06 KB, text/plain)
2013-03-14 05:44 UTC, Brian Kruger 		no flags Details
Comment (129.27 KB, text/plain)
2013-03-26 18:54 UTC, Brian Kruger 		no flags Details
View All


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2013:1581 0 normal SHIPPED_LIVE libvirt bug fix and enhancement update 2013-11-21 01:11:35 UTC

Description Brian Kruger 2013-03-14 05:44:16 UTC 
Created attachment 709876 [details]
Output from when it crashes when trying to remove storage pool.

Description of problem:

libvirt lvm storage pool usage seems to be broken.  virt-install can't find the lvm path and libvirtd also seems to crash when you try to undefine it.

Version-Release number of selected component (if applicable):

0.10.2-18.el6.x86_64

How reproducible:
95%

Steps to Reproduce:

 (we have an auto-install that uses /home for a majority of space)

Setup: 
1. cd /tmp
2. yes | sudo lvreduce --size $SIZE --resizefs --force /dev/sys/home
3. sudo lvcreate sys -n vg001 -l 100%FREE 
4. sudo vgcreate vg001 /dev/sys/vg001;
5. sudo service libvirtd start
6. sudo /usr/bin/virsh pool-define-as --name vg001 --type logical --target /dev/vg001
7. sudo /usr/bin/virsh pool-start vg001
8. sudo /usr/bin/virsh pool-autostart vg001
9. sudo /usr/bin/virt-install --name testhost --ram 4096 --disk pool=vg001,size=60 --boot network,hd,menu=on --graphics none --network bridge=br0,mac=52:54:00:34:51:4A   --vcpus 4 --os-variant=rhel6

Remove pool: 
1. sudo /usr/bin/virsh pool-destroy vg001
2. sudo /usr/bin/virsh pool-delete vg001
3 <this is where libvirtd seems to crash>
  
Actual results:

Attempt to create virtual machine (virt-install):

2013-03-13 22:22:30.017+0000: 6988: error : virCommandWait:2345 : internal error Child process (/sbin/lvcreate --name testhost.img -L 0K --virtualsize 62914560K vg001) unexpected exit status 5:   Unable to create new logical volume with no extents

2013-03-13 22:22:30.039+0000: 6988: error : virCommandWait:2345 : internal error Child process (/sbin/lvremove -f vg001/testhost.img) unexpected exit status 5:   One or more specified logical volume(s) not found.

2013-03-13 22:22:30.058+0000: 6988: error : virCommandWait:2345 : internal error Child process (/sbin/lvchange -aln vg001/testhost.img) unexpected exit status 5:   One or more specified logical volume(s) not found.
 --

Deleting Volume pool:

<see attached debug log file>

Expected results:

It should just work as it has with previous versions.

Additional info:
This maybe similar to: https://bugzilla.redhat.com/show_bug.cgi?id=872162 
However, if you follow command line from:
https://access.redhat.com/knowledge/docs/en-US/Red_Hat_Enterprise_Linux/6/html/Virtualization_Administration_Guide/chap-Virtualization_Administration_Guide-Storage_Pools-Storage_Pools.html#sect-Virtualization-Storage_Pools-Creating-LVM

this is still the result.

Had to downgrade back to 0.9.10 to make this work correctly

Also dies if defining pool this way:
sudo virsh pool-define-as vg001 logical - - /dev/mapper/sys-vg001 vg001 /dev/vg001
sudo virsh pool-start vg001

(didn't figure it'd make a difference, but tried anyway)

Here's notes of what was seen.

Starting install...
ERROR    internal error Child process (/sbin/lvchange -aln vg001/testhost.img) unexpected exit status 5:   One or more specified logical volume(s) not found.

XML dump:

<pool type='logical'>
  <name>vg001</name>
  <uuid>820d51e9-42ea-cda8-ba3b-d5b08ee29082</uuid>
  <capacity unit='bytes'>512460062720</capacity>
  <allocation unit='bytes'>0</allocation>
  <available unit='bytes'>512460062720</available>
  <source>
    <device path='/dev/mapper/sys-vg001'/>
    <name>vg001</name>
    <format type='lvm2'/>
  </source>
  <target>
    <path>/dev/vg001</path>
    <permissions>
      <mode>0755</mode>
      <owner>-1</owner>
      <group>-1</group>
    </permissions>
  </target>
</pool>
Comment 1 Jiri Denemark 2013-03-14 09:52:55 UTC 
There are two issues here. First one is that virt-install tries to create a sparse logical volume (which is not a good idea since it won't work out of the box) and I'll clone this bug to virt-install to solve it. The other issue is a libvirt bug causing libvirtd to crash when you try to delete the pool.
Comment 2 Martin Kletzander 2013-03-22 15:32:04 UTC 
Hello,

step 4 doesn't make sense for me.  Are you trying to create a VG on an LV and missed 'pvcreate /dev/sys/vg001' command between steps 3 and 4?  Thanks for the clarification.
Comment 3 Brian Kruger 2013-03-23 01:32:55 UTC 

With OS install, we have it automated to allocate X amount to each slice (/, /var, /home, etc...) on install. /home is done last and the remaineder is done then.

The script I have, resizes /home down to $SIZE (50 gigs) and leaves the rest for allocation to a LVM slice for VM Guest usage.   No pvcreate is done since /home was already part of a LVM and space was just given back and can be used for whatever from what I understand. (another slice, snapshots, etc).

This procedure does work with 0.9.* and below.  If you have an alternate suggestion I would be happy to try it.
Comment 4 Martin Kletzander 2013-03-26 15:25:35 UTC 
Could you try this with libvirt-0.10.2-18.el6_4.2 ?  Libvirt shouldn't fail, however, the volume will be only one extent in size.  If this is true, the fix is on its way.  But if it still fails, there is another problem.  Thanks for checking that.
Comment 5 Brian Kruger 2013-03-26 18:54:00 UTC 
Created attachment 915683 [details]
Comment

(This comment was longer than 65,535 characters and has been moved to an attachment by Red Hat Bugzilla).
Comment 6 Martin Kletzander 2013-03-29 10:50:34 UTC 
Moving to POST:

commit 2d73f2120f2073bff9e311626f6b6febcfc1e290
Author: Martin Kletzander <mkletzan>
Date:   Thu Mar 28 16:36:50 2013 +0100

    storage: Avoid double virCommandFree in virStorageBackendLogicalDeletePool
Comment 15 chhu 2013-07-11 08:21:19 UTC 
According to comment1, try to verify the issue: 
"libvirt bug causing libvirtd to crash when you try to delete the pool."

For the issue: "virt-install tries to create a sparse logical volume (which is not a good idea since it won't work out of the box)", will be verified in the cloned bug ?

Reproduce with libvirt-0.10.2-18.el6.x86_64.

Verified with the packages:
libvirt-0.10.2-19.el6.x86_64

Test steps:
1. cd /tmp
2. #yes| sudo lvreduce --size 30G --resizefs --force /dev/mapper/vg_intelq8200novt86-lv_home

3. #lvcreate vg_intelq8200novt86 -n vg001 -l 100%FREE
  Logical volume "vg001" created

4. #vgcreate vg001 /dev/vg_intelq8200novt86/vg001 
  Volume group "vg001" successfully created

5. #service libvirtd restart

6. #virsh pool-define-as --name vg001 --type logical --target /dev/vg001
   Pool vg001 defined

7. #virsh pool-start vg001
   Pool vg001 started

8. #virsh pool-autostart vg001
   Pool vg001 marked as autostarted

   # virsh pool-list --all 
   Name                 State      Autostart 
   -----------------------------------------
   default              active     yes       
   vg001                active     yes

9. #virt-install --name testhost --ram 1024 --disk pool=vg001,size=10 --boot network,hd,menu=on --graphics none --network network=default,mac=52:54:00:15:15:4A --vcpus 2 --os-variant=rhel6 --location http://rhel6-url
----------------------------------------------------------------------
Starting install...
Retrieving file .treeinfo...     | 3.5 kB     00:00 ... 
Retrieving file file vmlinuz...  | 7.7 MB     00:00 ... 
Retrieving file initrd.img...    |  60 MB     00:00 ... 
Allocating 'testhost1.img'       |  10 GB     00:00     
Creating domain...               |    0 B     00:00     
Connected to domain testhost1
Escape character is ^]

Didn't met error below:
---------------------------------------------------------------------
internal error Child process (/sbin/lvchange -aln vg001/testhost.img) unexpected exit status 5:   One or more specified logical volume(s) not found.
--------------------------------------------------------------------
Then stop the installation programe.

Remove pool: 
1. # virsh pool-destroy vg001
   Pool vg001 destroyed

2. # virsh pool-delete vg001 
   Pool vg001 deleted

3. # service libvirtd status
libvirtd (pid  9430) is running...

4. check the log:/var/log/libvirt/libvirtd.log, no libvirtd segfaults.

5. # virsh pool-undefine vg001
Pool vg001 has been undefined

Test results:
After running pool-delete, libvirtd is still running, without crash. Command work well.
Comment 16 Martin Kletzander 2013-07-11 09:47:55 UTC 
Yes, the virt-install part will be verified in the virt-install bug.
Comment 18 errata-xmlrpc 2013-11-21 08:50:34 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-1581.html
Note You need to log in before you can comment on or make changes to this bug.