RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 702260 - Libvirt can't remove logical volume because it doesn't deactivate it first
Summary: Libvirt can't remove logical volume because it doesn't deactivate it first
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: libvirt
Version: 6.2
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: rc
: ---
Assignee: Osier Yang
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On: 570359
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-05-05 08:11 UTC by David Jaša
Modified: 2012-06-20 06:27 UTC (History)
8 users (show)

Fixed In Version: libvirt-0.9.9-1.el6
Doc Type: Bug Fix
Doc Text:
No documentaion needed.
Clone Of:
Environment:
Last Closed: 2012-06-20 06:27:59 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2012:0748 0 normal SHIPPED_LIVE Low: libvirt security, bug fix, and enhancement update 2012-06-19 19:31:38 UTC

Description David Jaša 2011-05-05 08:11:03 UTC
Description of problem:
SSIA
related to bug #591390

Version-Release number of selected component (if applicable):
libvirt-0.8.7-18.el6.x86_64
libvirt-python-0.8.7-18.el6.x86_64
libvirt-client-0.8.7-18.el6.x86_64

How reproducible:
always

Steps to Reproduce:
1. create a new logical volume using virsh
2. attempt to delete logical volume created in previous step
3.
  
Actual results:
Deletion fails with this error:
error:Failed to delete vol lv-win2k8
error:internal error '/sbin/lvremove -f /path/to/lv' exited with non-zero status 5 and signal 0:   Can't remove open logical volume "lv"

Expected results:
LV gets successfully deleted

Additional info: 
'lvremove -f /path/to/lv' also fails when invoked from root shell, but when one runs 'lvchange -an /path/to/lv' and then 'lvremove /path/to/lv', the LV gets successfully removed. Full CLI output:

---------- 8< ------------------------

[root@dhcp-29-7 ~]# virsh
Vítejte na virsh, interaktivním virtualizačním terminálu.

Napište: 'help' pro nápovědu k příkazům
       'quit' pro skončení

virsh # vol-list root_vg
Jméno               Path                                    
-----------------------------------------
home                 /dev/root_vg/home                       
q-winxp              /dev/root_vg/q-winxp                    
rawhide              /dev/root_vg/rawhide                    
rhel6                /dev/root_vg/rhel6                      
swap                 /dev/root_vg/swap                       

virsh # vol-create-as root_vg lv-win2k8 25G --allocation 0 --format raw
Vol lv-win2k8 created

virsh # vol-list root_vg
Jméno               Path                                    
-----------------------------------------
home                 /dev/root_vg/home                       
lv-win2k8            /dev/root_vg/lv-win2k8                  
q-winxp              /dev/root_vg/q-winxp                    
rawhide              /dev/root_vg/rawhide                    
rhel6                /dev/root_vg/rhel6                      
swap                 /dev/root_vg/swap                       

virsh # vol-delete --pool root_vg lv-win2k8
chyba:Failed to delete vol lv-win2k8
chyba:vnitřní chyba '/sbin/lvremove -f /dev/root_vg/lv-win2k8' exited with non-zero status 5 and signal 0:   Can't remove open logical volume "lv-win2k8"


virsh # 
[root@dhcp-29-7 ~]# lvremove -f /dev/root_vg/lv-win2k8 
  Can't remove open logical volume "lv-win2k8"
[root@dhcp-29-7 ~]# lvchange -an /dev/root_vg/lv-win2k8 
[root@dhcp-29-7 ~]# lvremove /dev/root_vg/lv-win2k8 
  Logical volume "lv-win2k8" successfully removed

Comment 1 David Jaša 2011-05-05 08:35:52 UTC
One more CLI example, this time LV name is q-rhel in the same 



[root@dhcp-29-7 ~]# virsh vol-delete --pool root_vg q-rhel
chyba:Failed to delete vol q-rhel
chyba:vnitřní chyba '/sbin/lvremove -f /dev/root_vg/q-rhel' exited with non-zero status 5 and signal 0:   Can't remove open logical volume "q-rhel"


[root@dhcp-29-7 ~]# lvchange -an /dev/root_vg/q-rhel 
[root@dhcp-29-7 ~]# virsh vol-delete --pool root_vg q-rhel
Vol q-rhel deleted

Comment 2 Daniel Berrangé 2011-05-05 09:06:02 UTC
This looks like a  bug / race condition in the lvremove command to me. It is non-deterministic when I run it 

[root@lettuce ~]# virsh vol-delete --pool HostVG foo
error: Failed to delete vol foo
error: internal error '/sbin/lvremove -f /dev/HostVG/foo' exited with non-zero status 5 and signal 0:   LV HostVG/foo in use: not deactivating
  Unable to deactivate logical volume "foo"


[root@lettuce ~]# virsh vol-delete --pool HostVG foo
error: Failed to delete vol foo
error: internal error '/sbin/lvremove -f /dev/HostVG/foo' exited with non-zero status 5 and signal 0:   LV HostVG/foo in use: not deactivating
  Unable to deactivate logical volume "foo"


[root@lettuce ~]# virsh vol-delete --pool HostVG foo
Vol foo deleted

libvirt explicitly uses the '-f' argument to ensure active volumes are removed

 "  -f, --force
      Remove active logical volumes without confirmation."

Comment 3 David Jaša 2011-05-05 10:04:44 UTC
(In reply to comment #2)
> This looks like a  bug / race condition in the lvremove command to me. It is
> non-deterministic when I run it 
> 

It seems so, but doing a 'lvchange -an $LV' seems to be pretty viable workaround (numbers printed are numbers of retries before LV got removed), -f option isn't needed at all:


[root@dhcp-29-7 ~]# for I in {1..20};do virsh vol-create-as root_vg lv-example 25G --allocation 0 --format raw >/dev/null ; i=0; until lvremove -f /dev/root_vg/lv-example >/dev/null 2>&1; do i=$(($i+1)); done ; echo $i ; virsh pool-refresh root_vg >/dev/null;done
19
9
4
1
2
4
0
10
7
19
2
1
13
0
0
1
6
14
2
3
[root@dhcp-29-7 ~]# for I in {1..20};do virsh vol-create-as root_vg lv-example 25G --allocation 0 --format raw >/dev/null ; i=0; until { lvchange -an /dev/root_vg/lv-example; lvremove /dev/root_vg/lv-example >/dev/null 2>&1; } ; do i=$(($i+1)); done ; echo $i ; virsh pool-refresh root_vg >/dev/null;done
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0

Comment 4 Osier Yang 2011-06-08 06:28:25 UTC
This is race of lvremove, see BZ https://bugzilla.redhat.com/show_bug.cgi?id=570359, patch was applied in lvm upstream, https://www.redhat.com/archives/lvm-devel/2011-May/msg00025.html, but the patch can't fix the problem drastically, it just can minimize the possiability to reproduce the problem.

Comment 5 Osier Yang 2011-06-08 07:03:50 UTC
Both lvremove and "lvchange -an" invoke function "deactivate_lv(cmd, lv))" to deactive the lv, so using "lvchange -an" won't work for problem as <a href="https://bugzilla.redhat.com/show_bug.cgi?id=702260#c2">#c2</a>.

Using "lvremove" will work for David's case though, as lvremove do checking before deactive the lv to ensure the lv is not referred to by another existing LVs.

<quote>

        if (lv_info(cmd, lv, 0, &info, 1, 0)) {
                if (info.open_count) {
                        log_error("Can't remove open logical volume \"%s\"",
                                  lv->name);
                        return 0;
                }

                if (lv_is_active(lv) && (force == PROMPT) &&
                    lv_is_visible(lv) &&
                    yes_no_prompt("Do you really want to remove active "
                                  "%slogical volume %s? [y/n]: ",
                                  vg_is_clustered(vg) ? "clustered " : "",
                                  lv->name) == 'n') {
                        log_error("Logical volume %s not removed", lv->name);
                        return 0;
                }
        }
        
        /* Skipped codes here */

        if (!deactivate_lv(cmd, lv)) {
                log_error("Unable to deactivate logical volume \"%s\"",
                          lv->name);
                return 0;
        }
</quote>

I'm wondering if libvirt should use "lvchange -an" before "lvremove", as it at least could fix David's problem.

Comment 7 Osier Yang 2011-09-21 10:14:29 UTC
Patch posted to upstream

http://www.redhat.com/archives/libvir-list/2011-September/msg00802.html

Comment 8 Osier Yang 2011-09-22 00:21:37 UTC
patch committed to upstream.

Comment 12 Huang Wenlong 2012-01-10 03:03:27 UTC
Verify this bug with:
libvirt-0.9.9-1.el6.x86_64


1) add vg in the libvirt pool 

# virsh pool-dumpxml vg_inteli7124
<pool type='logical'>
  <name>vg_inteli7124</name>
  <uuid>794ac3ee-3a62-d96b-db6d-4c4857269cbb</uuid>
  <capacity>499579355136</capacity>
  <allocation>284264759296</allocation>
  <available>215314595840</available>
  <source>
    <device path='/dev/sda2'/>
    <name>vg_inteli7124</name>
    <format type='lvm2'/>
  </source>
  <target>
    <path>/dev/vg_inteli7124</path>
    <permissions>
      <mode>0700</mode>
      <owner>-1</owner>
      <group>-1</group>
    </permissions>
  </target>
</pool>

2) add volume 
# virsh vol-create-as --pool vg_inteli7124 --name lv_test --capacity 100M
Vol lv_test created

3)  check vol list 
# virsh vol-list  vg_inteli7124
Name                 Path                                    
-----------------------------------------
lv_home              /dev/vg_inteli7124/lv_home              
lv_root              /dev/vg_inteli7124/lv_root              
lv_swap              /dev/vg_inteli7124/lv_swap              
lv_test              /dev/vg_inteli7124/lv_test

4)  remove the volume 
#virsh vol-delete --pool vg_inteli7124 lv_test
Vol lv_test deleted

#virsh vol-list vg_inteli7124
Name                 Path                                    
-----------------------------------------
lv_home              /dev/vg_inteli7124/lv_home              
lv_root              /dev/vg_inteli7124/lv_root              
lv_swap              /dev/vg_inteli7124/lv_swap

Comment 13 Osier Yang 2012-05-04 10:05:10 UTC
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
No documentaion needed.

Comment 15 errata-xmlrpc 2012-06-20 06:27:59 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2012-0748.html


Note You need to log in before you can comment on or make changes to this bug.