Bug 1166592

Summary: Failed to create logical volume with specified xml
Product: Red Hat Enterprise Linux 7 Reporter: Yang Yang <yanyang>
Component: libvirtAssignee: John Ferlan <jferlan>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.1CC: dyuan, jferlan, pzhang, rbalakri, shyu, xuzhang, yanyang
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-1.2.8-11.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-03-05 07:47:38 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1091866    

Description Yang Yang 2014-11-21 10:14:21 UTC
Description of problem:
Failed to create logical volume with specified xml

Version-Release number of selected component (if applicable):
libvirt-1.2.8-7.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Prepare a logical pool
# cat logical-pool.xml
<pool type='logical'>
       <name>logical-pool</name>
       <source>
         <name>myvg</name>
         <format type='lvm2'/>
         <device path='/dev/sdc'/>
       </source>
       <target>
         <path>/dev/myvg</path>
       </target>
     </pool>

# virsh pool-define logical-pool.xml
Pool logical-pool defined from logical-pool.xml

# virsh pool-build logical-pool
Pool logical-pool built

# virsh pool-start logical-pool
Pool logical-pool started

2. create a volume with specified xml
# cat /tmp/xml_utils_temp_cgWzlE.xml
<volume>
<name>vol_create_test</name>
<allocation>1048576</allocation>
<capacity>1073741824</capacity>
</volume>

# virsh vol-create logical-pool /tmp/xml_utils_temp_cgWzlE.xml
error: Failed to create vol from /tmp/xml_utils_temp_cgWzlE.xml
error: key in virGetStorageVol must not be NULL

3. try to create volume not specifying xml
# virsh vol-create-as logical-pool vol_create_test 100M
error: Failed to create vol vol_create_test
error: internal error: Child process (/usr/sbin/lvcreate --name vol_create_test -L 102400K myvg) unexpected exit status 5:   Logical volume "vol_create_test" already exists in volume group "myvg"

4. try step 3 again
# virsh vol-create-as logical-pool vol_create_test 100M
Vol vol_create_test created

Actual results:
failed to create logical volume with specified xml

Expected results:
fix it

Additional info:

Comment 2 Pei Zhang 2014-11-28 09:13:52 UTC
The same issue was hit by virsh command vol-create-as with --allocation option.

1.prepare a logical pool

# virsh pool-list logical
 Name                 State      Autostart 
-------------------------------------------
 logical-pool         active     no        

# virsh vol-list logical-pool
 Name                 Path                                    
------------------------------------------------------------------------------
2. create volume in logical pool

# virsh vol-create-as logical-pool vol1 1G
Vol vol1 created

3.create volume with --allocation option , fail to create 

# virsh vol-create-as logical-pool vol2 1G --allocation 521M
error: Failed to create vol vol2
error: key in virGetStorageVol must not be NULL

# virsh vol-create-as logical-pool vol3 1G --allocation 0
error: Failed to create vol vol3
error: key in virGetStorageVol must not be NULL

4.check the volume 
# virsh vol-list logical-pool
 Name                 Path                                    
------------------------------------------------------------------------------
 vol1                 /dev/vg0/vol1                           

# ls -al /dev/vg0/
......
lrwxrwxrwx.  1 root root    7 Nov 28 16:52 vol1 -> ../dm-0
lrwxrwxrwx.  1 root root    7 Nov 28 17:02 vol2 -> ../dm-9    <===== failed to create volume

5. refresh the logical pool  
# virsh pool-refresh logical-pool
Pool logical-pool refreshed

# virsh vol-list logical-pool
 Name                 Path                                    
------------------------------------------------------------------------------
 vol1                 /dev/vg0/vol1                           

# ls /dev/vg0 -al
......
lrwxrwxrwx.  1 root root    7 Nov 28 16:52 vol1 -> ../dm-0
lrwxrwxrwx.  1 root root    7 Nov 28 17:02 vol2 -> ../dm-9


log :
2014-11-28 08:53:29.612+0000: 3659: debug : virCommandRun:2249 : Result status 0, stdout: '  Rounding up size to full physical extent 524.00 MiB
  Logical volume "vol2" created.
' stderr: '2014-11-28 08:53:28.404+0000: 4532: debug : virFileClose:99 : Closed fd 25
2014-11-28 08:53:28.404+0000: 4532: debug : virFileClose:99 : Closed fd 27
2014-11-28 08:53:28.404+0000: 4532: debug : virFileClose:99 : Closed fd 23
'
2014-11-28 08:53:29.612+0000: 3659: debug : virFileClose:99 : Closed fd 24
2014-11-28 08:53:29.612+0000: 3659: debug : virFileClose:99 : Closed fd 26
2014-11-28 08:53:29.612+0000: 3659: debug : virFileClose:99 : Closed fd 23
2014-11-28 08:53:29.612+0000: 3659: debug : virCommandRunAsync:2398 : About to run /usr/sbin/lvs --separator '#' --noheadings --units b --unbuffered --nosuffix --options lv_name,origin,uuid,devices,segtype,stripes,seg_size,vg_extent_size,size,lv_attr vg0
2014-11-28 08:53:29.613+0000: 3659: debug : virFileClose:99 : Closed fd 23
2014-11-28 08:53:29.613+0000: 3659: debug : virFileClose:99 : Closed fd 25
2014-11-28 08:53:29.613+0000: 3659: debug : virFileClose:99 : Closed fd 27
2014-11-28 08:53:29.613+0000: 3659: debug : virCommandRunAsync:2401 : Command result 0, with PID 4645
2014-11-28 08:53:29.627+0000: 3659: debug : virCommandRun:2249 : Result status 0, stdout: '  vol1##72ai9f-9RBy-JBkP-MnQm-78mi-CdHi-T01UVv#/dev/sda6(0)#linear#1#1073741824#4194304#1073741824#-wi-a-----
  lvol1##8lfDFq-s2ay-RcvX-4mt2-kGBO-Gd2Z-FWGZ6S#lvol1_tdata(0)#thin-pool#1#549453824#4194304#549453824#twi-a-tz--
  vol2##bU1kfG-RMwn-1cCA-58Cg-bYbj-rVzy-2uWF0Y##thin#0#1073741824#4194304#1073741824#Vwi-a-tz--
' stderr: '2014-11-28 08:53:29.614+0000: 4645: debug : virFileClose:99 : Closed fd 25
2014-11-28 08:53:29.614+0000: 4645: debug : virFileClose:99 : Closed fd 27
2014-11-28 08:53:29.614+0000: 4645: debug : virFileClose:99 : Closed fd 23
'
2014-11-28 08:53:29.627+0000: 3659: debug : virFileClose:99 : Closed fd 26
2014-11-28 08:53:29.627+0000: 3659: debug : virFileClose:99 : Closed fd 24
2014-11-28 08:53:29.627+0000: 3659: error : virGetStorageVol:488 : key in virGetStorageVol must not be NULL
2014-11-28 08:53:29.627+0000: 3659: debug : virStoragePoolFree:13137 : pool=0x7f60f4008170

Additional info:
if capacity and allocation has the same value , it can create successfully.
# virsh vol-create-as logical-pool vol4 1G --allocation 1G
Vol vol4 created

# virsh vol-list logical-pool
 Name                 Path                                    
------------------------------------------------------------------------------
 vol1                 /dev/vg0/vol1                           
 vol4                 /dev/vg0/vol4

Comment 3 John Ferlan 2014-12-10 16:51:29 UTC
What version of LVM?  I have:

# lvs --version
  LVM version:     2.02.106(2) (2014-04-10)
  Library version: 1.02.85 (2014-04-10)
  Driver version:  4.27.0
#

when I use the same sequence on commands on my system, I get the following:
...
2014-12-10 15:38:56.714+0000: 16590: debug : virCommandRunAsync:2408 : About to run /usr/sbin/lvcreate --name vol_create_test -L 4096K --virtualsize 8192K LVM_Test
...
2014-12-10 15:38:57.449+0000: 16590: debug : virCommandRunAsync:2408 : About to run /usr/sbin/lvs --separator '#' --noheadings --units b --unbuffered --nosuffix --options lv_name,origin,uuid,devices,segtype,stripes,seg_size,vg_extent_size,size,lv_attr LVM_Test
...
2014-12-10 15:38:57.466+0000: 16590: debug : virCommandRun:2259 : Result status 0, stdout: '  vol_create_test#[vol_create_test_vorigin]#MuULH8-LeWm-f0aF-JIew-EJuo-eVgd-IwFmY0#/dev/sda3(0)#linear#1#4194304#4194304#4194304#swi-a-s---
...

Where the first 's' after the # in that last sequence of characters is what libvirt looks for when "deciding" whether to mark the volume as sparse (although it may be wrong, but it could be because of an LVM change/bug - something I haven't researched yet).

Compare that to the output you have:


2014-11-28 08:53:29.627+0000: 3659: debug : virCommandRun:2249 : Result status 0, stdout: '  vol1##72ai9f-9RBy-JBkP-MnQm-78mi-CdHi-T01UVv#/dev/sda6(0)#linear#1#1073741824#4194304#1073741824#-wi-a-----
  lvol1##8lfDFq-s2ay-RcvX-4mt2-kGBO-Gd2Z-FWGZ6S#lvol1_tdata(0)#thin-pool#1#549453824#4194304#549453824#twi-a-tz--
  vol2##bU1kfG-RMwn-1cCA-58Cg-bYbj-rVzy-2uWF0Y##thin#0#1073741824#4194304#1073741824#Vwi-a-tz--
...

Where the character here is a 'V' (and the rest of the line a bit more verbose to include the 'thin-pool'.

Looking at the 'lvs' man page for 'lvs' under the lv_attr bits section:

Volume  type:  (C)ache,  (m)irrored,  (M)irrored  without  initial  sync, (o)rigin, (O)rigin with merging snapshot, (r)aid, (R)aid without initial sync, (s)napshot, merging (S)napshot, (p)vmove, (v)irtual, mirror or raid (i)mage, mirror or raid (I)mage out-of-sync, mirror (l)og device, under (c)onversion, thin  (V)olume,  (t)hin  pool,  (T)hin pool data, raid or pool m(e)tadata or pool metadata spare

Seems to indicate perhaps we should be looking for 'V' instead of 's', although not looking for 's' will cause back-compat issues.

Curious what you'd get if you ran the commands to create a thin volume out of the context of libvirt.

That is (using my numbers since I didn't want to convert 1G & 521M!)

lvcreate --name vol_create_test -L 4096K --virtualsize 8192K logical-pool
lvs --separator '#' --noheadings --units b --unbuffered --nosuffix --options lv_name,origin,uuid,devices,segtype,stripes,seg_size,vg_extent_size,size,lv_attr logical-pool

(that second command is long)  I get the following on the lvs:

lvs --separator '#' --noheadings --units b --unbuffered --nosuffix --options lv_name,origin,uuid,devices,segtype,stripes,seg_size,vg_extent_size,size,lv_attr LVM_Test
  vol2#[vol2_vorigin]#ceFsjd-Xsb8-Z5Jk-Hz1n-io3X-l1EK-oKZaq9#/dev/sda3(0)#linear#1#8388608#4194304#8388608#swi-a-s---


You could also simply use 'lvs' to show the Attr column:

# lvs
  LV   VG       Attr       LSize   Pool Origin         Data%  Move Log Cpy%Sync Convert
  vol2 LVM_Test swi-a-s---   8.00m      [vol2_vorigin]   0.00                          
...

Comment 4 John Ferlan 2014-12-10 22:47:08 UTC
Well I have an answer after some digging and installing rhel7.1 on a guest and the LVM version is 2.02.113(2)-RHEL7 w/ library version 1.02.92-RHEL7

The same install using a RHEL70 guest is 2.02.104(2)-RHEL7 w/ library 1.02.84-RHEL7

Creating an LVM volume on the RHEL70 guest uses the 's', while the RHEL71 guest uses the 'V' - so looks like this needs to be fixed in 7.1; otherwise, LVM thin volumes won't work correctly - I'm investigating how best to resolve.

I've changed the flags to put this on the 7.1 radar.

Comment 7 Pei Zhang 2014-12-30 08:38:48 UTC
verify version:

libvirt-1.2.8-11.el7.x86_64
qemu-kvm-rhev-2.1.2-17.el7.x86_64
kernel-3.10.0-217.el7.x86_64

# lvs --version
  LVM version:     2.02.114(2)-RHEL7 (2014-12-01)
  Library version: 1.02.92-RHEL7 (2014-12-01)
  Driver version:  4.29.0


verify steps:

1.prepare a logical pool 
# cat logical-pool.xml 
<pool type='logical'>
  <name>logical-pool</name>
  <uuid>04edd85b-137b-4f13-8eec-31949a9ee9f5</uuid>
  <source>
  <name>logical-pool</name>
    <device path='/dev/sdb'/>
    <format type='lvm2'/>
  </source>
  <target>
    <path>/dev/logical-pool</path>
    <permissions>
      <mode>0755</mode>
      <owner>-1</owner>
      <group>-1</group>
    </permissions>
  </target>
</pool>

2. create a volume in logical pool with specified XML .vol-create successfully .
# cat logical-volume.xml 
<volume>
  <name>lv1</name>
<capacity>8192000</capacity>
  <allocation>4096000</allocation>
</volume>

# virsh vol-create logical-pool logical-volume.xml 
Vol lv1 created from logical-volume.xml

# virsh vol-list logical-pool --details
 Name  Path                   Type   Capacity  Allocation
----------------------------------------------------------
 lv1   /dev/logical-pool/lv1  block  8.00 MiB    4.00 MiB

3. try to create volume in logical pool via vol-create-as with the allocation and capacity values differ 

# virsh vol-create-as logical-pool vol2 6M --allocation 2M
Vol vol2 created

# virsh vol-list logical-pool --details
 Name  Path                    Type     Capacity  Allocation
-------------------------------------------------------------
 lv1   /dev/logical-pool/lv1   block    8.00 MiB    4.00 MiB
 vol2  /dev/logical-pool/vol2  block    8.00 MiB    4.00 MiB

4.try to create volume in logical pool via vol-create-as with --allocation = 0
# virsh vol-create-as logical-pool vol3 6M --allocation 0
Vol vol3 created

# virsh vol-list logical-pool --details
 Name  Path                    Type     Capacity  Allocation
-------------------------------------------------------------
 lv1   /dev/logical-pool/lv1   block    8.00 MiB    4.00 MiB
 vol2  /dev/logical-pool/vol2  block    8.00 MiB    4.00 MiB
 vol3  /dev/logical-pool/vol3  block    8.00 MiB    4.00 MiB

5. check via lvs command .
# lvs
  LV    VG           Attr       LSize   Pool Origin         Data%  Meta%  Move Log Cpy%Sync Convert
  lv1   logical-pool swi-a-s---   4.00m      [lv1_vorigin]  0.00                                   
  vol2  logical-pool swi-a-s---   4.00m      [vol2_vorigin] 0.00                                   
  vol3  logical-pool swi-a-s---   4.00m      [vol3_vorigin] 0.00                                   

6. do some operations with created volume.

#virsh vol-dumpxml  --pool logical-pool lv1

<volume type='block'>
  <name>lv1</name>
  <key>joGltm-zjaq-dFyx-7cLB-S9lp-FkPb-yf9J3T</key>
  <source>
    <device path='/dev/sdb'>
      <extent start='0' end='4194304'/>
    </device>
  </source>
  <capacity unit='bytes'>8388608</capacity>
  <allocation unit='bytes'>4194304</allocation>
  <target>
    <path>/dev/logical-pool/lv1</path>
    <permissions>
      <mode>0600</mode>
      <owner>0</owner>
      <group>6</group>
      <label>system_u:object_r:fixed_disk_device_t:s0</label>
    </permissions>
    <timestamps>
      <atime>1419917996.781325879</atime>
      <mtime>1419917996.781325879</mtime>
      <ctime>1419917996.827326936</ctime>
    </timestamps>
  </target>
</volume>

#virsh vol-info     --pool logical-pool lv1
Name:           lv1
Type:           block
Capacity:       8.00 MiB
Allocation:     4.00 MiB

#virsh vol-key   --pool logical-pool lv1
joGltm-zjaq-dFyx-7cLB-S9lp-FkPb-yf9J3T

# virsh vol-name /dev/logical-pool/lv3
lv3
# virsh vol-pool /dev/logical-pool/lv3
logical-pool

#virsh vol-delete   --pool logical-pool lv1
Vol lv1 deleted

Logical volume can be created successfully. Move to verified.

Comment 9 errata-xmlrpc 2015-03-05 07:47:38 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0323.html