Bug 1138516 - The name and format of volume in disk pool are changed after pool refreshed
Summary: The name and format of volume in disk pool are changed after pool refreshed
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libvirt
Version: 7.2
Hardware: x86_64
OS: Linux
medium
medium
Target Milestone: rc
: ---
Assignee: John Ferlan
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks: 1138523
TreeView+ depends on / blocked
 
Reported: 2014-09-05 04:06 UTC by Yang Yang
Modified: 2015-11-19 05:48 UTC (History)
8 users (show)

Fixed In Version: libvirt-1.2.13-1.el7
Doc Type: Bug Fix
Doc Text:
Previously, using the "virsh pool-refresh" command, or restarting or refreshing the libvirtd service after renaming a virtual storage volume in some cases caused the "virsh vol-list" to display an incorrect name for the renamed storage volume. This update adds a check for the resulting name, which returns an error if the storage volume name is incorrect.
Clone Of:
: 1138523 (view as bug list)
Environment:
Last Closed: 2015-11-19 05:48:21 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2015:2202 0 normal SHIPPED_LIVE libvirt bug fix and enhancement update 2015-11-19 08:17:58 UTC

Description Yang Yang 2014-09-05 04:06:58 UTC
Description of problem:
Created volumes in disk pool. Refresh the pool. However, the name and format of the volumes are changed 


Version-Release number of selected component (if applicable):
3.10.0-152.el7.x86_64
libvirt-1.2.8-1.el7.x86_64

How reproducible:
100%

steps:

# parted /dev/sdc
GNU Parted 2.1
Using /dev/sdc
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) print                                                            
Model: Kingston DT Ultimate G2 (scsi)
Disk /dev/sdc: 16.0GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos


1. Define/Start a disk pool
# cat pool-disk.xml
<pool type='disk'>
  <name>disk-pool</name>
  <source>
    <device path='/dev/sdc'>
    </device>
    <format type='dos'/>
  </source>
  <target>
    <path>/dev</path>
    <permissions>
      <mode>0700</mode>
      <owner>0</owner>
      <group>0</group>
    </permissions>
  </target>
</pool>

#virsh pool-define pool-disk.xml
# virsh pool-start disk-pool

2. create volumes in disk pool
# for i in linux fat16 fat32 linux-swap; do virsh vol-create-as disk-pool vol-$i --format $i 1G; done
Vol vol-linux created

Vol vol-fat16 created

Vol vol-fat32 created

Vol vol-linux-swap created

3.List the volumes in disk pool
# virsh vol-list disk-pool
 Name                 Path                                    
------------------------------------------------------------------------------
 vol-fat16            /dev/sdc2                              
 vol-fat32            /dev/sdc3                              
 vol-linux            /dev/sdc1                              
 vol-linux-swap       /dev/sdc4  

4. check the volume format
# virsh vol-dumpxml vol-fat16 disk-pool
........
    <path>/dev/sdc2</path>
    <format type='fat16'/>
...........

5. refresh the disk pool
# virsh pool-refresh disk-pool
Pool disk-pool refreshed

6. List the volumes again
# virsh vol-list disk-pool
 Name                 Path                                    
------------------------------------------------------------------------------
 sdc1                 /dev/sdc1                              
 sdc2                 /dev/sdc2                              
 sdc3                 /dev/sdc3                              
 sdc4                 /dev/sdc4  

7. check the volume format again
# virsh vol-dumpxml sdc2 disk-pool
...........
    <path>/dev/sdc2</path>
    <format type='none'/>            -----changed
...........

Actual Results:
in step 6, the name of all the volumes changed
in step 7, the format of all the volumes changed into none

Expected Results:
in step6, 7, the name and format should remain the same after pool refreshed


Additional info:

Comment 2 John Ferlan 2014-12-18 19:01:12 UTC
Resolution of this issue for refresh could be done; however, a libvirt restart or start after reboot will still result in the 'name' and 'format type' reverting.

This is because these values cannot be saved via the parted commands libvirt uses in order to initially create the volume.  So while libvirtd could keep track while it's running, the end result is if libvirtd restarts, then we have nothing "easy" to reference in order to restore the name and format type initially created.

The name problem is not an issue for other storage pools since the name is typically the file, directory, lvm-name, etc. of the resource. I didn't check on where the 'format type' could/would be stored for other pool types.

I'll look into ways of perhaps being able to save the data somehow/somewhere for the reboot/restart cases.

Comment 3 John Ferlan 2015-01-28 23:19:34 UTC
My initial upstream patches to create a state directory to store the running pool's volume XML was rejected with the primary concern being the need. See:

http://www.redhat.com/archives/libvir-list/2015-January/msg00465.html

and follow-ups in 3/3.

The original expectation for a disk pool is that someone would create "correctly named" volumes. Where correctly named would take into account the path to the disk (e.g. /dev/sdb) and generate partitions as 'parted' generates and names the partitions /dev/sdb1, /dev/sdb2, etc. and the names utilzes would be likewise 'sdb1', 'sdb2', etc.

So I reworked the patches in order to require the name to be as expected, see:

http://www.redhat.com/archives/libvir-list/2015-January/msg00873.html

These patches were pushed with the final one being:

$ git describe 9bbbb9121654ad82931f21ef7f2fb1056670b955
v1.2.12-37-g9bbbb91
$

With respect to the "format type" value - that is expected to be the file system on the guest for the volume; however, there's no code within libvirt that "requires" a specific type, so it's a field that's not really used.

I've updated the man and web page in order to describe the fields/data more clearly.

Along the way of debugging, I uncovered a number of other minor issues which were fixed prior to the above named patch which is all that's really required in order to fix the problem. 

When libvirt is rebased all the patches will be picked up.

Comment 6 Yang Yang 2015-05-20 09:44:07 UTC
Hi John

Target format of extended disk backing volume changed to none after pool refreshed. Does it deserve a fix?

1. define/build/start a disk pool
# cat disk-pool.xml
<pool type="disk">
        <name>disk</name>
        <source>
          <device path='/dev/sdc'/>
<format type='dos'/>
        </source>
        <target>
          <path>/dev</path>
        </target>
      </pool>

2. create a primary partition
# virsh vol-create-as disk sdc1 100M
Vol sdc1 created

3. create an extended partition
# cat disk-vol.xml
<volume type='block'>
  <name>sdc2</name>
  <capacity unit='bytes'>105094656</capacity>
  <allocation unit='bytes'>105094656</allocation>
  <target>
    <format type='extended'/>
  </target>
</volume>

# virsh vol-create disk disk-vol.xml
Vol sdc2 created from disk-vol.xml

# virsh vol-list disk
 Name                 Path                                    
------------------------------------------------------------------------------
 sdc1                 /dev/sdc1                              
 sdc2                 /dev/sdc2  

# virsh vol-dumpxml sdc2 disk
<volume type='block'>
  <name>sdc2</name>
  <key>/dev/sdc2</key>
  <source>
    <device path='/dev/sdc'>
      <extent start='105126912' end='210253824'/>
    </device>
  </source>
  <capacity unit='bytes'>105126912</capacity>
  <allocation unit='bytes'>105126912</allocation>
  <target>
    <path>/dev/sdc2</path>
    <format type='extended'/>
    <permissions>
      <mode>0660</mode>
      <owner>0</owner>
      <group>6</group>
      <label>system_u:object_r:fixed_disk_device_t:s0</label>
    </permissions>
    <timestamps>
      <atime>1432111754.016640165</atime>
      <mtime>1432111754.016640165</mtime>
      <ctime>1432111754.016640165</ctime>
    </timestamps>
  </target>
</volume>

3. check vol format after pool refreshed
# virsh pool-refresh disk
Pool disk refreshed

Regards
Yang
[root@rhel7_test yy]# virsh vol-dumpxml sdc2 disk
<volume type='block'>
  <name>sdc2</name>
  <key>/dev/sdc2</key>
  <source>
    <device path='/dev/sdc'>
      <extent start='105126912' end='210253824'/>
    </device>
  </source>
  <capacity unit='bytes'>105126912</capacity>
  <allocation unit='bytes'>105126912</allocation>
  <target>
    <path>/dev/sdc2</path>
    <format type='none'/>         ========> changed
    <permissions>
      <mode>0660</mode>
      <owner>0</owner>
      <group>6</group>
      <label>system_u:object_r:fixed_disk_device_t:s0</label>
    </permissions>
    <timestamps>
      <atime>1432111754.111640179</atime>
      <mtime>1432111754.016640165</mtime>
      <ctime>1432111754.016640165</ctime>
    </timestamps>
  </target>
</volume>

4. check partition type parted shows
# parted /dev/sdc
GNU Parted 3.1
Using /dev/sdc
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) print                                                            
Model: Alcor Flash Disk (scsi)
Disk /dev/sdc: 1048MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:

Number  Start   End    Size   Type      File system  Flags
 1      32.3kB  105MB  105MB  primary
 2      105MB   210MB  105MB  extended               lba

Comment 7 John Ferlan 2015-05-20 15:45:44 UTC
No - that's as designed and documented and part of the patch for this bug...


See: http://libvirt.org/formatstorage.html
...
Target elements

A single target element is contained within the top level volume element. This tag is used to describe the mapping of the storage volume into the host filesystem. It can contain the following child elements: 
...

format
    Provides information about the pool specific volume format. For disk pools it will provide the partition table format type, but is not preserved after a pool refresh or libvirtd restart. Use extended in order to create an extended disk extent partition. For filesystem or directory pools it will provide the file format type, eg cow, qcow, vmdk, raw. If omitted when creating a volume, the pool's default format will be used. The actual format is specified via the type attribute. Consult the storage driver page for the list of valid volume format type values for each specific pool. The format will be ignored on input for pools without a volume format type value and the default pool format will be used. Since 0.4.1


...

The "new" text over previous version of the code being:
...
-        For disk pools it will provide the partition type. For filesystem
+        For disk pools it will provide the partition table format type, but is
+        not preserved after a pool refresh or libvirtd restart. Use extended
+        in order to create an extended disk extent partition. For filesystem
...

Comment 8 Yang Yang 2015-05-21 03:32:00 UTC
Thanks John.

I have another question. The pool alloc information are wrong after 2nd volume created. It's available after I did a pool-refresh. Does it deserve a fix? What's expected alloc while there exists an extended partition in disk pool ?

e.g.
# virsh pool-info disk
Name:           disk
UUID:           b0b3c21e-31df-44d0-913e-f8f7bda080d1
State:          running
Persistent:     yes
Autostart:      yes
Capacity:       996.22 MiB
Allocation:     0.00 B
Available:      996.19 MiB

[root@rhel7_test yy]# virsh vol-create-as disk sdc1 200M
Vol sdc1 created

[root@rhel7_test yy]# virsh pool-info disk
Name:           disk
UUID:           b0b3c21e-31df-44d0-913e-f8f7bda080d1
State:          running
Persistent:     yes
Autostart:      yes
Capacity:       996.22 MiB
Allocation:     200.48 MiB
Available:      795.70 MiB

[root@rhel7_test yy]# virsh vol-create-as disk sdc2 200M
Vol sdc2 created

[root@rhel7_test yy]# virsh pool-info disk
Name:           disk
UUID:           b0b3c21e-31df-44d0-913e-f8f7bda080d1
State:          running
Persistent:     yes
Autostart:      yes
Capacity:       996.22 MiB
Allocation:     200.51 MiB      ==========> alloc still 200M
Available:      595.19 MiB

# virsh pool-refresh disk
Pool disk refreshed

[root@rhel7_test yy]# virsh pool-info disk
Name:           disk
UUID:           b0b3c21e-31df-44d0-913e-f8f7bda080d1
State:          running
Persistent:     yes
Autostart:      yes
Capacity:       996.22 MiB
Allocation:     401.00 MiB     ===========> alloc is right after pool-refresh
Available:      595.19 MiB

# virsh vol-create-as disk sdc3 200M --format extended
Vol sdc3 created

[root@rhel7_test yy]# virsh pool-info disk
Name:           disk
UUID:           b0b3c21e-31df-44d0-913e-f8f7bda080d1
State:          running
Persistent:     yes
Autostart:      yes
Capacity:       996.22 MiB
Allocation:     0.00 B    =====> alloc is 0 after creating extended partition
Available:      595.16 MiB

# virsh pool-refresh disk
Pool disk refreshed

[root@rhel7_test yy]# virsh pool-info disk
Name:           disk
UUID:           b0b3c21e-31df-44d0-913e-f8f7bda080d1
State:          running
Persistent:     yes
Autostart:      yes
Capacity:       996.22 MiB
Allocation:     401.00 MiB ====> changed to 400M instead of 600M, expected ?
Available:      595.16 MiB


# parted /dev/sdc
GNU Parted 3.1
Using /dev/sdc
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) print                                                            
Model: Alcor Flash Disk (scsi)
Disk /dev/sdc: 1048MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags: 

Number  Start   End    Size   Type      File system  Flags
 1      32.3kB  210MB  210MB  primary
 2      210MB   421MB  210MB  primary
 3      421MB   631MB  210MB  extended               lba

Regards
Yang

Comment 9 John Ferlan 2015-05-21 17:26:57 UTC
Sure worthy of a new bug...

What actually happens is on volume create the pool fields are zeroed out and the allocation set to the allocation of the "current" partition.

I am working up a fix for it, but may as well have a new bz for it for tracking.

Comment 10 Yang Yang 2015-05-22 02:23:18 UTC
Thanks John

Open a new bz to track it
https://bugzilla.redhat.com/show_bug.cgi?id=1224018

Comment 11 Yang Yang 2015-05-22 02:51:28 UTC
Verified on libvirt-1.2.15-2.el7.x86_64

Steps
1. create disk backing volume with invalid name
#virsh vol-create-as disk vol1 100M
error: Failed to create vol vol1
error: invalid argument: invalid partition name 'vol1', expected 'sdc1'

2. create disk backing volume with valid name
#virsh vol-create-as disk sdc1 100M
Vol sdc1 created

# virsh vol-list disk
Name Path
------------------------------------------------------------------------------
sdc1 /dev/sdc1

#virsh vol-info sdc1 disk
Name: sdc1
Type: block
Capacity: 100.23 MiB
Allocation: 100.23 MiB

#virsh vol-dumpxml sdc1 disk
<volume type='block'>
<name>sdc1</name>
<key>/dev/sdc1</key>
<source>
<device path='/dev/sdc'>
<extent start='32256' end='105126912'/>
</device>
</source>
<capacity unit='bytes'>105094656</capacity>
<allocation unit='bytes'>105094656</allocation>
<target>
<path>/dev/sdc1</path>
<format type='none'/>
<permissions>
<mode>0660</mode>
<owner>0</owner>
<group>6</group>
<label>system_u:object_r:fixed_disk_device_t:s0</label>
</permissions>
<timestamps>
<atime>1432108245.950256003</atime>
<mtime>1432108245.950256003</mtime>
<ctime>1432108245.950256003</ctime>
</timestamps>
</target>
</volume>

3. create extended partition

#cat disk-vol.xml
 <volume type='block'>
 <name>sdc2</name>
 <capacity unit='bytes'>105094656</capacity>
 <allocation unit='bytes'>105094656</allocation>
 <target>
 <format type='extended'/>
 </target>
 </volume>

#virsh vol-create disk disk-vol.xml
 Vol sdc2 created from disk-vol.xml

#virsh vol-dumpxml sdc2 disk
 <volume type='block'>
 <name>sdc2</name>
 <key>/dev/sdc2</key>
 <source>
 <device path='/dev/sdc'>
 <extent start='105126912' end='210253824'/>
 </device>
 </source>
 <capacity unit='bytes'>105126912</capacity>
 <allocation unit='bytes'>105126912</allocation>
 <target>
 <path>/dev/sdc2</path>
 <format type='extended'/>
 <permissions>
 <mode>0660</mode>
 <owner>0</owner>
 <group>6</group>
 <label>system_u:object_r:fixed_disk_device_t:s0</label>
 </permissions>
 <timestamps>
 <atime>1432111754.016640165</atime>
 <mtime>1432111754.016640165</mtime>
 <ctime>1432111754.016640165</ctime>
 </timestamps>
 </target>
 </volume>

#cat disk-vol.xml
 <volume type='block'>
 <name>sdc3</name>
 <capacity unit='bytes'>105094656</capacity>
 <allocation unit='bytes'>105094656</allocation>
 <target>
 <format type='extended'/>
 </target>
 </volume>

4. create 1 more extended partition
#virsh vol-create disk disk-vol.xml
 error: Failed to create vol from disk-vol.xml
 error: internal error: extended partition already exists

#virsh pool-refresh disk
 Pool disk refreshed

# virsh vol-dumpxml sdc2 disk
<volume type='block'>
<name>sdc2</name>
<key>/dev/sdc2</key>
<source>
<device path='/dev/sdc'>
<extent start='105126912' end='210253824'/>
</device>
</source>
<capacity unit='bytes'>105126912</capacity>
<allocation unit='bytes'>105126912</allocation>
<target>
<path>/dev/sdc2</path>
<format type='none'/> ---> changed
<permissions>
<mode>0660</mode>
<owner>0</owner>
<group>6</group>
<label>system_u:object_r:fixed_disk_device_t:s0</label>
</permissions>
<timestamps>
<atime>1432111132.997546293</atime>
<mtime>1432111132.902546283</mtime>
<ctime>1432111132.902546283</ctime>
</timestamps>
</target>
</volume>

#parted /dev/sdc
GNU Parted 3.1
Using /dev/sdc
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) print
Model: Alcor Flash Disk (scsi)
Disk /dev/sdc: 1048MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:

Number Start End Size Type File system Flags
1 32.3kB 105MB 105MB primary
2 105MB 210MB 105MB extended lba

[root@rhel7_test yy]# virsh vol-list disk
Name Path
------------------------------------------------------------------------------
sdc1 /dev/sdc1
sdc2 /dev/sdc2

5. delete vol
#virsh vol-delete sdc2 disk
Vol sdc2 deleted

#virsh vol-create-as disk sdc2 100M
 Vol sdc2 created

[root@rhel7_test yy]# virsh vol-create-as disk sdc3 100M
Vol sdc3 created

[root@rhel7_test yy]# virsh vol-create-as disk sdc4 600M --format extended
Vol sdc4 created

#virsh vol-create-as disk sdc5 100M
 Vol sdc5 created

#parted /dev/sdc
 GNU Parted 3.1
 Using /dev/sdc
 Welcome to GNU Parted! Type 'help' to view a list of commands.
 (parted) print
 Model: Alcor Flash Disk (scsi)
 Disk /dev/sdc: 1048MB
 Sector size (logical/physical): 512B/512B
 Partition Table: msdos
 Disk Flags:

Number Start End Size Type File system Flags
1 32.3kB 105MB 105MB primary
2 105MB 210MB 105MB primary
3 210MB 315MB 105MB primary
4 315MB 945MB 630MB extended lba
5 315MB 421MB 105MB logical

6. delete extended partition and check logical partitions are removed also
#virsh vol-delete sdc4 disk
Vol sdc4 deleted

[root@rhel7_test yy]# virsh vol-list disk
Name Path
------------------------------------------------------------------------------
sdc1 /dev/sdc1
sdc2 /dev/sdc2
sdc3 /dev/sdc3

Comment 13 errata-xmlrpc 2015-11-19 05:48:21 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2202.html


Note You need to log in before you can comment on or make changes to this bug.