Bug 1187533 - [RHEL 7 FEAT] libvirt is missing ceph support (rbd)
Summary: [RHEL 7 FEAT] libvirt is missing ceph support (rbd)
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libvirt
Version: 7.0
Hardware: All
OS: Linux
high
high
Target Milestone: rc
: ---
Assignee: Peter Krempa
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks: 1205796 1250958 1251599
TreeView+ depends on / blocked
 
Reported: 2015-01-30 11:11 UTC by Alexandros Gkesos
Modified: 2021-07-30 17:30 UTC (History)
10 users (show)

Fixed In Version: libvirt-1.2.14-1.el7
Doc Type: Enhancement
Doc Text:
Feature: Enable support for the ceph storage driver backend so that ceph storage pools can be accessed via libvirt APIs. (note: VMs with ceph disks were usable even without this) Reason: Libvirt already had the code to deal with ceph volumes but the support was not compiled in. Result: The storage driver works with ceph(rbd) storage pools. Users are able to use libvirt APIs as with other storage driver backends.
Clone Of:
: 1251599 (view as bug list)
Environment:
Last Closed: 2015-11-19 06:09:25 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2015:2202 0 normal SHIPPED_LIVE libvirt bug fix and enhancement update 2015-11-19 08:17:58 UTC

Comment 1 Peter Krempa 2015-03-05 15:30:52 UTC
Support enabled in upstream spec file:

commit 6cfc8834c858849cc74c3082078dc91fb1cbae38
Author: Peter Krempa <pkrempa@redhat.com>
Date:   Thu Mar 5 11:40:54 2015 +0100

    spec: Enable RBD storage driver in RHEL-7
    
    Use correct package names too as they differ.

v1.2.13-62-g6cfc883

Comment 3 Yang Yang 2015-05-20 03:21:40 UTC
Verified on libvirt-1.2.15-2.el7.x86_64

Steps
=================pool related virsh command==================
1. define/build/start rbd pool with following xml

# cat rbd-pool.xml
<pool type="rbd">
        <name>rbd</name>
        <source>
          <name>libvirt-pool</name>
          <host name='10.66.5.219' port='6789'/>
          <host name='osd2.redhat.com' port='6789'/>
          <host name='osd3.redhat.com' port='6789'/>

    <auth type='ceph' username='libvirt'>
      <secret usage='client.libvirt secret'/>
    </auth>
        </source>
      </pool>


# virsh pool-define rbd-pool.xml
Pool rbd defined from rbd-pool.xml

[root@rhel7_test yy]# virsh pool-build rbd
Pool rbd built

[root@rhel7_test yy]#  virsh pool-start rbd
Pool rbd started

[root@rhel7_test yy]# virsh vol-list rbd
 Name                 Path                                   
------------------------------------------------------------------------------
 yy1.img              libvirt-pool/yy1.img                   

2. refresh rbd pool

[root@rhel7_test yy]# virsh pool-refresh rbd
Pool rbd refreshed


3.display rbd pool

[root@rhel7_test yy]# virsh pool-list --type rbd
 Name                 State      Autostart
-------------------------------------------
 rbd                  active     no       

4.check rbd pool xml

[root@rhel7_test yy]# virsh pool-dumpxml rbd
<pool type='rbd'>
  <name>rbd</name>
  <uuid>1e6388ad-a8ca-4363-8a9d-68bb81b9735c</uuid>
  <capacity unit='bytes'>139701780480</capacity>
  <allocation unit='bytes'>135</allocation>
  <available unit='bytes'>99701280768</available>
  <source>
    <host name='10.66.5.219' port='6789'/>
    <host name='osd2.redhat.com' port='6789'/>
    <host name='osd3.redhat.com' port='6789'/>
    <name>libvirt-pool</name>

 <auth type='ceph' username='libvirt'>
      <secret usage='client.libvirt secret'/>
    </auth>
  </source>
</pool>

5.check pool-info, pool-uuid, pool-name

[root@rhel7_test yy]# virsh pool-info rbd
Name:           rbd
UUID:           1e6388ad-a8ca-4363-8a9d-68bb81b9735c
State:          running
Persistent:     yes
Autostart:      no
Capacity:       130.11 GiB
Allocation:     135.00 B
Available:      92.85 GiB

[root@rhel7_test yy]# virsh pool-uuid rbd
1e6388ad-a8ca-4363-8a9d-68bb81b9735c

[root@rhel7_test yy]# virsh pool-name 1e6388ad-a8ca-4363-8a9d-68bb81b9735c
rbd

6. check pool-autostart

[root@rhel7_test yy]# virsh pool-autostart rbd
Pool rbd marked as autostarted

[root@rhel7_test yy]# virsh pool-list rbd
 Name                 State      Autostart
-------------------------------------------
 rbd                  active     yes      

[root@rhel7_test yy]# virsh pool-autostart rbd --disable
Pool rbd unmarked as autostarted

[root@rhel7_test yy]# virsh pool-list rbd
 Name                 State      Autostart
-------------------------------------------
 rbd                  active     no       

7. check find-storage-pool-sources-as

[root@rhel7_test yy]# virsh find-storage-pool-sources-as rbd 10.66.5.219
error: Failed to find any rbd pool sources
error: this function is not supported by the connection driver: pool type 'rbd' does not support source discovery

8. edit pool

# virsh pool-edit rbd
Pool rbd XML configuration edited.

9. check pool-define-as

# virsh pool-define-as rbd-tmp rbd 10.66.4.150 --source-name libvirt-pool --auth-type ceph --auth-username libvirt --secret-usage client.libvirt\ secret
Pool rbd-tmp defined

# virsh pool-dumpxml rbd-tmp
<pool type='rbd'>
  <name>rbd-tmp</name>
  <uuid>4951d0b9-1122-4713-87b2-17a021dd1fba</uuid>
  <capacity unit='bytes'>139701780480</capacity>
  <allocation unit='bytes'>151</allocation>
  <available unit='bytes'>99698761728</available>
  <source>
    <host name='10.66.4.150'/>
    <name>libvirt-pool</name>
    <auth type='ceph' username='libvirt'>
      <secret usage='client.libvirt secret'/>
    </auth>
  </source>
</pool>


# virsh pool-start rbd-tmp
Pool rbd-tmp started

[root@rhel7_test yy]# virsh vol-list rbd-tmp
 Name                 Path                                   
------------------------------------------------------------------------------
 yy1.img              libvirt-pool/yy1.img    

10. check pool-create-as

# virsh pool-create-as rbd-tmp rbd 10.66.4.150 --source-name libvirt-pool --auth-type ceph --auth-username libvirt --secret-usage client.libvirt\ secret
Pool rbd-tmp created


[root@rhel7_test yy]# virsh pool-list --all
 Name                 State      Autostart
-------------------------------------------
 default              active     yes      
 disk                 inactive   no       
 fs                   inactive   no       
 HostVG               inactive   no       
 iscsi                inactive   no       
 rbd                  active     no       
 rbd-tmp              active     no       
 rbd-tmp-1            active     no 

# virsh pool-destroy rbd-tmp-1
Pool rbd-tmp-1 destroyed

[root@rhel7_test yy]# virsh pool-list --type rbd
 Name                 State      Autostart
-------------------------------------------
 rbd                  active     no       
 rbd-tmp              active     no  

11. check pool-destroy, pool-undefine

# virsh pool-destroy rbd-tmp
Pool rbd-tmp destroyed

[root@rhel7_test yy]# virsh pool-undefine rbd-tmp
Pool rbd-tmp has been undefined

[root@rhel7_test yy]# virsh pool-list --type rbd
 Name                 State      Autostart
-------------------------------------------
 rbd                  active     no 

12. check pool-delete

# virsh pool-delete rbd
error: Failed to delete pool rbd
error: this function is not supported by the connection driver: pool does not support pool deletion

==========================vol related virsh command==================
2. check vol-create, vol-list

# cat rbd.vol
<volume type='network'>
  <name>yy1.img</name>
  <source>
  </source>
  <capacity unit='bytes'>5368709120</capacity>
  <allocation unit='bytes'>5368709120</allocation>
  <target>
    <format type='unknown'/>
  </target>
</volume>

# virsh vol-create rbd rbd.vol
Vol yy1.img created from rbd.vol

[root@rhel7_test yy]# virsh vol-list rbd
 Name                 Path                                   
------------------------------------------------------------------------------
 yy1.img              libvirt-pool/yy1.img                   

3.check vol-dumpxml, vol-info

# virsh vol-dumpxml yy1.img rbd
<volume type='network'>
  <name>yy1.img</name>
  <key>libvirt-pool/yy1.img</key>
  <source>
  </source>
  <capacity unit='bytes'>5368709120</capacity>
  <allocation unit='bytes'>5368709120</allocation>
  <target>
    <path>libvirt-pool/yy1.img</path>
    <format type='unknown'/>                
  </target>
</volume>

# virsh vol-info libvirt-pool/yy1.img
Name:           yy1.img
Type:           network
Capacity:       5.00 GiB
Allocation:     5.00 GiB

4.check vol-key, vol-path, vol-pool, vol-name

# virsh vol-key yy1.img rbd
libvirt-pool/yy1.img

[root@rhel7_test yy]# virsh vol-path yy1.img rbd
libvirt-pool/yy1.img

[root@rhel7_test yy]# virsh vol-pool libvirt-pool/yy1.img
rbd

[root@rhel7_test yy]# virsh vol-name libvirt-pool/yy1.img
yy1.img

5. check vol-create-as

# virsh vol-create-as rbd yy2.img 1G --format raw
Vol yy2.img created

[root@rhel7_test yy]# virsh vol-list rbd
 Name                 Path                                   
------------------------------------------------------------------------------
 yy1.img              libvirt-pool/yy1.img                   
 yy2.img              libvirt-pool/yy2.img                   

[root@rhel7_test yy]# virsh vol-dumpxml libvirt-pool/yy2.img
<volume type='network'>
  <name>yy2.img</name>
  <key>libvirt-pool/yy2.img</key>
  <source>
  </source>
  <capacity unit='bytes'>1073741824</capacity>
  <allocation unit='bytes'>1073741824</allocation>
  <target>
    <path>libvirt-pool/yy2.img</path>
    <format type='unknown'/>
    <permissions>
      <mode>0600</mode>
      <owner>-1</owner>
      <group>-1</group>
    </permissions>
  </target>
</volume>

6. check vol-resize

# virsh vol-resize yy2.img 2G rbd
Size of volume 'yy2.img' successfully changed to 2G

[root@rhel7_test yy]# virsh vol-info yy2.img rbd
Name:           yy2.img
Type:           network
Capacity:       2.00 GiB
Allocation:     2.00 GiB

7. check vol-clone

# virsh vol-clone yy2.img yy3.img rbd
error: Failed to clone vol from yy2.img
error: this function is not supported by the connection driver: storage pool does not support volume creation from an existing volume

8. check vol-create-from

# virsh vol-create-from rbd rbd.vol yy2.img rbd
error: Failed to create vol from rbd.vol
error: this function is not supported by the connection driver: storage pool does not support volume creation from an existing volume


# cat rbd.vol
<volume type='network'>
  <name>yy3.img</name>
  <source>
  </source>
  <capacity unit='bytes'>2147483648</capacity>
  <allocation unit='bytes'>2147483648</allocation>
  <target>
    <format type='unknown'/>
    <permissions>
      <mode>0600</mode>
      <owner>-1</owner>
      <group>-1</group>
    </permissions>
  </target>
</volume>

# virsh vol-create rbd rbd.vol
Vol yy3.img created from rbd.vol

# virsh vol-delete yy3.img rbd
Vol yy3.img deleted

9. check vol-create-as

# virsh vol-create-as rbd yy3.img --capacity 2G --allocation 1G --format qcow2
Vol yy3.img created

# virsh vol-info yy3.img rbd
Name:           yy3.img
Type:           network
Capacity:       2.00 GiB
Allocation:     2.00 GiB

[root@rhel7_test yy]# virsh vol-dumpxml yy3.img rbd
<volume type='network'>
  <name>yy3.img</name>
  <key>libvirt-pool/yy3.img</key>
  <source>
  </source>
  <capacity unit='bytes'>2147483648</capacity>
  <allocation unit='bytes'>2147483648</allocation>
  <target>
    <path>libvirt-pool/yy3.img</path>
    <format type='unknown'/>
    <permissions>
      <mode>0600</mode>
      <owner>-1</owner>
      <group>-1</group>
    </permissions>
  </target>
</volume>

10. check vol-upload, vol-download

# virsh vol-upload --pool rbd --vol yy2.img rbd.vol
error: cannot upload to volume yy2.img
error: this function is not supported by the connection driver: storage pool doesn't support volume upload

# virsh vol-download --pool rbd --vol yy2.img rbd.vol
error: cannot download from volume yy2.img
error: this function is not supported by the connection driver: storage pool doesn't support volume download

11. check vol-wipe

# virsh vol-wipe yy3.img rbd
error: Failed to wipe vol yy3.img
error: this function is not supported by the connection driver: storage pool doesn't support volume wiping

12. use rbd volume as virtual disk

<disk type='volume' device='disk'>
      <driver name='qemu' type='raw'/>
      <source pool='rbd' volume='yy1.img'/>
      <target dev='vda' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </disk>

# virsh start vm5
error: Failed to start domain vm5
error: unsupported configuration: using 'rbd' pools for backing 'volume' disks isn't yet supported

Comment 5 errata-xmlrpc 2015-11-19 06:09:25 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2202.html

Comment 6 Yuri Arabadji 2021-07-30 15:36:22 UTC
What have you verified, darling? You've confirmed it DOESN'T work. Why was this bug closed if the verification confirmed the defect?

```
# virsh start vm5
error: Failed to start domain vm5
error: unsupported configuration: using 'rbd' pools for backing 'volume' disks isn't yet supported
```

Same happens over here on EL7: libvirt-4.5.0-36.el7_9.5.x86_64

Comment 7 Peter Krempa 2021-07-30 15:52:47 UTC
(In reply to Yuri Arabadji from comment #6)
> What have you verified, darling? You've confirmed it DOESN'T work. Why was
> this bug closed if the verification confirmed the defect?
> 
> ```
> # virsh start vm5
> error: Failed to start domain vm5
> error: unsupported configuration: using 'rbd' pools for backing 'volume'
> disks isn't yet supported
> ```

Note that the error is saying that the conversion function which converts a <disk type='volume' to the appropriate config isn't implemented. This is true even for current upstream as nobody ever contributed this function. You can still use <disk type='network'><source protocol='rbd'. This bug was about enabling the storage pool support. Arguably this by itself isn't very useful without the above convertor.

Comment 8 Yuri Arabadji 2021-07-30 17:30:08 UTC
Thanks for clarifying, Peter. That's exactly what I was confused about.


Note You need to log in before you can comment on or make changes to this bug.