RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1187533 - [RHEL 7 FEAT] libvirt is missing ceph support (rbd)
Summary: [RHEL 7 FEAT] libvirt is missing ceph support (rbd)
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libvirt
Version: 7.0
Hardware: All
OS: Linux
high
high
Target Milestone: rc
: ---
Assignee: Peter Krempa
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks: 1205796 1250958 1251599
TreeView+ depends on / blocked
 
Reported: 2015-01-30 11:11 UTC by Alexandros Gkesos
Modified: 2021-07-30 17:30 UTC (History)
10 users (show)

Fixed In Version: libvirt-1.2.14-1.el7
Doc Type: Enhancement
Doc Text:
Feature: Enable support for the ceph storage driver backend so that ceph storage pools can be accessed via libvirt APIs. (note: VMs with ceph disks were usable even without this) Reason: Libvirt already had the code to deal with ceph volumes but the support was not compiled in. Result: The storage driver works with ceph(rbd) storage pools. Users are able to use libvirt APIs as with other storage driver backends.
Clone Of:
: 1251599 (view as bug list)
Environment:
Last Closed: 2015-11-19 06:09:25 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2015:2202 0 normal SHIPPED_LIVE libvirt bug fix and enhancement update 2015-11-19 08:17:58 UTC

Comment 1 Peter Krempa 2015-03-05 15:30:52 UTC 
Support enabled in upstream spec file:

commit 6cfc8834c858849cc74c3082078dc91fb1cbae38
Author: Peter Krempa <pkrempa>
Date:   Thu Mar 5 11:40:54 2015 +0100

    spec: Enable RBD storage driver in RHEL-7
    
    Use correct package names too as they differ.

v1.2.13-62-g6cfc883
Comment 3 Yang Yang 2015-05-20 03:21:40 UTC 
Verified on libvirt-1.2.15-2.el7.x86_64

Steps
=================pool related virsh command==================
1. define/build/start rbd pool with following xml

# cat rbd-pool.xml
<pool type="rbd">
        <name>rbd</name>
        <source>
          <name>libvirt-pool</name>
          <host name='10.66.5.219' port='6789'/>
          <host name='osd2.redhat.com' port='6789'/>
          <host name='osd3.redhat.com' port='6789'/>

    <auth type='ceph' username='libvirt'>
      <secret usage='client.libvirt secret'/>
    </auth>
        </source>
      </pool>


# virsh pool-define rbd-pool.xml
Pool rbd defined from rbd-pool.xml

[root@rhel7_test yy]# virsh pool-build rbd
Pool rbd built

[root@rhel7_test yy]#  virsh pool-start rbd
Pool rbd started

[root@rhel7_test yy]# virsh vol-list rbd
 Name                 Path                                   
------------------------------------------------------------------------------
 yy1.img              libvirt-pool/yy1.img                   

2. refresh rbd pool

[root@rhel7_test yy]# virsh pool-refresh rbd
Pool rbd refreshed


3.display rbd pool

[root@rhel7_test yy]# virsh pool-list --type rbd
 Name                 State      Autostart
-------------------------------------------
 rbd                  active     no       

4.check rbd pool xml

[root@rhel7_test yy]# virsh pool-dumpxml rbd
<pool type='rbd'>
  <name>rbd</name>
  <uuid>1e6388ad-a8ca-4363-8a9d-68bb81b9735c</uuid>
  <capacity unit='bytes'>139701780480</capacity>
  <allocation unit='bytes'>135</allocation>
  <available unit='bytes'>99701280768</available>
  <source>
    <host name='10.66.5.219' port='6789'/>
    <host name='osd2.redhat.com' port='6789'/>
    <host name='osd3.redhat.com' port='6789'/>
    <name>libvirt-pool</name>

 <auth type='ceph' username='libvirt'>
      <secret usage='client.libvirt secret'/>
    </auth>
  </source>
</pool>

5.check pool-info, pool-uuid, pool-name

[root@rhel7_test yy]# virsh pool-info rbd
Name:           rbd
UUID:           1e6388ad-a8ca-4363-8a9d-68bb81b9735c
State:          running
Persistent:     yes
Autostart:      no
Capacity:       130.11 GiB
Allocation:     135.00 B
Available:      92.85 GiB

[root@rhel7_test yy]# virsh pool-uuid rbd
1e6388ad-a8ca-4363-8a9d-68bb81b9735c

[root@rhel7_test yy]# virsh pool-name 1e6388ad-a8ca-4363-8a9d-68bb81b9735c
rbd

6. check pool-autostart

[root@rhel7_test yy]# virsh pool-autostart rbd
Pool rbd marked as autostarted

[root@rhel7_test yy]# virsh pool-list rbd
 Name                 State      Autostart
-------------------------------------------
 rbd                  active     yes      

[root@rhel7_test yy]# virsh pool-autostart rbd --disable
Pool rbd unmarked as autostarted

[root@rhel7_test yy]# virsh pool-list rbd
 Name                 State      Autostart
-------------------------------------------
 rbd                  active     no       

7. check find-storage-pool-sources-as

[root@rhel7_test yy]# virsh find-storage-pool-sources-as rbd 10.66.5.219
error: Failed to find any rbd pool sources
error: this function is not supported by the connection driver: pool type 'rbd' does not support source discovery

8. edit pool

# virsh pool-edit rbd
Pool rbd XML configuration edited.

9. check pool-define-as

# virsh pool-define-as rbd-tmp rbd 10.66.4.150 --source-name libvirt-pool --auth-type ceph --auth-username libvirt --secret-usage client.libvirt\ secret
Pool rbd-tmp defined

# virsh pool-dumpxml rbd-tmp
<pool type='rbd'>
  <name>rbd-tmp</name>
  <uuid>4951d0b9-1122-4713-87b2-17a021dd1fba</uuid>
  <capacity unit='bytes'>139701780480</capacity>
  <allocation unit='bytes'>151</allocation>
  <available unit='bytes'>99698761728</available>
  <source>
    <host name='10.66.4.150'/>
    <name>libvirt-pool</name>
    <auth type='ceph' username='libvirt'>
      <secret usage='client.libvirt secret'/>
    </auth>
  </source>
</pool>


# virsh pool-start rbd-tmp
Pool rbd-tmp started

[root@rhel7_test yy]# virsh vol-list rbd-tmp
 Name                 Path                                   
------------------------------------------------------------------------------
 yy1.img              libvirt-pool/yy1.img    

10. check pool-create-as

# virsh pool-create-as rbd-tmp rbd 10.66.4.150 --source-name libvirt-pool --auth-type ceph --auth-username libvirt --secret-usage client.libvirt\ secret
Pool rbd-tmp created


[root@rhel7_test yy]# virsh pool-list --all
 Name                 State      Autostart
-------------------------------------------
 default              active     yes      
 disk                 inactive   no       
 fs                   inactive   no       
 HostVG               inactive   no       
 iscsi                inactive   no       
 rbd                  active     no       
 rbd-tmp              active     no       
 rbd-tmp-1            active     no 

# virsh pool-destroy rbd-tmp-1
Pool rbd-tmp-1 destroyed

[root@rhel7_test yy]# virsh pool-list --type rbd
 Name                 State      Autostart
-------------------------------------------
 rbd                  active     no       
 rbd-tmp              active     no  

11. check pool-destroy, pool-undefine

# virsh pool-destroy rbd-tmp
Pool rbd-tmp destroyed

[root@rhel7_test yy]# virsh pool-undefine rbd-tmp
Pool rbd-tmp has been undefined

[root@rhel7_test yy]# virsh pool-list --type rbd
 Name                 State      Autostart
-------------------------------------------
 rbd                  active     no 

12. check pool-delete

# virsh pool-delete rbd
error: Failed to delete pool rbd
error: this function is not supported by the connection driver: pool does not support pool deletion

==========================vol related virsh command==================
2. check vol-create, vol-list

# cat rbd.vol
<volume type='network'>
  <name>yy1.img</name>
  <source>
  </source>
  <capacity unit='bytes'>5368709120</capacity>
  <allocation unit='bytes'>5368709120</allocation>
  <target>
    <format type='unknown'/>
  </target>
</volume>

# virsh vol-create rbd rbd.vol
Vol yy1.img created from rbd.vol

[root@rhel7_test yy]# virsh vol-list rbd
 Name                 Path                                   
------------------------------------------------------------------------------
 yy1.img              libvirt-pool/yy1.img                   

3.check vol-dumpxml, vol-info

# virsh vol-dumpxml yy1.img rbd
<volume type='network'>
  <name>yy1.img</name>
  <key>libvirt-pool/yy1.img</key>
  <source>
  </source>
  <capacity unit='bytes'>5368709120</capacity>
  <allocation unit='bytes'>5368709120</allocation>
  <target>
    <path>libvirt-pool/yy1.img</path>
    <format type='unknown'/>                
  </target>
</volume>

# virsh vol-info libvirt-pool/yy1.img
Name:           yy1.img
Type:           network
Capacity:       5.00 GiB
Allocation:     5.00 GiB

4.check vol-key, vol-path, vol-pool, vol-name

# virsh vol-key yy1.img rbd
libvirt-pool/yy1.img

[root@rhel7_test yy]# virsh vol-path yy1.img rbd
libvirt-pool/yy1.img

[root@rhel7_test yy]# virsh vol-pool libvirt-pool/yy1.img
rbd

[root@rhel7_test yy]# virsh vol-name libvirt-pool/yy1.img
yy1.img

5. check vol-create-as

# virsh vol-create-as rbd yy2.img 1G --format raw
Vol yy2.img created

[root@rhel7_test yy]# virsh vol-list rbd
 Name                 Path                                   
------------------------------------------------------------------------------
 yy1.img              libvirt-pool/yy1.img                   
 yy2.img              libvirt-pool/yy2.img                   

[root@rhel7_test yy]# virsh vol-dumpxml libvirt-pool/yy2.img
<volume type='network'>
  <name>yy2.img</name>
  <key>libvirt-pool/yy2.img</key>
  <source>
  </source>
  <capacity unit='bytes'>1073741824</capacity>
  <allocation unit='bytes'>1073741824</allocation>
  <target>
    <path>libvirt-pool/yy2.img</path>
    <format type='unknown'/>
    <permissions>
      <mode>0600</mode>
      <owner>-1</owner>
      <group>-1</group>
    </permissions>
  </target>
</volume>

6. check vol-resize

# virsh vol-resize yy2.img 2G rbd
Size of volume 'yy2.img' successfully changed to 2G

[root@rhel7_test yy]# virsh vol-info yy2.img rbd
Name:           yy2.img
Type:           network
Capacity:       2.00 GiB
Allocation:     2.00 GiB

7. check vol-clone

# virsh vol-clone yy2.img yy3.img rbd
error: Failed to clone vol from yy2.img
error: this function is not supported by the connection driver: storage pool does not support volume creation from an existing volume

8. check vol-create-from

# virsh vol-create-from rbd rbd.vol yy2.img rbd
error: Failed to create vol from rbd.vol
error: this function is not supported by the connection driver: storage pool does not support volume creation from an existing volume


# cat rbd.vol
<volume type='network'>
  <name>yy3.img</name>
  <source>
  </source>
  <capacity unit='bytes'>2147483648</capacity>
  <allocation unit='bytes'>2147483648</allocation>
  <target>
    <format type='unknown'/>
    <permissions>
      <mode>0600</mode>
      <owner>-1</owner>
      <group>-1</group>
    </permissions>
  </target>
</volume>

# virsh vol-create rbd rbd.vol
Vol yy3.img created from rbd.vol

# virsh vol-delete yy3.img rbd
Vol yy3.img deleted

9. check vol-create-as

# virsh vol-create-as rbd yy3.img --capacity 2G --allocation 1G --format qcow2
Vol yy3.img created

# virsh vol-info yy3.img rbd
Name:           yy3.img
Type:           network
Capacity:       2.00 GiB
Allocation:     2.00 GiB

[root@rhel7_test yy]# virsh vol-dumpxml yy3.img rbd
<volume type='network'>
  <name>yy3.img</name>
  <key>libvirt-pool/yy3.img</key>
  <source>
  </source>
  <capacity unit='bytes'>2147483648</capacity>
  <allocation unit='bytes'>2147483648</allocation>
  <target>
    <path>libvirt-pool/yy3.img</path>
    <format type='unknown'/>
    <permissions>
      <mode>0600</mode>
      <owner>-1</owner>
      <group>-1</group>
    </permissions>
  </target>
</volume>

10. check vol-upload, vol-download

# virsh vol-upload --pool rbd --vol yy2.img rbd.vol
error: cannot upload to volume yy2.img
error: this function is not supported by the connection driver: storage pool doesn't support volume upload

# virsh vol-download --pool rbd --vol yy2.img rbd.vol
error: cannot download from volume yy2.img
error: this function is not supported by the connection driver: storage pool doesn't support volume download

11. check vol-wipe

# virsh vol-wipe yy3.img rbd
error: Failed to wipe vol yy3.img
error: this function is not supported by the connection driver: storage pool doesn't support volume wiping

12. use rbd volume as virtual disk

<disk type='volume' device='disk'>
      <driver name='qemu' type='raw'/>
      <source pool='rbd' volume='yy1.img'/>
      <target dev='vda' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </disk>

# virsh start vm5
error: Failed to start domain vm5
error: unsupported configuration: using 'rbd' pools for backing 'volume' disks isn't yet supported
Comment 5 errata-xmlrpc 2015-11-19 06:09:25 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2202.html
Comment 6 Yuri Arabadji 2021-07-30 15:36:22 UTC 
What have you verified, darling? You've confirmed it DOESN'T work. Why was this bug closed if the verification confirmed the defect?

```
# virsh start vm5
error: Failed to start domain vm5
error: unsupported configuration: using 'rbd' pools for backing 'volume' disks isn't yet supported
```

Same happens over here on EL7: libvirt-4.5.0-36.el7_9.5.x86_64
Comment 7 Peter Krempa 2021-07-30 15:52:47 UTC 
(In reply to Yuri Arabadji from comment #6)
> What have you verified, darling? You've confirmed it DOESN'T work. Why was
> this bug closed if the verification confirmed the defect?
> 
> ```
> # virsh start vm5
> error: Failed to start domain vm5
> error: unsupported configuration: using 'rbd' pools for backing 'volume'
> disks isn't yet supported
> ```

Note that the error is saying that the conversion function which converts a <disk type='volume' to the appropriate config isn't implemented. This is true even for current upstream as nobody ever contributed this function. You can still use <disk type='network'><source protocol='rbd'. This bug was about enabling the storage pool support. Arguably this by itself isn't very useful without the above convertor.
Comment 8 Yuri Arabadji 2021-07-30 17:30:08 UTC 
Thanks for clarifying, Peter. That's exactly what I was confused about.
Note You need to log in before you can comment on or make changes to this bug.