Bug 1187533
| Summary: | [RHEL 7 FEAT] libvirt is missing ceph support (rbd) | |||
|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Alexandros Gkesos <agkesos> | |
| Component: | libvirt | Assignee: | Peter Krempa <pkrempa> | |
| Status: | CLOSED ERRATA | QA Contact: | Virtualization Bugs <virt-bugs> | |
| Severity: | high | Docs Contact: | ||
| Priority: | high | |||
| Version: | 7.0 | CC: | bgilbert, dyuan, lmiksik, mzhan, ppostler, rbalakri, shyu, xuzhang, yanyang, yuri | |
| Target Milestone: | rc | Keywords: | FutureFeature | |
| Target Release: | --- | |||
| Hardware: | All | |||
| OS: | Linux | |||
| Whiteboard: | ||||
| Fixed In Version: | libvirt-1.2.14-1.el7 | Doc Type: | Enhancement | |
| Doc Text: |
Feature:
Enable support for the ceph storage driver backend so that ceph storage pools can be accessed via libvirt APIs.
(note: VMs with ceph disks were usable even without this)
Reason:
Libvirt already had the code to deal with ceph volumes but the support was not compiled in.
Result:
The storage driver works with ceph(rbd) storage pools. Users are able to use libvirt APIs as with other storage driver backends.
|
Story Points: | --- | |
| Clone Of: | ||||
| : | 1251599 (view as bug list) | Environment: | ||
| Last Closed: | 2015-11-19 06:09:25 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | ||||
| Bug Blocks: | 1205796, 1250958, 1251599 | |||
Verified on libvirt-1.2.15-2.el7.x86_64
Steps
=================pool related virsh command==================
1. define/build/start rbd pool with following xml
# cat rbd-pool.xml
<pool type="rbd">
<name>rbd</name>
<source>
<name>libvirt-pool</name>
<host name='10.66.5.219' port='6789'/>
<host name='osd2.redhat.com' port='6789'/>
<host name='osd3.redhat.com' port='6789'/>
<auth type='ceph' username='libvirt'>
<secret usage='client.libvirt secret'/>
</auth>
</source>
</pool>
# virsh pool-define rbd-pool.xml
Pool rbd defined from rbd-pool.xml
[root@rhel7_test yy]# virsh pool-build rbd
Pool rbd built
[root@rhel7_test yy]# virsh pool-start rbd
Pool rbd started
[root@rhel7_test yy]# virsh vol-list rbd
Name Path
------------------------------------------------------------------------------
yy1.img libvirt-pool/yy1.img
2. refresh rbd pool
[root@rhel7_test yy]# virsh pool-refresh rbd
Pool rbd refreshed
3.display rbd pool
[root@rhel7_test yy]# virsh pool-list --type rbd
Name State Autostart
-------------------------------------------
rbd active no
4.check rbd pool xml
[root@rhel7_test yy]# virsh pool-dumpxml rbd
<pool type='rbd'>
<name>rbd</name>
<uuid>1e6388ad-a8ca-4363-8a9d-68bb81b9735c</uuid>
<capacity unit='bytes'>139701780480</capacity>
<allocation unit='bytes'>135</allocation>
<available unit='bytes'>99701280768</available>
<source>
<host name='10.66.5.219' port='6789'/>
<host name='osd2.redhat.com' port='6789'/>
<host name='osd3.redhat.com' port='6789'/>
<name>libvirt-pool</name>
<auth type='ceph' username='libvirt'>
<secret usage='client.libvirt secret'/>
</auth>
</source>
</pool>
5.check pool-info, pool-uuid, pool-name
[root@rhel7_test yy]# virsh pool-info rbd
Name: rbd
UUID: 1e6388ad-a8ca-4363-8a9d-68bb81b9735c
State: running
Persistent: yes
Autostart: no
Capacity: 130.11 GiB
Allocation: 135.00 B
Available: 92.85 GiB
[root@rhel7_test yy]# virsh pool-uuid rbd
1e6388ad-a8ca-4363-8a9d-68bb81b9735c
[root@rhel7_test yy]# virsh pool-name 1e6388ad-a8ca-4363-8a9d-68bb81b9735c
rbd
6. check pool-autostart
[root@rhel7_test yy]# virsh pool-autostart rbd
Pool rbd marked as autostarted
[root@rhel7_test yy]# virsh pool-list rbd
Name State Autostart
-------------------------------------------
rbd active yes
[root@rhel7_test yy]# virsh pool-autostart rbd --disable
Pool rbd unmarked as autostarted
[root@rhel7_test yy]# virsh pool-list rbd
Name State Autostart
-------------------------------------------
rbd active no
7. check find-storage-pool-sources-as
[root@rhel7_test yy]# virsh find-storage-pool-sources-as rbd 10.66.5.219
error: Failed to find any rbd pool sources
error: this function is not supported by the connection driver: pool type 'rbd' does not support source discovery
8. edit pool
# virsh pool-edit rbd
Pool rbd XML configuration edited.
9. check pool-define-as
# virsh pool-define-as rbd-tmp rbd 10.66.4.150 --source-name libvirt-pool --auth-type ceph --auth-username libvirt --secret-usage client.libvirt\ secret
Pool rbd-tmp defined
# virsh pool-dumpxml rbd-tmp
<pool type='rbd'>
<name>rbd-tmp</name>
<uuid>4951d0b9-1122-4713-87b2-17a021dd1fba</uuid>
<capacity unit='bytes'>139701780480</capacity>
<allocation unit='bytes'>151</allocation>
<available unit='bytes'>99698761728</available>
<source>
<host name='10.66.4.150'/>
<name>libvirt-pool</name>
<auth type='ceph' username='libvirt'>
<secret usage='client.libvirt secret'/>
</auth>
</source>
</pool>
# virsh pool-start rbd-tmp
Pool rbd-tmp started
[root@rhel7_test yy]# virsh vol-list rbd-tmp
Name Path
------------------------------------------------------------------------------
yy1.img libvirt-pool/yy1.img
10. check pool-create-as
# virsh pool-create-as rbd-tmp rbd 10.66.4.150 --source-name libvirt-pool --auth-type ceph --auth-username libvirt --secret-usage client.libvirt\ secret
Pool rbd-tmp created
[root@rhel7_test yy]# virsh pool-list --all
Name State Autostart
-------------------------------------------
default active yes
disk inactive no
fs inactive no
HostVG inactive no
iscsi inactive no
rbd active no
rbd-tmp active no
rbd-tmp-1 active no
# virsh pool-destroy rbd-tmp-1
Pool rbd-tmp-1 destroyed
[root@rhel7_test yy]# virsh pool-list --type rbd
Name State Autostart
-------------------------------------------
rbd active no
rbd-tmp active no
11. check pool-destroy, pool-undefine
# virsh pool-destroy rbd-tmp
Pool rbd-tmp destroyed
[root@rhel7_test yy]# virsh pool-undefine rbd-tmp
Pool rbd-tmp has been undefined
[root@rhel7_test yy]# virsh pool-list --type rbd
Name State Autostart
-------------------------------------------
rbd active no
12. check pool-delete
# virsh pool-delete rbd
error: Failed to delete pool rbd
error: this function is not supported by the connection driver: pool does not support pool deletion
==========================vol related virsh command==================
2. check vol-create, vol-list
# cat rbd.vol
<volume type='network'>
<name>yy1.img</name>
<source>
</source>
<capacity unit='bytes'>5368709120</capacity>
<allocation unit='bytes'>5368709120</allocation>
<target>
<format type='unknown'/>
</target>
</volume>
# virsh vol-create rbd rbd.vol
Vol yy1.img created from rbd.vol
[root@rhel7_test yy]# virsh vol-list rbd
Name Path
------------------------------------------------------------------------------
yy1.img libvirt-pool/yy1.img
3.check vol-dumpxml, vol-info
# virsh vol-dumpxml yy1.img rbd
<volume type='network'>
<name>yy1.img</name>
<key>libvirt-pool/yy1.img</key>
<source>
</source>
<capacity unit='bytes'>5368709120</capacity>
<allocation unit='bytes'>5368709120</allocation>
<target>
<path>libvirt-pool/yy1.img</path>
<format type='unknown'/>
</target>
</volume>
# virsh vol-info libvirt-pool/yy1.img
Name: yy1.img
Type: network
Capacity: 5.00 GiB
Allocation: 5.00 GiB
4.check vol-key, vol-path, vol-pool, vol-name
# virsh vol-key yy1.img rbd
libvirt-pool/yy1.img
[root@rhel7_test yy]# virsh vol-path yy1.img rbd
libvirt-pool/yy1.img
[root@rhel7_test yy]# virsh vol-pool libvirt-pool/yy1.img
rbd
[root@rhel7_test yy]# virsh vol-name libvirt-pool/yy1.img
yy1.img
5. check vol-create-as
# virsh vol-create-as rbd yy2.img 1G --format raw
Vol yy2.img created
[root@rhel7_test yy]# virsh vol-list rbd
Name Path
------------------------------------------------------------------------------
yy1.img libvirt-pool/yy1.img
yy2.img libvirt-pool/yy2.img
[root@rhel7_test yy]# virsh vol-dumpxml libvirt-pool/yy2.img
<volume type='network'>
<name>yy2.img</name>
<key>libvirt-pool/yy2.img</key>
<source>
</source>
<capacity unit='bytes'>1073741824</capacity>
<allocation unit='bytes'>1073741824</allocation>
<target>
<path>libvirt-pool/yy2.img</path>
<format type='unknown'/>
<permissions>
<mode>0600</mode>
<owner>-1</owner>
<group>-1</group>
</permissions>
</target>
</volume>
6. check vol-resize
# virsh vol-resize yy2.img 2G rbd
Size of volume 'yy2.img' successfully changed to 2G
[root@rhel7_test yy]# virsh vol-info yy2.img rbd
Name: yy2.img
Type: network
Capacity: 2.00 GiB
Allocation: 2.00 GiB
7. check vol-clone
# virsh vol-clone yy2.img yy3.img rbd
error: Failed to clone vol from yy2.img
error: this function is not supported by the connection driver: storage pool does not support volume creation from an existing volume
8. check vol-create-from
# virsh vol-create-from rbd rbd.vol yy2.img rbd
error: Failed to create vol from rbd.vol
error: this function is not supported by the connection driver: storage pool does not support volume creation from an existing volume
# cat rbd.vol
<volume type='network'>
<name>yy3.img</name>
<source>
</source>
<capacity unit='bytes'>2147483648</capacity>
<allocation unit='bytes'>2147483648</allocation>
<target>
<format type='unknown'/>
<permissions>
<mode>0600</mode>
<owner>-1</owner>
<group>-1</group>
</permissions>
</target>
</volume>
# virsh vol-create rbd rbd.vol
Vol yy3.img created from rbd.vol
# virsh vol-delete yy3.img rbd
Vol yy3.img deleted
9. check vol-create-as
# virsh vol-create-as rbd yy3.img --capacity 2G --allocation 1G --format qcow2
Vol yy3.img created
# virsh vol-info yy3.img rbd
Name: yy3.img
Type: network
Capacity: 2.00 GiB
Allocation: 2.00 GiB
[root@rhel7_test yy]# virsh vol-dumpxml yy3.img rbd
<volume type='network'>
<name>yy3.img</name>
<key>libvirt-pool/yy3.img</key>
<source>
</source>
<capacity unit='bytes'>2147483648</capacity>
<allocation unit='bytes'>2147483648</allocation>
<target>
<path>libvirt-pool/yy3.img</path>
<format type='unknown'/>
<permissions>
<mode>0600</mode>
<owner>-1</owner>
<group>-1</group>
</permissions>
</target>
</volume>
10. check vol-upload, vol-download
# virsh vol-upload --pool rbd --vol yy2.img rbd.vol
error: cannot upload to volume yy2.img
error: this function is not supported by the connection driver: storage pool doesn't support volume upload
# virsh vol-download --pool rbd --vol yy2.img rbd.vol
error: cannot download from volume yy2.img
error: this function is not supported by the connection driver: storage pool doesn't support volume download
11. check vol-wipe
# virsh vol-wipe yy3.img rbd
error: Failed to wipe vol yy3.img
error: this function is not supported by the connection driver: storage pool doesn't support volume wiping
12. use rbd volume as virtual disk
<disk type='volume' device='disk'>
<driver name='qemu' type='raw'/>
<source pool='rbd' volume='yy1.img'/>
<target dev='vda' bus='virtio'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
</disk>
# virsh start vm5
error: Failed to start domain vm5
error: unsupported configuration: using 'rbd' pools for backing 'volume' disks isn't yet supported
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2015-2202.html What have you verified, darling? You've confirmed it DOESN'T work. Why was this bug closed if the verification confirmed the defect? ``` # virsh start vm5 error: Failed to start domain vm5 error: unsupported configuration: using 'rbd' pools for backing 'volume' disks isn't yet supported ``` Same happens over here on EL7: libvirt-4.5.0-36.el7_9.5.x86_64 (In reply to Yuri Arabadji from comment #6) > What have you verified, darling? You've confirmed it DOESN'T work. Why was > this bug closed if the verification confirmed the defect? > > ``` > # virsh start vm5 > error: Failed to start domain vm5 > error: unsupported configuration: using 'rbd' pools for backing 'volume' > disks isn't yet supported > ``` Note that the error is saying that the conversion function which converts a <disk type='volume' to the appropriate config isn't implemented. This is true even for current upstream as nobody ever contributed this function. You can still use <disk type='network'><source protocol='rbd'. This bug was about enabling the storage pool support. Arguably this by itself isn't very useful without the above convertor. Thanks for clarifying, Peter. That's exactly what I was confused about. |
Support enabled in upstream spec file: commit 6cfc8834c858849cc74c3082078dc91fb1cbae38 Author: Peter Krempa <pkrempa> Date: Thu Mar 5 11:40:54 2015 +0100 spec: Enable RBD storage driver in RHEL-7 Use correct package names too as they differ. v1.2.13-62-g6cfc883