Bug 1539837 - [RFE][tracker][Cinder] - Improve integration and support with storage using cinderlib
Summary: [RFE][tracker][Cinder] - Improve integration and support with storage using c...
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: RFEs
Version: 4.3.0
Hardware: Unspecified
OS: Unspecified
high with 3 votes
Target Milestone: ovirt-4.5.0
: ---
Assignee: Stephen Gordon
QA Contact: Avihai
Depends On: 1113414 1155275 1172354 1296880 1343676 1477496 1516330 1532546 1664588 1664740 1664742 1664748 1665922 1667002 1668722 1668727 1668749 1669042 1669047 1669830 1669978 1669997 1672065 1684889 1686280 1686538 1689884
Blocks: 717319 798947 928820 1258911 1331544 1364869 1647196
TreeView+ depends on / blocked
Reported: 2018-01-29 17:32 UTC by Yaniv Lavi
Modified: 2022-04-24 19:44 UTC (History)
12 users (show)

Fixed In Version:
Doc Type: Enhancement
Doc Text:
Clone Of:
Last Closed: 2022-04-20 11:54:49 UTC
oVirt Team: Storage
pm-rhel: ovirt-4.5?
ylavi: planning_ack+
ylavi: devel_ack?
ylavi: testing_ack?

Attachments (Terms of Use)

System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1025400 0 unspecified CLOSED [RFE] Allow template export/import through Glance Storage Domain 2021-02-22 00:41:40 UTC
Red Hat Bugzilla 1029489 0 unspecified CLOSED [RFE][oVirt] Export not exporting direct lun disk 2021-02-22 00:41:40 UTC
Red Hat Bugzilla 1069557 0 high CLOSED [RFE] disk clone: use xcopy if available (depends on platform BZ for qemu-img to support xcopy - bug 1482537) 2021-02-22 00:41:40 UTC
Red Hat Bugzilla 1082540 0 medium CLOSED [RFE] If someone is using direct LUN(as a boot disk) then RHEV should allow to start vm without adding storage domain 2021-02-22 00:41:40 UTC
Red Hat Bugzilla 1095666 0 low CLOSED [RFE] Support read only VirtIO-SCSI direct LUNs in oVirt 2021-02-22 00:41:40 UTC
Red Hat Bugzilla 1113414 0 medium CLOSED [RFE] - Manage floating\shared disks with snapshots 2021-06-10 10:49:37 UTC
Red Hat Bugzilla 1126945 0 high CLOSED [RFE][Cinder] - Add multipath for LUNs 2021-02-22 00:41:40 UTC
Red Hat Bugzilla 1155275 0 high CLOSED [RFE] - Online update LUN size to the Guest after LUN resize 2023-03-24 13:30:08 UTC
oVirt gerrit 96991 0 master MERGED storage: Add database for managed volumes 2021-02-05 13:46:11 UTC
oVirt gerrit 97113 0 master MERGED managedvolume: support RBD local attachment 2021-02-05 13:46:11 UTC

Description Yaniv Lavi 2018-01-29 17:32:04 UTC
Description of problem:
Improve integration and support with the cinder service external provider. This should include:

- Providing Cinder standalone as a service.
- Support FC/ISCSI storage backends.
- Support Ceph. 
- Support Cinder volume with multipath support on the hypervisor.
- Support triggering vDisk live snapshots from engine, even if not managing them.
- Support migration from storage domains vDisk to Cinder based volume.
- Support migration between Cinder backends (retyping).
- Support ManageIQ cloud storage provider for oVirt infrastructure provider.
- Support snapshot tree representation and consistency groups management in ManageIQ.
- Support wipe after delete.
- Support downloading/uploading OVAs with Cinder volumes.

In different to the existing integration, we should treat Cinder volumes as externally managed objects that require minimal integration with engine for ease of use.

Comment 2 Yaniv Lavi 2018-05-16 10:52:22 UTC
Additional requirements to scope: 
- Download/upload volumes from Cinder.
- Download/upload OVA with Cinder volumes.

Comment 3 Yaniv Lavi 2018-05-16 11:10:31 UTC
Additional requirements to scope: 
- Using Cinder volumes without needing storage domain and SPM in the DC.
- Online update of Cinder volume re-size for the VM after external LUN resize via Cinder.
- Display and manage unattached/attached Cinder volumes with snapshots (UI plugin?).
- Host assisted Cinder volume retype (cold/live).
- Multipath for Cinder volumes.
- SSL support for Cinder external provider.
- Allow read only Cinder volumes.

Comment 5 Sandro Bonazzola 2019-01-28 09:43:52 UTC
This bug has not been marked as blocker for oVirt 4.3.0.
Since we are releasing it tomorrow, January 29th, this bug has been re-targeted to 4.3.1.

Comment 6 Konstantin Shalygin 2019-02-19 04:18:08 UTC
Sandro, Yaniv

Please, keep in mind that before drop current cinder integration we need some time and some way to migrate current Ceph integrations.
On the whole world not a couple of clusters like this. I know clusters with hundreds Tbytes and VMs, my cluster is 0.5Pb, 210 librbd clients (oVirt disks), 110 VMs, 1 Storage Providers with 4 auth keys (Cinder volume types aka Ceph pools).


Comment 7 Logan Kuhn 2019-02-19 13:12:45 UTC
I second what Konstantin said.  We have a similar sized deployment, 530T, 290 libvirt clients, over 700 RBDs and we really need a viable upgrade path from 4.2 with cinder as an external provider to 4.3 with cinderlib.

Comment 8 rumanzo 2019-02-20 09:11:50 UTC
I am glad to see that there is a movement towards simplifying the use of such storage facilities as ceph.
As already mentioned, it would be good to work out the migration procedure from the solution ovirt + cinder.
Regarding item "Support migration from storage domains vDisk to Cinder based volume." - it's great, but still it would be nice to work out the creation of virtual machines from openstack glance, now ovirt can create vm from glance image to hosted_storage only, and not support external providers.
And I know that this is not included in this thread, but is it possible to initiate the implementation of a hosted_engine installation on a ceph rbd/cephfs?

Comment 10 Michal Skrivanek 2020-06-23 12:34:46 UTC
This request is not currently committed to 4.4.z, moving it to 4.5

Comment 11 Konstantin Shalygin 2020-12-07 11:48:05 UTC
I look to the Cinderlib integration oVirt doc, provided by Michal Skrivanek [1]

Don't understand the moment, why oVirt hosts need ceph-common package? ceph-common is meta package and will install userland utilites, such as rbd, rados. Actually hosts need only libvirt-daemon-driver-storage-rbd package for rbd support in qemu.
Also I don't see how-to configure ceph monitors array. Current OpenStack Provider Cinder code get's all disk-related configuration from Cinder - host have zero configuration for this...

Example how-to cinder.conf backends looks like:

volume_driver = cinder.volume.drivers.rbd.RBDDriver               
volume_backend_name = replicated-rbd                              
rbd_pool = replicated_rbd                                         
rbd_ceph_conf = /etc/ceph/ceph.conf                               
rbd_flatten_volume_from_snapshot = false                          
rbd_max_clone_depth = 5                                           
rbd_store_chunk_size = 4                                          
rados_connect_timeout = -1                                        
glance_api_version = 2                                            
rbd_user = cinder                                                 
rbd_secret_uuid = e0828f39-2832-4d82-90ee-23b26fc7b20a            
report_discard_supported = true                                   

[1] https://www.ovirt.org/develop/release-management/features/storage/cinderlib-integration.html

Comment 17 Sandro Bonazzola 2022-03-29 16:16:40 UTC
We are past 4.5.0 feature freeze, please re-target.

Comment 18 Michal Skrivanek 2022-04-20 11:54:49 UTC
In 4.5 we concluded the improvements. Not all are implemented and the feature has to stay at Tech Preview, but it is more or less stable and working ok. Closing a tracker as no fundamental changes are planned.

(slightly) more up-to-date information about usage are at https://blogs.ovirt.org/2021/07/using-ceph-only-storage-for-ovirt-datacenter/

Note You need to log in before you can comment on or make changes to this bug.