Bug 1539837

Summary: [RFE][tracker][Cinder] - Improve integration and support with storage using cinderlib
Product: [oVirt] ovirt-engine Reporter: Yaniv Lavi <ylavi>
Component: RFEsAssignee: Stephen Gordon <sgordon>
Status: CLOSED CURRENTRELEASE QA Contact: Avihai <aefrat>
Severity: high Docs Contact:
Priority: high    
Version: 4.3.0CC: ahadas, bugs, ebenahar, eric, jcall, jwang, logank, michal.skrivanek, mkalinin, rumanzo, shalygin.k, tnisan
Target Milestone: ovirt-4.5.0Keywords: FutureFeature, TechPreview, Tracking
Target Release: ---Flags: pm-rhel: ovirt-4.5?
ylavi: planning_ack+
ylavi: devel_ack?
ylavi: testing_ack?
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-04-20 11:54:49 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Storage RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1113414, 1155275, 1172354, 1296880, 1343676, 1477496, 1516330, 1532546, 1664588, 1664740, 1664742, 1664748, 1665922, 1667002, 1668722, 1668727, 1668749, 1669042, 1669047, 1669830, 1669978, 1669997, 1672065, 1684889, 1686280, 1686538, 1689884    
Bug Blocks: 717319, 798947, 928820, 1258911, 1331544, 1364869, 1647196    

Description Yaniv Lavi 2018-01-29 17:32:04 UTC
Description of problem:
Improve integration and support with the cinder service external provider. This should include:

- Providing Cinder standalone as a service.
- Support FC/ISCSI storage backends.
- Support Ceph. 
- Support Cinder volume with multipath support on the hypervisor.
- Support triggering vDisk live snapshots from engine, even if not managing them.
- Support migration from storage domains vDisk to Cinder based volume.
- Support migration between Cinder backends (retyping).
- Support ManageIQ cloud storage provider for oVirt infrastructure provider.
- Support snapshot tree representation and consistency groups management in ManageIQ.
- Support wipe after delete.
- Support downloading/uploading OVAs with Cinder volumes.

In different to the existing integration, we should treat Cinder volumes as externally managed objects that require minimal integration with engine for ease of use.

Comment 2 Yaniv Lavi 2018-05-16 10:52:22 UTC
Additional requirements to scope: 
- Download/upload volumes from Cinder.
- Download/upload OVA with Cinder volumes.

Comment 3 Yaniv Lavi 2018-05-16 11:10:31 UTC
Additional requirements to scope: 
- Using Cinder volumes without needing storage domain and SPM in the DC.
- Online update of Cinder volume re-size for the VM after external LUN resize via Cinder.
- Display and manage unattached/attached Cinder volumes with snapshots (UI plugin?).
- Host assisted Cinder volume retype (cold/live).
- Multipath for Cinder volumes.
- SSL support for Cinder external provider.
- Allow read only Cinder volumes.
-

Comment 5 Sandro Bonazzola 2019-01-28 09:43:52 UTC
This bug has not been marked as blocker for oVirt 4.3.0.
Since we are releasing it tomorrow, January 29th, this bug has been re-targeted to 4.3.1.

Comment 6 Konstantin Shalygin 2019-02-19 04:18:08 UTC
Sandro, Yaniv

Please, keep in mind that before drop current cinder integration we need some time and some way to migrate current Ceph integrations.
On the whole world not a couple of clusters like this. I know clusters with hundreds Tbytes and VMs, my cluster is 0.5Pb, 210 librbd clients (oVirt disks), 110 VMs, 1 Storage Providers with 4 auth keys (Cinder volume types aka Ceph pools).

Thanks.

Comment 7 Logan Kuhn 2019-02-19 13:12:45 UTC
I second what Konstantin said.  We have a similar sized deployment, 530T, 290 libvirt clients, over 700 RBDs and we really need a viable upgrade path from 4.2 with cinder as an external provider to 4.3 with cinderlib.

Comment 8 rumanzo 2019-02-20 09:11:50 UTC
I am glad to see that there is a movement towards simplifying the use of such storage facilities as ceph.
As already mentioned, it would be good to work out the migration procedure from the solution ovirt + cinder.
Regarding item "Support migration from storage domains vDisk to Cinder based volume." - it's great, but still it would be nice to work out the creation of virtual machines from openstack glance, now ovirt can create vm from glance image to hosted_storage only, and not support external providers.
And I know that this is not included in this thread, but is it possible to initiate the implementation of a hosted_engine installation on a ceph rbd/cephfs?

Comment 10 Michal Skrivanek 2020-06-23 12:34:46 UTC
This request is not currently committed to 4.4.z, moving it to 4.5

Comment 11 Konstantin Shalygin 2020-12-07 11:48:05 UTC
I look to the Cinderlib integration oVirt doc, provided by Michal Skrivanek [1]

Don't understand the moment, why oVirt hosts need ceph-common package? ceph-common is meta package and will install userland utilites, such as rbd, rados. Actually hosts need only libvirt-daemon-driver-storage-rbd package for rbd support in qemu.
Also I don't see how-to configure ceph monitors array. Current OpenStack Provider Cinder code get's all disk-related configuration from Cinder - host have zero configuration for this...

Example how-to cinder.conf backends looks like:

[replicated-rbd]                                                  
volume_driver = cinder.volume.drivers.rbd.RBDDriver               
volume_backend_name = replicated-rbd                              
rbd_pool = replicated_rbd                                         
rbd_ceph_conf = /etc/ceph/ceph.conf                               
rbd_flatten_volume_from_snapshot = false                          
rbd_max_clone_depth = 5                                           
rbd_store_chunk_size = 4                                          
rados_connect_timeout = -1                                        
glance_api_version = 2                                            
rbd_user = cinder                                                 
rbd_secret_uuid = e0828f39-2832-4d82-90ee-23b26fc7b20a            
report_discard_supported = true                                   


[1] https://www.ovirt.org/develop/release-management/features/storage/cinderlib-integration.html

Comment 17 Sandro Bonazzola 2022-03-29 16:16:40 UTC
We are past 4.5.0 feature freeze, please re-target.

Comment 18 Michal Skrivanek 2022-04-20 11:54:49 UTC
In 4.5 we concluded the improvements. Not all are implemented and the feature has to stay at Tech Preview, but it is more or less stable and working ok. Closing a tracker as no fundamental changes are planned.

(slightly) more up-to-date information about usage are at https://blogs.ovirt.org/2021/07/using-ceph-only-storage-for-ovirt-datacenter/