Bug 1647041

Summary:	[RFE] Support for sparse image upload to ceph
Product:	Red Hat OpenStack	Reporter:	Cody Swanson <cswanson>
Component:	python-glance-store	Assignee:	Cyril Roelandt <cyril>
Status:	CLOSED ERRATA	QA Contact:	Mike Abrams <mabrams>
Severity:	high	Docs Contact:	Kim Nylander <knylande>
Priority:	high
Version:	10.0 (Newton)	CC:	akaris, akekane, apevec, cyril, eglynn, gcharot, gfidente, lhh, mgarciac, ndeevy, nlevinki, rurena, senrique, spower
Target Milestone:	Alpha	Keywords:	FeatureBackport, Triaged
Target Release:	16.2 (Train on RHEL 8.4)
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:	python-glance-store-1.0.2-2.20210205164924.bc62bb4.el8 openstack-tripleo-heat-templates-11.4.1-2.20210208205916.dfc4da5.el8 puppet-tripleo-11.5.1-2.20210209011041.0d42695.el8 puppet-glance-15.4.1-2.20210208204921.aa0e036.el8	Doc Type:	Enhancement
Doc Text:		Story Points:	---
Clone Of:
Clones:	1919160 (view as bug list)		Environment:
Last Closed:	2021-09-15 07:07:46 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1802774, 1389112, 1433388, 1476900, 1866741, 1919160

Description Cody Swanson 2018-11-06 15:13:03 UTC

Description of problem:
Images uploaded to ceph via the glance command line utility do not maintain image file sparseness.

Version-Release number of selected component (if applicable):
All current versions.

How reproducible:
Every time.

Steps to Reproduce:
1. Upload sparse raw image to ceph backed glance images via command line. 


Actual results:
Image is uploaded but file sparseness is not maintained.

Expected results:
Image sparseness maintained thus reducing ceph storage overhead.

Additional info:
I suspect this issue is not limited to just ceph backed glance images but all images uploaded with the glance utility. We've found the following two previous bugs requesting this feature.

- https://bugzilla.redhat.com/show_bug.cgi?id=1373643
- http://tracker.ceph.com/issues/17178

Additionally there is a workaround published on the ceph website that shows how to put sparse images into ceph and then add them into glance manually. 

- https://ceph.com/planet/importing-an-existing-ceph-rbd-image-into-glance/

Comment 1 Cyril Roelandt 2018-11-08 14:11:09 UTC

So, do you know whether this feature is available in librbd?

Comment 2 Cody Swanson 2018-11-08 22:04:19 UTC

Hi Cyril,

I tried uploading a sparse raw image to ceph in my lab using qemu-img as outlined in the ceph.com article linked above. The image appears to be uploaded to ceph via librdb and appears to be sparsely stored in ceph using that proceedure so my assumption is that librbd does support sparse file upload. 

[root@controller-0 ~]# ldd /bin/qemu-img  |grep librbd
	librbd.so.1 => /lib64/librbd.so.1 (0x00007f82e099e000)

[root@controller-0 ~]# rpm -qa|grep glance
puppet-glance-9.6.0-3.el7ost.noarch
python-glance-store-0.18.0-3.el7ost.noarch
openstack-glance-13.0.0-2.el7ost.noarch
python-glanceclient-2.5.0-2.el7ost.noarch
python-glance-13.0.0-2.el7ost.noarch

[root@controller-0 ~]# rpm -qa|grep ceph
ceph-mon-10.2.10-17.el7cp.x86_64
ceph-base-10.2.10-17.el7cp.x86_64
python-cephfs-10.2.10-17.el7cp.x86_64
puppet-ceph-2.4.2-1.el7ost.noarch
ceph-common-10.2.10-17.el7cp.x86_64
libcephfs1-10.2.10-17.el7cp.x86_64
ceph-radosgw-10.2.10-17.el7cp.x86_64
ceph-selinux-10.2.10-17.el7cp.x86_64
ceph-osd-10.2.10-17.el7cp.x86_64

Comment 4 Cyril Roelandt 2019-01-15 22:45:00 UTC

So apparently, this RFE was denied upstream (see https://bugs.launchpad.net/glance-store/+bug/810493). I'm afraid there is not much interest in this feature, as pointed out by Erno on IRC :/

Comment 7 Cyril Roelandt 2019-08-05 14:32:22 UTC

I'm adding Greg to this BZ.

@Greg: do you think this should be an RFE for OSP(>16)?

Comment 8 Gregory Charot 2019-08-05 15:43:47 UTC

Yes that's something we want to review post OSP16

Comment 9 Rafael Urena 2020-07-13 13:09:41 UTC

Is There any information I can pass on to the customer?

Rafael Ureña
Technical Account Manager

Comment 10 Abhishek Kekane 2020-07-13 15:31:21 UTC

(In reply to Rafael Urena from comment #9)
> Is There any information I can pass on to the customer?
> 
> Rafael Ureña
> Technical Account Manager

We are going to implement this in upstream Victoria cycle.
A upstream design specification is available and under review, https://review.opendev.org/733157

Comment 12 Gregory Charot 2020-11-12 11:12:23 UTC

*** Bug 1389529 has been marked as a duplicate of this bug. ***

Comment 13 Cyril Roelandt 2021-02-09 03:04:25 UTC

Actually, the RPMs have been created:

python-glance-store-1.0.2-2.20210205164924.bc62bb4.el8
openstack-tripleo-heat-templates-11.4.1-2.20210208205916.dfc4da5.el8
puppet-tripleo-11.5.1-2.20210209011041.0d42695.el8
puppet-glance-15.4.1-2.20210208204921.aa0e036.el8


I'm updating the "Fixed-in" version and moving this to MODIFIED.

Comment 17 ndeevy 2021-04-12 15:06:23 UTC

Doc work for this (and 1866741) being tracked in JIRA: https://issues.redhat.com/browse/RHOSPDOC-481

doc_ack'ed

Comment 19 Cyril Roelandt 2021-07-27 15:03:01 UTC

I filled the doc text for the related ooo bug https://bugzilla.redhat.com/show_bug.cgi?id=1866741 . I'm not sure whether it is really useful to also fill it here.

Comment 22 errata-xmlrpc 2021-09-15 07:07:46 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat OpenStack Platform (RHOSP) 16.2 enhancement advisory), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2021:3483