Bug 1647041 - [RFE] Support for sparse image upload to ceph
Summary: [RFE] Support for sparse image upload to ceph
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: python-glance-store
Version: 10.0 (Newton)
Hardware: All
OS: Linux
high
high
Target Milestone: Alpha
: 16.2 (Train on RHEL 8.4)
Assignee: Cyril Roelandt
QA Contact: Mike Abrams
Kim Nylander
URL:
Whiteboard:
: 1389529 (view as bug list)
Depends On:
Blocks: 1802774 1389112 ciscoosp13features, ciscoosp13rfe 1476900 1866741 1919160
TreeView+ depends on / blocked
 
Reported: 2018-11-06 15:13 UTC by Cody Swanson
Modified: 2021-09-15 07:08 UTC (History)
14 users (show)

Fixed In Version: python-glance-store-1.0.2-2.20210205164924.bc62bb4.el8 openstack-tripleo-heat-templates-11.4.1-2.20210208205916.dfc4da5.el8 puppet-tripleo-11.5.1-2.20210209011041.0d42695.el8 puppet-glance-15.4.1-2.20210208204921.aa0e036.el8
Doc Type: Enhancement
Doc Text:
Clone Of:
: 1919160 (view as bug list)
Environment:
Last Closed: 2021-09-15 07:07:46 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 810493 0 None None None 2019-01-15 22:44:59 UTC
OpenStack gerrit 733157 0 None MERGED Handle sparse images in glance_store 2021-02-16 13:49:19 UTC
OpenStack gerrit 744282 0 None MERGED Handle sparse images in glance_store 2021-02-16 13:49:19 UTC
Red Hat Product Errata RHEA-2021:3483 0 None None None 2021-09-15 07:08:28 UTC

Description Cody Swanson 2018-11-06 15:13:03 UTC
Description of problem:
Images uploaded to ceph via the glance command line utility do not maintain image file sparseness.

Version-Release number of selected component (if applicable):
All current versions.

How reproducible:
Every time.

Steps to Reproduce:
1. Upload sparse raw image to ceph backed glance images via command line. 


Actual results:
Image is uploaded but file sparseness is not maintained.

Expected results:
Image sparseness maintained thus reducing ceph storage overhead.

Additional info:
I suspect this issue is not limited to just ceph backed glance images but all images uploaded with the glance utility. We've found the following two previous bugs requesting this feature.

- https://bugzilla.redhat.com/show_bug.cgi?id=1373643
- http://tracker.ceph.com/issues/17178

Additionally there is a workaround published on the ceph website that shows how to put sparse images into ceph and then add them into glance manually. 

- https://ceph.com/planet/importing-an-existing-ceph-rbd-image-into-glance/

Comment 1 Cyril Roelandt 2018-11-08 14:11:09 UTC
So, do you know whether this feature is available in librbd?

Comment 2 Cody Swanson 2018-11-08 22:04:19 UTC
Hi Cyril,

I tried uploading a sparse raw image to ceph in my lab using qemu-img as outlined in the ceph.com article linked above. The image appears to be uploaded to ceph via librdb and appears to be sparsely stored in ceph using that proceedure so my assumption is that librbd does support sparse file upload. 

[root@controller-0 ~]# ldd /bin/qemu-img  |grep librbd
	librbd.so.1 => /lib64/librbd.so.1 (0x00007f82e099e000)

[root@controller-0 ~]# rpm -qa|grep glance
puppet-glance-9.6.0-3.el7ost.noarch
python-glance-store-0.18.0-3.el7ost.noarch
openstack-glance-13.0.0-2.el7ost.noarch
python-glanceclient-2.5.0-2.el7ost.noarch
python-glance-13.0.0-2.el7ost.noarch

[root@controller-0 ~]# rpm -qa|grep ceph
ceph-mon-10.2.10-17.el7cp.x86_64
ceph-base-10.2.10-17.el7cp.x86_64
python-cephfs-10.2.10-17.el7cp.x86_64
puppet-ceph-2.4.2-1.el7ost.noarch
ceph-common-10.2.10-17.el7cp.x86_64
libcephfs1-10.2.10-17.el7cp.x86_64
ceph-radosgw-10.2.10-17.el7cp.x86_64
ceph-selinux-10.2.10-17.el7cp.x86_64
ceph-osd-10.2.10-17.el7cp.x86_64

Comment 4 Cyril Roelandt 2019-01-15 22:45:00 UTC
So apparently, this RFE was denied upstream (see https://bugs.launchpad.net/glance-store/+bug/810493). I'm afraid there is not much interest in this feature, as pointed out by Erno on IRC :/

Comment 7 Cyril Roelandt 2019-08-05 14:32:22 UTC
I'm adding Greg to this BZ.

@Greg: do you think this should be an RFE for OSP(>16)?

Comment 8 Gregory Charot 2019-08-05 15:43:47 UTC
Yes that's something we want to review post OSP16

Comment 9 Rafael Urena 2020-07-13 13:09:41 UTC
Is There any information I can pass on to the customer?

Rafael Ureña
Technical Account Manager

Comment 10 Abhishek Kekane 2020-07-13 15:31:21 UTC
(In reply to Rafael Urena from comment #9)
> Is There any information I can pass on to the customer?
> 
> Rafael Ureña
> Technical Account Manager

We are going to implement this in upstream Victoria cycle.
A upstream design specification is available and under review, https://review.opendev.org/733157

Comment 12 Gregory Charot 2020-11-12 11:12:23 UTC
*** Bug 1389529 has been marked as a duplicate of this bug. ***

Comment 13 Cyril Roelandt 2021-02-09 03:04:25 UTC
Actually, the RPMs have been created:

python-glance-store-1.0.2-2.20210205164924.bc62bb4.el8
openstack-tripleo-heat-templates-11.4.1-2.20210208205916.dfc4da5.el8
puppet-tripleo-11.5.1-2.20210209011041.0d42695.el8
puppet-glance-15.4.1-2.20210208204921.aa0e036.el8


I'm updating the "Fixed-in" version and moving this to MODIFIED.

Comment 17 ndeevy 2021-04-12 15:06:23 UTC
Doc work for this (and 1866741) being tracked in JIRA: https://issues.redhat.com/browse/RHOSPDOC-481

doc_ack'ed

Comment 19 Cyril Roelandt 2021-07-27 15:03:01 UTC
I filled the doc text for the related ooo bug https://bugzilla.redhat.com/show_bug.cgi?id=1866741 . I'm not sure whether it is really useful to also fill it here.

Comment 22 errata-xmlrpc 2021-09-15 07:07:46 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat OpenStack Platform (RHOSP) 16.2 enhancement advisory), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2021:3483


Note You need to log in before you can comment on or make changes to this bug.