Bug 2011771

Summary:	[OSP 18] n-cpu uploads volume snapshot images with an empty string as data to Glance causing bogus hashes to be calculated
Product:	Red Hat OpenStack	Reporter:	Lee Yarwood <lyarwood>
Component:	openstack-nova	Assignee:	OSP DFG:Compute <osp-dfg-compute>
Status:	CLOSED MIGRATED	QA Contact:	OSP DFG:Compute <osp-dfg-compute>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	17.0 (Wallaby)	CC:	alifshit, astupnik, dasmith, eglynn, jhakimra, kchamart, sbauza, sgordon, vromanso
Target Milestone:	---	Keywords:	Triaged
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2024-01-10 20:31:15 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Lee Yarwood 2021-10-07 10:43:12 UTC

Description of problem:

As discussed in the following update to bug #1998184.

https://bugzilla.redhat.com/show_bug.cgi?id=1998184#c27

(In reply to Lee Yarwood from comment #27)
> I've finally worked out why we end up with the different behaviour between
> an ephemeral rbd and volume rbd snapshot when using Nova's imageCreate API.
> 
> When creating a volume backed snapshot imageCreate will set the size of the
> image to 0 in the image_meta [1]. This in turn activates an odd hackaround
> within Nova's image code that asks glance to create the image with an empty
> string for data, instead of None [2][3]. This causes the Glance backend to
> actually create an image on the backend, write data and compute the
> associated hash properties.
> 
> When creating an ephemeral snapshot that uses the underlying direct snapshot
> mechanism in Nova imageCreate doesn't do this, instead creating an image
> initially with an unset size [4][5] that results in nothing being created in
> Glance's backend. As we don't upload any further data to the image in this
> case [6] the hash properties are not recalculated and left empty.
> 
> I can try to create this initial image with a size of 0 in this flow but I'm
> worried that Glance will not recalculate the hashes later for other backends
> when we actually upload data, only one way to find out I guess.
> 
> FWIW I also this behaviour when shelving an ephemeral backed instance using
> rbd, with the call to create the image [7] not activating the image. Backup
> should also hit this but I haven't tested as yet [8].
> 
> [1]
> https://github.com/openstack/nova/blob/
> 1c502ebaec29615f08d4af7dc6680f3141d70e67/nova/compute/api.py#L3307-L3310
> [2]
> https://github.com/openstack/nova/blob/
> 1c502ebaec29615f08d4af7dc6680f3141d70e67/nova/image/glance.py#L531-L536
> [3]
> https://github.com/openstack/nova/blob/
> 1c502ebaec29615f08d4af7dc6680f3141d70e67/nova/image/glance.py#L632-L644
> [4]
> https://github.com/openstack/nova/blob/
> 1c502ebaec29615f08d4af7dc6680f3141d70e67/nova/compute/api.py#L3247-L3249
> [5]
> https://github.com/openstack/nova/blob/
> 1c502ebaec29615f08d4af7dc6680f3141d70e67/nova/compute/utils.py#L1203-L1233
> [6]
> https://github.com/openstack/nova/blob/
> 1c502ebaec29615f08d4af7dc6680f3141d70e67/nova/virt/libvirt/driver.py#L2932-
> L2933
> [7]
> https://github.com/openstack/nova/blob/
> 1c502ebaec29615f08d4af7dc6680f3141d70e67/nova/compute/api.py#L4285-L4287
> [8]
> https://github.com/openstack/nova/blob/
> 1c502ebaec29615f08d4af7dc6680f3141d70e67/nova/compute/api.py#L3218-L3220

IMHO we should stop uploading an empty string as data in the volume snapshot case and leave the hash values blank for both as no actual image data resides within Glance.

Version-Release number of selected component (if applicable):
Master

How reproducible:
Always

Steps to Reproduce:
1. Snapshot a volume backed instance using Nova's imageCreate API.
2. Inspect the image associated with the snapshot.

Actual results:
The image has os_hash_* properties defined

Expected results:
As no image data resides in Glance these properties should not be set.

Additional info:

Comment 1 Alex Stupnikov 2023-04-13 15:57:31 UTC

I may be wrong here, but after taking a look at https://github.com/openstack/nova/blob/1c502ebaec29615f08d4af7dc6680f3141d70e67/nova/image/glance.py#L632-L644 and trying few things in my lab, it looks like that this bug report is a bit inconsistent:

- glance create images in two steps: first there is a POST call that creates an image [1], then there is PUT call that uploads data [2]. Image is stuck in queued status until data is uploaded
- when it comes to PUT call [2], data is expected and AFAIU it can't really be none
- as a result, providing an empty string looks like the simplest solution to move image from queued state to active. And it is actually stated in https://github.com/openstack/nova/blob/1c502ebaec29615f08d4af7dc6680f3141d70e67/nova/image/glance.py#L632-L644

I tried to set data to None instead of empty string. As a result, data is not uploaded and image is stuck in queued state https://github.com/openstack/nova/blob/1c502ebaec29615f08d4af7dc6680f3141d70e67/nova/image/glance.py#L663-L664

Obviously this makes image unusable. So it looks like bugreport is a bit ambiguous or requires significant changes in Nova/Glance.

[1]
https://docs.openstack.org/api-ref/image/v2/index.html?expanded=upload-binary-image-data-detail#create-image

[2]
https://docs.openstack.org/api-ref/image/v2/index.html?expanded=upload-binary-image-data-detail#upload-binary-image-data