Bug 1984304
| Summary: | Original volume size is lost when uploading a volume to a qcow2 image | ||
|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Takashi Kajinami <tkajinam> |
| Component: | openstack-cinder | Assignee: | Cinder Bugs List <cinder-bugs> |
| Status: | NEW --- | QA Contact: | Evelina Shames <eshames> |
| Severity: | high | Docs Contact: | Andy Stillman <astillma> |
| Priority: | medium | ||
| Version: | 16.1 (Train) | CC: | ltoscano, nnavarat, rdhasman, sputhenp |
| Target Milestone: | --- | Keywords: | Triaged |
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | Type: | Bug | |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
One possible solution here is to make cinder set the min_disk attribute based on size of the source volume. With this attribute set, creating a new volume with smaller size is rejected in api layer with a clear 400 response. ~~~ (overcloud) [stack@undercloud-0 ~]$ openstack image set testimage --min-disk 2 (overcloud) [stack@undercloud-0 ~]$ openstack image show testimage +------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | checksum | dc9ef2e7cf57a420ef03a408178a4076 | | container_format | bare | | created_at | 2021-07-21T07:48:24Z | | disk_format | qcow2 | | file | /v2/images/e1272ef7-f39b-460e-a529-89f5eb0c8e14/file | | id | e1272ef7-f39b-460e-a529-89f5eb0c8e14 | | min_disk | 2 | | min_ram | 0 | | name | testimage | | owner | 942783ae248c4e9eb353a6e6b327bda5 | | properties | direct_url='swift+config://ref1/glance/e1272ef7-f39b-460e-a529-89f5eb0c8e14', os_hash_algo='sha512', os_hash_value='fe600797d0cc7e4a2a598841b129709a327981a52e7de3a1b13dc09427c53136ca42df9bb174b79984f829c1ba8e8ee64891de542194cd9243d149c6593680db', os_hidden='False', stores='default_backend' | | protected | False | | schema | /v2/schemas/image | | size | 197120 | | status | active | | tags | | | updated_at | 2021-07-21T08:09:20Z | | virtual_size | None | | visibility | shared | +------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ (overcloud) [stack@undercloud-0 ~]$ openstack volume create smallervolume --image testimage --size 1 Invalid input received: Volume size 1GB cannot be smaller than the image minDisk size 2GB. (HTTP 400) (Request-ID: req-40eff0d2-8a19-4b4d-9dd3-3bedac288531) ~~~ Hi, from the description in comment #1, "It is because that virtual size of the image(2GB) doesn't fit in a volume size(1GB), but this error is not exposed to users. This makes it difficult to let users know actual cause of the error." I think there is a request with the BZ regarding a new user message that will notify user regarding any async failure with the given operation. If my assumption is correct, i can work on adding one. Hi Rajat, I think there are two points we can improve related to this problem. The first thing is, as you mention, cinder doesn't tell why volume creation fails if there is something wrong with the source image. (If the image data is in wrong image format, actual size of the image doesn't fit in the volume size) I think we can add more clear message on API response but that would require more careful consideration so that cinder doesn't expose any sensitive data, and I tend to regard this as a long term topic. On the other hand, cinder already has a mechanism to validate a given volume size based on min_disk property of a source image. Cinder knows size of a source volume when creating a image from that volume, so it can add that source volume size as min_disk property of the image, then subsequent volume creation from that image should fail if a user tries to create a volume smaller than the source volume. This avoids the situation caused by volume-to-image and image-to-volume I described, and I was expecting this direction initially as more direct fix for the issue. One benefit of setting min_disk property is that the validation logic is implemented in api layer, so an error can be returned to a volume creation request directly, instead of making the volume in error status asynchronously. (In reply to Takashi Kajinami from comment #8) > Hi Rajat, > > I think there are two points we can improve related to this problem. > > The first thing is, as you mention, cinder doesn't tell why volume creation > fails if there is something wrong with the source image. > (If the image data is in wrong image format, actual size of the image > doesn't fit in the volume size) > I think we can add more clear message on API response but that would require > more careful consideration so that cinder doesn't expose > any sensitive data, and I tend to regard this as a long term topic. > > On the other hand, cinder already has a mechanism to validate a given volume > size based on min_disk property of a source image. > Cinder knows size of a source volume when creating a image from that volume, > so it can add that source volume size > as min_disk property of the image, then subsequent volume creation from that > image should fail if a user tries to create > a volume smaller than the source volume. This avoids the situation caused by > volume-to-image and image-to-volume I described, > and I was expecting this direction initially as more direct fix for the > issue. > One benefit of setting min_disk property is that the validation logic is > implemented in api layer, so an error can be returned > to a volume creation request directly, instead of making the volume in error > status asynchronously. So both need to be implemented at the same time to consider this issue fixed ( the property may not be present for old volumes or just because it is removed, or it may be incorrect). |
Description of problem: Currently when uploading a cinder volume to a glance image in qcow2 format, size of the newly created glance image is computed as size of the actual qcow2 file instead of original volume size. For example if we create an empty 1GiB volume ~~~ (overcloud) [stack@undercloud-0 ~]$ openstack volume create sourcevolume --size 2 ... (overcloud) [stack@undercloud-0 ~]$ openstack volume show sourcevolume +--------------------------------+---------------------------------------+ | Field | Value | +--------------------------------+---------------------------------------+ | attachments | [] | | availability_zone | nova | | bootable | false | | consistencygroup_id | None | | created_at | 2021-07-21T07:47:18.000000 | | description | None | | encrypted | False | | id | cb2113ce-2768-412c-81c2-190bdd7e7b2c | | migration_status | None | | multiattach | False | | name | sourcevolume | | os-vol-host-attr:host | hostgroup@tripleo_iscsi#tripleo_iscsi | | os-vol-mig-status-attr:migstat | None | | os-vol-mig-status-attr:name_id | None | | os-vol-tenant-attr:tenant_id | 942783ae248c4e9eb353a6e6b327bda5 | | properties | | | replication_status | None | | size | 2 | | snapshot_id | None | | source_volid | None | | status | available | | type | tripleo | | updated_at | 2021-07-21T07:47:19.000000 | | user_id | 06b547a0af8f49fd8239c85ce5d9571b | +--------------------------------+---------------------------------------+ ~~~ ... and then create a qcow2 image from the volume ~~~ (overcloud) [stack@undercloud-0 ~]$ openstack image create --volume sourcevolume --disk-format qcow2 testimage ... ~~~ ... then the image gets 197120 Bi which is the actual size of qcow2 file. ~~~ (overcloud) [stack@undercloud-0 ~]$ openstack image show testimage +------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | checksum | dc9ef2e7cf57a420ef03a408178a4076 | | container_format | bare | | created_at | 2021-07-21T07:48:24Z | | disk_format | qcow2 | | file | /v2/images/e1272ef7-f39b-460e-a529-89f5eb0c8e14/file | | id | e1272ef7-f39b-460e-a529-89f5eb0c8e14 | | min_disk | 0 | | min_ram | 0 | | name | testimage | | owner | 942783ae248c4e9eb353a6e6b327bda5 | | properties | direct_url='swift+config://ref1/glance/e1272ef7-f39b-460e-a529-89f5eb0c8e14', os_hash_algo='sha512', os_hash_value='fe600797d0cc7e4a2a598841b129709a327981a52e7de3a1b13dc09427c53136ca42df9bb174b79984f829c1ba8e8ee64891de542194cd9243d149c6593680db', os_hidden='False', stores='default_backend' | | protected | False | | schema | /v2/schemas/image | | size | 197120 | | status | active | | tags | | | updated_at | 2021-07-21T07:48:28Z | | virtual_size | None | | visibility | shared | +------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ ~~~ If we try to create a 1GB volume form that image, the request is accpeted but the volume eventually becomes error status. ~~~ (overcloud) [stack@undercloud-0 ~]$ openstack volume create smallervolume --image testimage --size 1 ... (overcloud) [stack@undercloud-0 ~]$ openstack volume show smallervolume +--------------------------------+---------------------------------------+ | Field | Value | +--------------------------------+---------------------------------------+ | attachments | [] | | availability_zone | nova | | bootable | false | | consistencygroup_id | None | | created_at | 2021-07-21T07:52:37.000000 | | description | None | | encrypted | False | | id | a72f04f5-0351-4a3a-856d-2d7a9586a4fc | | migration_status | None | | multiattach | False | | name | smallervolume | | os-vol-host-attr:host | hostgroup@tripleo_iscsi#tripleo_iscsi | | os-vol-mig-status-attr:migstat | None | | os-vol-mig-status-attr:name_id | None | | os-vol-tenant-attr:tenant_id | 942783ae248c4e9eb353a6e6b327bda5 | | properties | | | replication_status | None | | size | 1 | | snapshot_id | None | | source_volid | None | | status | error | | type | tripleo | | updated_at | 2021-07-21T07:52:39.000000 | | user_id | 06b547a0af8f49fd8239c85ce5d9571b | | volume_image_metadata | {'signature_verified': 'False'} | +--------------------------------+---------------------------------------+ ~~~ It is because that virtual size of the image(2GB) doesn't fit in a volume size(1GB), but this error is not exposed to users. This makes it difficult to let users know actual cause of the error. ~~~ 2021-07-21 07:52:39.678 74 ERROR oslo_messaging.rpc.server [req-30713ed0-564d-4f28-8035-252b63e697a5 06b547a0af8f49fd8239c85ce5d9571b 942783ae248c4e9eb353a6e6b327bda5 - default default] Exception during message handling: cinder.exception. ImageUnacceptable: Image e1272ef7-f39b-460e-a529-89f5eb0c8e14 is unacceptable: Image virtual size is 2GB and doesn't fit in a volume of size 1GB. 2021-07-21 07:52:39.678 74 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2021-07-21 07:52:39.678 74 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming 2021-07-21 07:52:39.678 74 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) ... 2021-07-21 07:52:39.678 74 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/cinder/volume/flows/manager/create_volume.py", line 1035, in _create_from_image 2021-07-21 07:52:39.678 74 ERROR oslo_messaging.rpc.server image_service) 2021-07-21 07:52:39.678 74 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/cinder/volume/flows/manager/create_volume.py", line 915, in _create_from_image_cache_or_download 2021-07-21 07:52:39.678 74 ERROR oslo_messaging.rpc.server data.virtual_size, volume.size, image_id) 2021-07-21 07:52:39.678 74 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/cinder/image/image_utils.py", line 742, in check_virtual_size 2021-07-21 07:52:39.678 74 ERROR oslo_messaging.rpc.server reason=reason) 2021-07-21 07:52:39.678 74 ERROR oslo_messaging.rpc.server cinder.exception.ImageUnacceptable: Image e1272ef7-f39b-460e-a529-89f5eb0c8e14 is unacceptable: Image virtual size is 2GB and doesn't fit in a volume of size 1GB. 2021-07-21 07:52:39.678 74 ERROR oslo_messaging.rpc.server ~~~ Version-Release number of selected component (if applicable): How reproducible: Always Steps to Reproduce: 1. Create a 2GB empty volume 2. Create qcow2 image from the 2GB volume 3. Create a 1GB volume from the image Actual results: Request to create a 1GB volume is accepted but the volume becomes error status Expected results: Request to create a 1GB volume is rejected according to image virtual size. Additional info: