Bug 1826741

Summary:	No longer possible to select storage by image property
Product:	Red Hat OpenStack	Reporter:	Chris Ricker <chricker>
Component:	openstack-cinder	Assignee:	Brian Rosmaita <brian.rosmaita>
Status:	CLOSED ERRATA	QA Contact:	Tzach Shefi <tshefi>
Severity:	medium	Docs Contact:	Chuck Copello <ccopello>
Priority:	medium
Version:	16.0 (Train)	CC:	amcleod, brian.rosmaita, gfidente, jamsmith, mburns
Target Milestone:	z2	Keywords:	Regression, Triaged
Target Release:	16.1 (Train on RHEL 8.2)
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	openstack-cinder-15.3.1-1.20200824153426.70fa875.el8ost	Doc Type:	Bug Fix
Doc Text:	Before this update, the Block Storage service (cinder) assigned the default volume type in a `volume create` request, ignoring alternative methods of specifying the volume type. + With this update, the Block Storage service performs correctly: + * If you specify a `source_volid` in the request, the volume type that the Block Storage service sets is the volume type of the source volume. * If you specify a `snapshot_id` in the request, the volume type is inferred from the volume type of the snapshot. * If you specify an `imageRef` in the request, and the image has a `cinder_img_volume_type` image property, the volume type is inferred from the value of the image property. + Otherwise, the Block Storage service sets the volume type as the default volume type that you configure. If you do not configure a volume type, the Block Storage service uses the system default volume type, `__DEFAULT__`. + When you specify a volume type explicitly in the `volume create` request, the Block Storage service uses the type that you specify.	Story Points:	---
Clone Of:		Environment:
Last Closed:	2020-10-28 15:45:55 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Chris Ricker 2020-04-22 13:25:53 UTC

Description of problem:

I have two storage backends defined in Cinder, create a volume type for each, and set the volume_backend_name appropriately for each:

openstack volume type create ceph
openstack volume type set ceph --property volume_backend_name=tripleo_ceph
openstack volume type create solidfire
openstack volume type set --property volume_backend_name=solidfire solidfire

Then, I create two images in glance and set properties on each for the backend storage on which they should deploy

image-ceph -- set property cinder_img_volume_type=ceph
image-solidfire -- set property cinder_img_volume_type=solidfire

In earlier versions of OSP, this approach worked to let users determine which storage their images deploy on when they deploy via horizon and do boot from volume [just confirmed with an older OpenStack cloud -- looks like newton / OSP 10]

In OSP 16, after I do the above, images booted from volume in horizon always boot from the default storage backend and ignore the cinder_img_volume_type. I think it's because in OSP 16 cinder.conf has default_volume_type set and I don't see any way in the templating to unset that

How do I get this same workflow back for my users in OSP 16, of picking the storage backend they prefer their VM to boot on by selecting an appropriately tagged image?

Comment 2 Brian Rosmaita 2020-05-19 21:50:02 UTC

This is a regression from Change I4da0c13b5b3f8174a30b8557f968d6b9e641b091 in Train.

It happens here at the REST API layer:
[0] https://opendev.org/openstack/cinder/src/commit/85e60732e21ee30937d43d8b77295e4525e1c8fc/cinder/api/v3/volumes.py#L324-L328
(that's from v3, but the same code is in v2)

The change was part of an effort to eliminate untyped volumes in Cinder.  The problem is that it sets a default type (either configured by the operator or __DEFAULT__) too soon (probably so that the volume_type element in the API response wouldn't be null.  But as a result, it bypasses the selection code in the cinder-volume service [1], which would infer the volume_type based on source volume type, snapshot volume type, or image volume type.  So basically, unless a user explicitly sets a volume_type in the original API request, that user is going to wind up with the default type.

I think the way to fix this is to (roughly) move the code from [0] into [1].  We could just document that the volume_type can be null in the API response while a volume is still building.

Change I4da0c13b5b3f8174a30b8557f968d6b9e641b091, however, touched a *lot* of files, so there may be some dependencies on the volume_type being set early that I'm not seeing at first glance.

[1] https://opendev.org/openstack/cinder/src/commit/85e60732e21ee30937d43d8b77295e4525e1c8fc/cinder/volume/flows/api/create_volume.py#L356-L371

Comment 10 Tzach Shefi 2020-10-14 11:01:53 UTC

Verified on:
openstack-cinder-15.3.1-1.20200914173114.00ac80b.el8ost.noarch

Installed OSP16.1z2 with dual Cinder backend Ceph and Netapp iscsi.


Uploaded two cirros images:
(overcloud) [stack@undercloud-0 ~]$ glance image-list
+--------------------------------------+--------------+
| ID                                   | Name         |
+--------------------------------------+--------------+
| e9e4c0ad-f0b9-4040-bb54-f5dcadb32641 | cirrosCeph   |
| db5bfa6e-6f79-4b20-afe0-dc9cebe47a92 | cirrosNetapp |
+--------------------------------------+--------------+

Updated images backend types via:

(overcloud) [stack@undercloud-0 ~]$ openstack image set cirrosCeph --property cinder_img_volume_type=tripleo_ceph
(overcloud) [stack@undercloud-0 ~]$ openstack image set cirrosNetapp --property cinder_img_volume_type=netapp

Created two volumes one from each image as source:

(overcloud) [stack@undercloud-0 ~]$ cinder create 5 --image cirrosCeph --name ShouldBeCeph
+--------------------------------+--------------------------------------+
| Property                       | Value                                |
+--------------------------------+--------------------------------------+
| attachments                    | []                                   |
| availability_zone              | nova                                 |
| bootable                       | false                                |
| consistencygroup_id            | None                                 |
| created_at                     | 2020-10-14T10:53:32.000000           |
| description                    | None                                 |
| encrypted                      | False                                |
| id                             | fc9e0043-3d81-447e-8f92-744f67b73443 |
| metadata                       | {}                                   |
| migration_status               | None                                 |
| multiattach                    | False                                |
| name                           | ShouldBeCeph                         |
| os-vol-host-attr:host          | None                                 |
| os-vol-mig-status-attr:migstat | None                                 |
| os-vol-mig-status-attr:name_id | None                                 |
| os-vol-tenant-attr:tenant_id   | 031277c0ec8f4816828516c1a8e219ad     |
| replication_status             | None                                 |
| size                           | 5                                    |
| snapshot_id                    | None                                 |
| source_volid                   | None                                 |
| status                         | creating                             |
| updated_at                     | None                                 |
| user_id                        | 5e144f7432a247ac9a9b7c58262de725     |
| volume_type                    | tripleo_ceph                         |
+--------------------------------+--------------------------------------+


And second volume:
(overcloud) [stack@undercloud-0 ~]$ cinder create 5 --image cirrosNetapp  --name ShouldBeNetapp
+--------------------------------+--------------------------------------+
| Property                       | Value                                |
+--------------------------------+--------------------------------------+
| attachments                    | []                                   |
| availability_zone              | nova                                 |
| bootable                       | false                                |
| consistencygroup_id            | None                                 |
| created_at                     | 2020-10-14T10:53:59.000000           |
| description                    | None                                 |
| encrypted                      | False                                |
| id                             | f52f27a8-a40b-4eff-8def-3dbb5655e683 |
| metadata                       | {}                                   |
| migration_status               | None                                 |
| multiattach                    | False                                |
| name                           | ShouldBeNetapp                       |
| os-vol-host-attr:host          | None                                 |
| os-vol-mig-status-attr:migstat | None                                 |
| os-vol-mig-status-attr:name_id | None                                 |
| os-vol-tenant-attr:tenant_id   | 031277c0ec8f4816828516c1a8e219ad     |
| replication_status             | None                                 |
| size                           | 5                                    |
| snapshot_id                    | None                                 |
| source_volid                   | None                                 |
| status                         | creating                             |
| updated_at                     | None                                 |
| user_id                        | 5e144f7432a247ac9a9b7c58262de725     |
| volume_type                    | netapp                               |
+--------------------------------+--------------------------------------+



As can be seen both volumes were successfully and correctly created on corresponding backend:

(overcloud) [stack@undercloud-0 ~]$ cinder list
+--------------------------------------+-----------+----------------+------+--------------+----------+-------------+
| ID                                   | Status    | Name           | Size | Volume Type  | Bootable | Attached to |
+--------------------------------------+-----------+----------------+------+--------------+----------+-------------+
| f52f27a8-a40b-4eff-8def-3dbb5655e683 | available | ShouldBeNetapp | 5    | netapp       | true     |             |
| fc9e0043-3d81-447e-8f92-744f67b73443 | available | ShouldBeCeph   | 5    | tripleo_ceph | true     |             |
+--------------------------------------+-----------+----------------+------+--------------+----------+-------------+


Now lets try snapshot case, created two volumes one from each backend type:

(overcloud) [stack@undercloud-0 ~]$ cinder list
+--------------------------------------+-----------+----------------+------+--------------+----------+-------------+
| ID                                   | Status    | Name           | Size | Volume Type  | Bootable | Attached to |
+--------------------------------------+-----------+----------------+------+--------------+----------+-------------+
| 0b9891ae-05a0-4001-8d4d-abc1443cc4fd | available | NetAppVol      | 5    | netapp       | true     |             |
| 1e3510a0-8f88-42cc-aa49-81046551f2d4 | available | CephVol        | 5    | tripleo_ceph | true     |             |


Created snapshots one from each volume:
(overcloud) [stack@undercloud-0 ~]$ cinder snapshot-list
+--------------------------------------+--------------------------------------+-----------+------------+------+
| ID                                   | Volume ID                            | Status    | Name       | Size |
+--------------------------------------+--------------------------------------+-----------+------------+------+
| 28c03230-689d-47c9-8fdd-dd56bcea0e51 | 0b9891ae-05a0-4001-8d4d-abc1443cc4fd | available | NetappSnap | 5    |
| d816b9d3-e121-4086-8c34-0c01e4169cd8 | 1e3510a0-8f88-42cc-aa49-81046551f2d4 | available | CephSnap   | 5    |
+--------------------------------------+--------------------------------------+-----------+------------+------+

Now lets create two volumes using both images as sources:
(overcloud) [stack@undercloud-0 ~]$ cinder create 5 --snapshot-id 28c03230-689d-47c9-8fdd-dd56bcea0e51 --name ShouldBeNetapp2 
+--------------------------------+--------------------------------------+
| Property                       | Value                                |
+--------------------------------+--------------------------------------+
| attachments                    | []                                   |
| availability_zone              | nova                                 |
| bootable                       | true                                 |
| consistencygroup_id            | None                                 |
| created_at                     | 2020-10-14T10:59:16.000000           |
| description                    | None                                 |
| encrypted                      | False                                |
| id                             | 449432cc-6a95-4e9d-a897-27f6e050724d |
| metadata                       | {}                                   |
| migration_status               | None                                 |
| multiattach                    | False                                |
| name                           | ShouldBeNetapp2                      |
| os-vol-host-attr:host          | hostgroup@netapp#rhos_cinder         |
| os-vol-mig-status-attr:migstat | None                                 |
| os-vol-mig-status-attr:name_id | None                                 |
| os-vol-tenant-attr:tenant_id   | 031277c0ec8f4816828516c1a8e219ad     |
| replication_status             | None                                 |
| size                           | 5                                    |
| snapshot_id                    | 28c03230-689d-47c9-8fdd-dd56bcea0e51 |
| source_volid                   | None                                 |
| status                         | creating                             |
| updated_at                     | 2020-10-14T10:59:16.000000           |
| user_id                        | 5e144f7432a247ac9a9b7c58262de725     |
| volume_type                    | netapp                               |
+--------------------------------+--------------------------------------+


And second volume/snapshot:
(overcloud) [stack@undercloud-0 ~]$ cinder create 5 --snapshot-id d816b9d3-e121-4086-8c34-0c01e4169cd8 --name ShouldBeCeph2
+--------------------------------+--------------------------------------+
| Property                       | Value                                |
+--------------------------------+--------------------------------------+
| attachments                    | []                                   |
| availability_zone              | nova                                 |
| bootable                       | true                                 |
| consistencygroup_id            | None                                 |
| created_at                     | 2020-10-14T10:59:59.000000           |
| description                    | None                                 |
| encrypted                      | False                                |
| id                             | e05f5545-3eec-45f5-9d19-c9b95c02a56d |
| metadata                       | {}                                   |
| migration_status               | None                                 |
| multiattach                    | False                                |
| name                           | ShouldBeCeph2                        |
| os-vol-host-attr:host          | None                                 |
| os-vol-mig-status-attr:migstat | None                                 |
| os-vol-mig-status-attr:name_id | None                                 |
| os-vol-tenant-attr:tenant_id   | 031277c0ec8f4816828516c1a8e219ad     |
| replication_status             | None                                 |
| size                           | 5                                    |
| snapshot_id                    | d816b9d3-e121-4086-8c34-0c01e4169cd8 |
| source_volid                   | None                                 |
| status                         | creating                             |
| updated_at                     | None                                 |
| user_id                        | 5e144f7432a247ac9a9b7c58262de725     |
| volume_type                    | tripleo_ceph                         |
+--------------------------------+--------------------------------------+


Again we notice the image or rather snapshot source in this case automatically set the new volumes type/backend as expected:
(overcloud) [stack@undercloud-0 ~]$ cinder list
+--------------------------------------+-----------+-----------------+------+--------------+----------+-------------+
| ID                                   | Status    | Name            | Size | Volume Type  | Bootable | Attached to |
+--------------------------------------+-----------+-----------------+------+--------------+----------+-------------+
| 449432cc-6a95-4e9d-a897-27f6e050724d | available | ShouldBeNetapp2 | 5    | netapp       | true     |             |
| e05f5545-3eec-45f5-9d19-c9b95c02a56d | available | ShouldBeCeph2   | 5    | tripleo_ceph | true     |             |


Good to verify.

Comment 14 errata-xmlrpc 2020-10-28 15:45:55 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: openstack-cinder security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:4283