+++ This bug was initially created as a clone of Bug #1947283 +++ Currently filesystem type drivers (like nfs) rely on format auto-detection for operations like resize. This causes a problem when we write a qcow2 image file on a raw volume and the format is wrongly assumed to be qcow2 causing problems similar to[1]. [1] https://bugzilla.redhat.com/show_bug.cgi?id=1838653 --- Additional comment from RHEL Program Management on 2021-04-21 09:17:18 UTC --- This item has been properly Triaged and planned for the release, and Target Release is now set to match the release flag. For details, see https://mojo.redhat.com/docs/DOC-1195410
Verified on: python3-glance-store-1.0.2-2.20220111043148.el8ost.noarch On a deployment with Glance over Cinder over Netapp NFS. Left Cinder's default raw volumes (nfs_qcow2_volumes = false). I booted up two instance: (overcloud) [stack@undercloud-0 ~]$ nova list +--------------------------------------+-------+--------+------------+-------------+-----------------------------------+ | ID | Name | Status | Task State | Power State | Networks | +--------------------------------------+-------+--------+------------+-------------+-----------------------------------+ | e6a34a4f-edbf-4c26-934e-1f8e5a2b5562 | inst1 | ACTIVE | - | Running | internal=192.168.0.25, 10.0.0.219 | | 2701c104-e854-4e94-b855-e09a0946c78c | inst2 | ACTIVE | - | Running | internal=192.168.0.12, 10.0.0.222 | +--------------------------------------+-------+--------+------------+-------------+-----------------------------------+ Lets create two volumes, and empty volume and then a volume from an image: (overcloud) [stack@undercloud-0 ~]$ cinder create 2 --name EmptyVol +--------------------------------+--------------------------------------+ | Property | Value | +--------------------------------+--------------------------------------+ | attachments | [] | | availability_zone | nova | | bootable | false | | consistencygroup_id | None | | created_at | 2022-02-23T12:05:19.000000 | | description | None | | encrypted | False | | id | 986c9329-ed35-4074-baff-a3b5d27003dd | | metadata | {} | | migration_status | None | | multiattach | False | | name | EmptyVol | | os-vol-host-attr:host | None | | os-vol-mig-status-attr:migstat | None | | os-vol-mig-status-attr:name_id | None | | os-vol-tenant-attr:tenant_id | 9f5853b196334a2cba5b83259839ee4e | | replication_status | None | | size | 2 | | snapshot_id | None | | source_volid | None | | status | creating | | updated_at | None | | user_id | b8e993c94d29417992f72a4538e92da5 | | volume_type | tripleo | +--------------------------------+--------------------------------------+ (overcloud) [stack@undercloud-0 ~]$ cinder create 2 --name VolFromImage --image d72d130c-5fc0-476d-85d4-530ff2e72629 +--------------------------------+--------------------------------------+ | Property | Value | +--------------------------------+--------------------------------------+ | attachments | [] | | availability_zone | nova | | bootable | false | | consistencygroup_id | None | | created_at | 2022-02-23T12:05:51.000000 | | description | None | | encrypted | False | | id | e55d8947-d99c-4b96-a4ae-c3e761837367 | | metadata | {} | | migration_status | None | | multiattach | False | | name | VolFromImage | | os-vol-host-attr:host | None | | os-vol-mig-status-attr:migstat | None | | os-vol-mig-status-attr:name_id | None | | os-vol-tenant-attr:tenant_id | 9f5853b196334a2cba5b83259839ee4e | | replication_status | None | | size | 2 | | snapshot_id | None | | source_volid | None | | status | creating | | updated_at | None | | user_id | b8e993c94d29417992f72a4538e92da5 | | volume_type | tripleo | +--------------------------------+--------------------------------------+ Now lets attach each volume to an instance: (overcloud) [stack@undercloud-0 ~]$ nova volume-attach inst1 986c9329-ed35-4074-baff-a3b5d27003dd +-----------------------+--------------------------------------+ | Property | Value | +-----------------------+--------------------------------------+ | delete_on_termination | False | | device | /dev/vdb | | id | 986c9329-ed35-4074-baff-a3b5d27003dd | | serverId | e6a34a4f-edbf-4c26-934e-1f8e5a2b5562 | | tag | - | | volumeId | 986c9329-ed35-4074-baff-a3b5d27003dd | +-----------------------+--------------------------------------+ (overcloud) [stack@undercloud-0 ~]$ nova volume-attach inst2 e55d8947-d99c-4b96-a4ae-c3e761837367 +-----------------------+--------------------------------------+ | Property | Value | +-----------------------+--------------------------------------+ | delete_on_termination | False | | device | /dev/vdb | | id | e55d8947-d99c-4b96-a4ae-c3e761837367 | | serverId | 2701c104-e854-4e94-b855-e09a0946c78c | | tag | - | | volumeId | e55d8947-d99c-4b96-a4ae-c3e761837367 | +-----------------------+--------------------------------------+ And lets show connection info for both cases: cinder --os-volume-api-version 3.44 attachment-list +--------------------------------------+--------------------------------------+----------+--------------------------------------+ | ID | Volume ID | Status | Server ID | +--------------------------------------+--------------------------------------+----------+--------------------------------------+ | 63691c6e-0ad3-4fdf-bef3-f9960193cdd0 | 986c9329-ed35-4074-baff-a3b5d27003dd | attached | e6a34a4f-edbf-4c26-934e-1f8e5a2b5562 | | a9bc10bb-4b82-4aaa-bb1b-ce20704dbd96 | e55d8947-d99c-4b96-a4ae-c3e761837367 | attached | 2701c104-e854-4e94-b855-e09a0946c78c | +--------------------------------------+--------------------------------------+----------+--------------------------------------+ First/inst1: (overcloud) [stack@undercloud-0 ~]$ cinder --os-volume-api-version 3.44 attachment-show 63691c6e-0ad3-4fdf-bef3-f9960193cdd0 +-------------+--------------------------------------+ | Property | Value | +-------------+--------------------------------------+ | attach_mode | rw | | attached_at | 2022-02-23T12:07:08.000000 | | detached_at | | | id | 63691c6e-0ad3-4fdf-bef3-f9960193cdd0 | | instance | e6a34a4f-edbf-4c26-934e-1f8e5a2b5562 | | status | attached | | volume_id | 986c9329-ed35-4074-baff-a3b5d27003dd | +-------------+--------------------------------------+ +--------------------+---------------------------------------------+ | Property | Value | +--------------------+---------------------------------------------+ | access_mode | rw | | attachment_id | 63691c6e-0ad3-4fdf-bef3-f9960193cdd0 | | driver_volume_type | nfs | | encrypted | False | | export | 10.46.29.88:/cinder_nfs | | format | raw | ------> correct info | mount_point_base | /var/lib/cinder/mnt | | name | volume-986c9329-ed35-4074-baff-a3b5d27003dd | | options | None | | qos_specs | None | +--------------------+---------------------------------------------+ Second/inst2 (overcloud) [stack@undercloud-0 ~]$ cinder --os-volume-api-version 3.44 attachment-show a9bc10bb-4b82-4aaa-bb1b-ce20704dbd96 +-------------+--------------------------------------+ | Property | Value | +-------------+--------------------------------------+ | attach_mode | rw | | attached_at | 2022-02-23T12:07:18.000000 | | detached_at | | | id | a9bc10bb-4b82-4aaa-bb1b-ce20704dbd96 | | instance | 2701c104-e854-4e94-b855-e09a0946c78c | | status | attached | | volume_id | e55d8947-d99c-4b96-a4ae-c3e761837367 | +-------------+--------------------------------------+ +--------------------+---------------------------------------------+ | Property | Value | +--------------------+---------------------------------------------+ | access_mode | rw | | attachment_id | a9bc10bb-4b82-4aaa-bb1b-ce20704dbd96 | | driver_volume_type | nfs | | encrypted | False | | export | 10.46.29.88:/cinder_nfs | | format | raw | --> Correct info | mount_point_base | /var/lib/cinder/mnt | | name | volume-e55d8947-d99c-4b96-a4ae-c3e761837367 | | options | None | | qos_specs | None | +--------------------+---------------------------------------------+ As can be seen above on both cases, the Cinder volume which resides on a netapp NFS share is raw. Now lets set on Cinder.conf under the backend section nfs_qcow2_volumes = true, plus restart c-vol\c-api. Create / attach new volumes and recheck connection info. (overcloud) [stack@undercloud-0 ~]$ cinder create 2 --name EmptyVolB +--------------------------------+--------------------------------------+ | Property | Value | +--------------------------------+--------------------------------------+ | attachments | [] | | availability_zone | nova | | bootable | false | | consistencygroup_id | None | | created_at | 2022-02-24T07:40:57.000000 | | description | None | | encrypted | False | | id | 27b79103-e568-4da5-9c62-a29ac3e7251e | | metadata | {} | | migration_status | None | | multiattach | False | | name | EmptyVolB | | os-vol-host-attr:host | None | | os-vol-mig-status-attr:migstat | None | | os-vol-mig-status-attr:name_id | None | | os-vol-tenant-attr:tenant_id | 9f5853b196334a2cba5b83259839ee4e | | replication_status | None | | size | 2 | | snapshot_id | None | | source_volid | None | | status | creating | | updated_at | None | | user_id | b8e993c94d29417992f72a4538e92da5 | | volume_type | tripleo | +--------------------------------+--------------------------------------+ (overcloud) [stack@undercloud-0 ~]$ cinder list +--------------------------------------+-----------+---------------+------+-------------+----------+-------------+ | ID | Status | Name | Size | Volume Type | Bootable | Attached to | +--------------------------------------+-----------+---------------+------+-------------+----------+-------------+ | 27b79103-e568-4da5-9c62-a29ac3e7251e | available | EmptyVolB | 2 | tripleo | false | | +--------------------------------------+-----------+---------------+------+-------------+----------+-------------+ On the nfs mount lets confirm volume is indeed qcow2 [root@controller-1 de7de858b94dc496d8c1a0c52a2c79a8]# qemu-img info volume-27b79103-e568-4da5-9c62-a29ac3e7251e image: volume-27b79103-e568-4da5-9c62-a29ac3e7251e file format: qcow2 -----------------------> qcow2 as expected virtual size: 2 GiB (2147483648 bytes) disk size: 560 KiB cluster_size: 65536 Format specific information: compat: 1.1 compression type: zlib lazy refcounts: false refcount bits: 16 corrupt: false extended l2: false By mistake I had deleted the original instances, created news ones instead, explains why the same instance names have other UUIDs. Now lets attach this volume to inst1 (overcloud) [stack@undercloud-0 ~]$ nova volume-attach inst1 27b79103-e568-4da5-9c62-a29ac3e7251e +-----------------------+--------------------------------------+ | Property | Value | +-----------------------+--------------------------------------+ | delete_on_termination | False | | device | /dev/vdb | | id | 27b79103-e568-4da5-9c62-a29ac3e7251e | | serverId | 3703d88b-60c0-4d34-849d-2f0d30eb71b9 | | tag | - | | volumeId | 27b79103-e568-4da5-9c62-a29ac3e7251e | +-----------------------+--------------------------------------+ (overcloud) [stack@undercloud-0 ~]$ cinder --os-volume-api-version 3.44 attachment-list +--------------------------------------+--------------------------------------+----------+--------------------------------------+ | ID | Volume ID | Status | Server ID | +--------------------------------------+--------------------------------------+----------+--------------------------------------+ | 5b0ab5b9-ec05-48b5-94c9-6efbdfcbfba0 | 27b79103-e568-4da5-9c62-a29ac3e7251e | attached | 3703d88b-60c0-4d34-849d-2f0d30eb71b9 | +--------------------------------------+--------------------------------------+----------+--------------------------------------+ (overcloud) [stack@undercloud-0 ~]$ cinder --os-volume-api-version 3.44 attachment-show 5b0ab5b9-ec05-48b5-94c9-6efbdfcbfba0 +-------------+--------------------------------------+ | Property | Value | +-------------+--------------------------------------+ | attach_mode | rw | | attached_at | 2022-02-24T08:07:38.000000 | | detached_at | | | id | 5b0ab5b9-ec05-48b5-94c9-6efbdfcbfba0 | | instance | 3703d88b-60c0-4d34-849d-2f0d30eb71b9 | | status | attached | | volume_id | 27b79103-e568-4da5-9c62-a29ac3e7251e | +-------------+--------------------------------------+ +--------------------+---------------------------------------------+ | Property | Value | +--------------------+---------------------------------------------+ | access_mode | rw | | attachment_id | 5b0ab5b9-ec05-48b5-94c9-6efbdfcbfba0 | | driver_volume_type | nfs | | encrypted | False | | export | 10.46.29.88:/cinder_nfs | | format | qcow2 | ----> Great the correct format is shown! | mount_point_base | /var/lib/cinder/mnt | | name | volume-27b79103-e568-4da5-9c62-a29ac3e7251e | | options | None | | qos_specs | None | +--------------------+---------------------------------------------+ Now lets try it again this time using a volume from an image: overcloud) [stack@undercloud-0 ~]$ cinder create 2 --image 228387e8-1597-449f-aa88-804ce16bc1da --name VolFromImageB +--------------------------------+------------------------------------------------------+ | Property | Value | +--------------------------------+------------------------------------------------------+ | attachments | [] | | availability_zone | nova | | bootable | false | | consistencygroup_id | None | | created_at | 2022-02-24T09:01:02.000000 | | description | None | | encrypted | False | | id | 740e6825-f8c2-4b66-a1fa-0a17eec8148b | | metadata | {} | | migration_status | None | | multiattach | False | | name | VolFromImageB | | os-vol-host-attr:host | hostgroup@tripleo_netapp_nfs#10.46.29.88:/cinder_nfs | | os-vol-mig-status-attr:migstat | None | | os-vol-mig-status-attr:name_id | None | | os-vol-tenant-attr:tenant_id | 9f5853b196334a2cba5b83259839ee4e | | replication_status | None | | size | 2 | | snapshot_id | None | | source_volid | None | | status | creating | | updated_at | 2022-02-24T09:01:02.000000 | | user_id | b8e993c94d29417992f72a4538e92da5 | | volume_type | tripleo | +--------------------------------+------------------------------------------------------+ We check qemu-image [root@controller-1 de7de858b94dc496d8c1a0c52a2c79a8]# qemu-img info volume-740e6825-f8c2-4b66-a1fa-0a17eec8148b image: volume-740e6825-f8c2-4b66-a1fa-0a17eec8148b file format: raw virtual size: 2 GiB (2147483648 bytes) disk size: 13.2 MiB Notice is says raw and it should be qcow2, expected due to qemu but if we check the db we see the correct qcow2 format is indeed: 2022-02-24 09:01:12 | NULL | NULL | 0 | 91 | 740e6825-f8c2-4b66-a1fa-0a17eec8148b | format | qcow2 | Attaching vol to instance, plus checking connection info (overcloud) [stack@undercloud-0 ~]$ nova volume-attach inst2 740e6825-f8c2-4b66-a1fa-0a17eec8148b +-----------------------+--------------------------------------+ | Property | Value | +-----------------------+--------------------------------------+ | delete_on_termination | False | | device | /dev/vdb | | id | 740e6825-f8c2-4b66-a1fa-0a17eec8148b | | serverId | d198e0eb-d969-4fb9-9f71-9def21fe08b9 | | tag | - | | volumeId | 740e6825-f8c2-4b66-a1fa-0a17eec8148b | +-----------------------+--------------------------------------+ (overcloud) [stack@undercloud-0 ~]$ cinder --os-volume-api-version 3.44 attachment-list +--------------------------------------+--------------------------------------+----------+--------------------------------------+ | ID | Volume ID | Status | Server ID | +--------------------------------------+--------------------------------------+----------+--------------------------------------+ | 5b0ab5b9-ec05-48b5-94c9-6efbdfcbfba0 | 27b79103-e568-4da5-9c62-a29ac3e7251e | attached | 3703d88b-60c0-4d34-849d-2f0d30eb71b9 | | 84a78e67-2ae3-43a5-85e4-43b46faffe6e | 740e6825-f8c2-4b66-a1fa-0a17eec8148b | attached | d198e0eb-d969-4fb9-9f71-9def21fe08b9 | +--------------------------------------+--------------------------------------+----------+--------------------------------------+ (overcloud) [stack@undercloud-0 ~]$ cinder --os-volume-api-version 3.44 attachment-show 84a78e67-2ae3-43a5-85e4-43b46faffe6e +-------------+--------------------------------------+ | Property | Value | +-------------+--------------------------------------+ | attach_mode | rw | | attached_at | 2022-02-24T09:13:10.000000 | | detached_at | | | id | 84a78e67-2ae3-43a5-85e4-43b46faffe6e | | instance | d198e0eb-d969-4fb9-9f71-9def21fe08b9 | | status | attached | | volume_id | 740e6825-f8c2-4b66-a1fa-0a17eec8148b | +-------------+--------------------------------------+ +--------------------+---------------------------------------------+ | Property | Value | +--------------------+---------------------------------------------+ | access_mode | rw | | attachment_id | 84a78e67-2ae3-43a5-85e4-43b46faffe6e | | driver_volume_type | nfs | | encrypted | False | | export | 10.46.29.88:/cinder_nfs | | format | qcow2 |-------> indeed, a volume from an image, is attached correctly as qcow2. | mount_point_base | /var/lib/cinder/mnt | | name | volume-740e6825-f8c2-4b66-a1fa-0a17eec8148b | | options | None | | qos_specs | None | +--------------------+---------------------------------------------+ We proved that correct format info for filesystem type drivers happens during attachment of either empty volumes as well as volumes from an image. And that the above is true for both cases: the default nfs_qcow2_volumes=false, as well as nfs_qcow2_volumes=true. Good to verify.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Release of components for Red Hat OpenStack Platform 16.2.2), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2022:1001