Bug 1948260 - rbd_max_clone_depth flatten kicks in on depth x+1
Summary: rbd_max_clone_depth flatten kicks in on depth x+1
Keywords:
Status: CLOSED DEFERRED
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-cinder
Version: 16.2 (Train)
Hardware: Unspecified
OS: Unspecified
low
low
Target Milestone: ---
: ---
Assignee: Cinder Bugs List
QA Contact: Evelina Shames
RHOS Documentation Team
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-04-11 07:00 UTC by Tzach Shefi
Modified: 2024-12-11 19:57 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2024-12-11 19:57:44 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
CInder c-vol log (1.44 MB, text/plain)
2021-04-12 07:28 UTC, Tzach Shefi
no flags Details
c-vol.log retest 16.2 rbd clone v1 (565.46 KB, text/plain)
2021-04-13 07:47 UTC, Tzach Shefi
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker OSP-2284 0 None None None 2022-11-24 08:47:47 UTC

Description Tzach Shefi 2021-04-11 07:00:12 UTC
Description of problem: Recntly noticed rbd_max_clone_depth starts flatting but only on X+1 depth. 


Version-Release number of selected component (if applicable):
openstack-cinder-15.4.2-2.20210225045015

How reproducible:
Unsure first time I hit this. 

Steps to Reproduce:
1. Create a volume with data, volA 
I only noticed the max depth motice on 7th volume which is the 6th clone -> volG cloned from volF. 


Create an empty volume, attach to instance write some data on it:
(overcloud) [stack@undercloud-0 ~]$ cinder create 2 --name volA
+--------------------------------+--------------------------------------+
| Property                       | Value                                |
+--------------------------------+--------------------------------------+
| attachments                    | []                                   |
| availability_zone              | nova                                 |
| bootable                       | false                                |
| consistencygroup_id            | None                                 |
| created_at                     | 2021-03-29T10:16:26.000000           |
| description                    | None                                 |
| encrypted                      | False                                |
| id                             | 7ed0860e-46b2-400d-ba92-cf3422eee35c |
| metadata                       | {}                                   |
| migration_status               | None                                 |
| multiattach                    | False                                |
| name                           | volA                                 |
| os-vol-host-attr:host          | None                                 |
| os-vol-mig-status-attr:migstat | None                                 |
| os-vol-mig-status-attr:name_id | None                                 |
| os-vol-tenant-attr:tenant_id   | ee7b2b94ecd44b3695368c0342841fa6     |
| replication_status             | None                                 |
| size                           | 2                                    |
| snapshot_id                    | None                                 |
| source_volid                   | None                                 |
| status                         | creating                             |
| updated_at                     | None                                 |
| user_id                        | c909ee22438e4eb7bc72a4b37da41b7b     |
| volume_type                    | tripleo                              |
+--------------------------------+--------------------------------------+

(overcloud) [stack@undercloud-0 ~]$ nova volume-attach inst1 7ed0860e-46b2-400d-ba92-cf3422eee35c
+-----------------------+--------------------------------------+
| Property              | Value                                |
+-----------------------+--------------------------------------+
| delete_on_termination | False                                |
| device                | /dev/vdb                             |
| id                    | 7ed0860e-46b2-400d-ba92-cf3422eee35c |
| serverId              | 1318458a-c4b0-48fc-83d6-76e48253d50c |
| tag                   | -                                    |
| volumeId              | 7ed0860e-46b2-400d-ba92-cf3422eee35c |
+-----------------------+--------------------------------------+

Detach volume after writing data:
(overcloud) [stack@undercloud-0 ~]$ nova volume-detach inst1 7ed0860e-46b2-400d-ba92-cf3422eee35c

2. Create a new volume VolB cloned from volA
Now lets create clones:
                       |
(overcloud) [stack@undercloud-0 ~]$ cinder create 2 --source-volid 7ed0860e-46b2-400d-ba92-cf3422eee35c --name volB
+--------------------------------+--------------------------------------+
| Property                       | Value                                |
+--------------------------------+--------------------------------------+
| attachments                    | []                                   |
| availability_zone              | nova                                 |
| bootable                       | false                                |
| consistencygroup_id            | None                                 |
| created_at                     | 2021-03-29T10:27:28.000000           |
| description                    | None                                 |
| encrypted                      | False                                |
| id                             | 95116b3a-6aed-4b80-8b97-05831f361389 |
| metadata                       | {}                                   |
| migration_status               | None                                 |
| multiattach                    | False                                |
| name                           | volB                                 |
| os-vol-host-attr:host          | None                                 |
| os-vol-mig-status-attr:migstat | None                                 |
| os-vol-mig-status-attr:name_id | None                                 |
| os-vol-tenant-attr:tenant_id   | ee7b2b94ecd44b3695368c0342841fa6     |
| replication_status             | None                                 |
| size                           | 2                                    |
| snapshot_id                    | None                                 |
| source_volid                   | 7ed0860e-46b2-400d-ba92-cf3422eee35c |
| status                         | creating                             |
| updated_at                     | None                                 |
| user_id                        | c909ee22438e4eb7bc72a4b37da41b7b     |
| volume_type                    | tripleo                              |
+--------------------------------+--------------------------------------+


3. Repeat till we notice rbd flatting on c-vol log

And a clone of volB
(overcloud) [stack@undercloud-0 ~]$ cinder create 2 --source-volid 95116b3a-6aed-4b80-8b97-05831f361389 --name volC
+--------------------------------+--------------------------------------+
| Property                       | Value                                |
+--------------------------------+--------------------------------------+
| attachments                    | []                                   |
| availability_zone              | nova                                 |
| bootable                       | false                                |
| consistencygroup_id            | None                                 |
| created_at                     | 2021-03-29T10:28:14.000000           |
| description                    | None                                 |
| encrypted                      | False                                |
| id                             | 48e85f60-72a0-41f3-bbaf-63960d61d67a |
| metadata                       | {}                                   |
| migration_status               | None                                 |
| multiattach                    | False                                |
| name                           | volC                                 |
| os-vol-host-attr:host          | None                                 |
| os-vol-mig-status-attr:migstat | None                                 |
| os-vol-mig-status-attr:name_id | None                                 |
| os-vol-tenant-attr:tenant_id   | ee7b2b94ecd44b3695368c0342841fa6     |
| replication_status             | None                                 |
| size                           | 2                                    |
| snapshot_id                    | None                                 |
| source_volid                   | 95116b3a-6aed-4b80-8b97-05831f361389 |
| status                         | creating                             |
| updated_at                     | None                                 |
| user_id                        | c909ee22438e4eb7bc72a4b37da41b7b     |
| volume_type                    | tripleo                              |
+--------------------------------+--------------------------------------+

Clone of volc:
(overcloud) [stack@undercloud-0 ~]$ cinder create 2 --source-volid 48e85f60-72a0-41f3-bbaf-63960d61d67a --name volD
+--------------------------------+--------------------------------------+
| Property                       | Value                                |
+--------------------------------+--------------------------------------+
| attachments                    | []                                   |
| availability_zone              | nova                                 |
| bootable                       | false                                |
| consistencygroup_id            | None                                 |
| created_at                     | 2021-03-29T10:28:52.000000           |
| description                    | None                                 |
| encrypted                      | False                                |
| id                             | fe9e40e6-9b3b-48b7-b11b-5229628bfc5a |
| metadata                       | {}                                   |
| migration_status               | None                                 |
| multiattach                    | False                                |
| name                           | volD                                 |
| os-vol-host-attr:host          | None                                 |
| os-vol-mig-status-attr:migstat | None                                 |
| os-vol-mig-status-attr:name_id | None                                 |
| os-vol-tenant-attr:tenant_id   | ee7b2b94ecd44b3695368c0342841fa6     |
| replication_status             | None                                 |
| size                           | 2                                    |
| snapshot_id                    | None                                 |
| source_volid                   | 48e85f60-72a0-41f3-bbaf-63960d61d67a |
| status                         | creating                             |
| updated_at                     | None                                 |
| user_id                        | c909ee22438e4eb7bc72a4b37da41b7b     |
| volume_type                    | tripleo                              |
+--------------------------------+--------------------------------------+

Clone of volD:
(overcloud) [stack@undercloud-0 ~]$ cinder create 2 --source-volid fe9e40e6-9b3b-48b7-b11b-5229628bfc5a --name volE
+--------------------------------+--------------------------------------+
| Property                       | Value                                |
+--------------------------------+--------------------------------------+
| attachments                    | []                                   |
| availability_zone              | nova                                 |
| bootable                       | false                                |
| consistencygroup_id            | None                                 |
| created_at                     | 2021-03-29T10:29:28.000000           |
| description                    | None                                 |
| encrypted                      | False                                |
| id                             | dcb1a9d7-cd1e-4829-afc7-910763339c79 |
| metadata                       | {}                                   |
| migration_status               | None                                 |
| multiattach                    | False                                |
| name                           | volE                                 |
| os-vol-host-attr:host          | None                                 |
| os-vol-mig-status-attr:migstat | None                                 |
| os-vol-mig-status-attr:name_id | None                                 |
| os-vol-tenant-attr:tenant_id   | ee7b2b94ecd44b3695368c0342841fa6     |
| replication_status             | None                                 |
| size                           | 2                                    |
| snapshot_id                    | None                                 |
| source_volid                   | fe9e40e6-9b3b-48b7-b11b-5229628bfc5a |
| status                         | creating                             |
| updated_at                     | None                                 |
| user_id                        | c909ee22438e4eb7bc72a4b37da41b7b     |
| volume_type                    | tripleo                              |
+--------------------------------+--------------------------------------+


Now lets pass the 5th level with volF cloned from volE:
(overcloud) [stack@undercloud-0 ~]$ cinder create 2 --source-volid dcb1a9d7-cd1e-4829-afc7-910763339c79 --name volF
+--------------------------------+--------------------------------------+
| Property                       | Value                                |
+--------------------------------+--------------------------------------+
| attachments                    | []                                   |
| availability_zone              | nova                                 |
| bootable                       | false                                |
| consistencygroup_id            | None                                 |
| created_at                     | 2021-03-29T10:31:35.000000           |
| description                    | None                                 |
| encrypted                      | False                                |
| id                             | cd5126a8-3bd5-47a6-9b9f-67ff33fb5ba8 |
| metadata                       | {}                                   |
| migration_status               | None                                 |
| multiattach                    | False                                |
| name                           | volF                                 |
| os-vol-host-attr:host          | None                                 |
| os-vol-mig-status-attr:migstat | None                                 |
| os-vol-mig-status-attr:name_id | None                                 |
| os-vol-tenant-attr:tenant_id   | ee7b2b94ecd44b3695368c0342841fa6     |
| replication_status             | None                                 |
| size                           | 2                                    |
| snapshot_id                    | None                                 |
| source_volid                   | dcb1a9d7-cd1e-4829-afc7-910763339c79 |
| status                         | creating                             |
| updated_at                     | None                                 |
| user_id                        | c909ee22438e4eb7bc72a4b37da41b7b     |
| volume_type                    | tripleo                              |
+--------------------------------+--------------------------------------+


No traceback or errors were reported along the way.
All volumes are available:
(overcloud) [stack@undercloud-0 ~]$ cinder list
+--------------------------------------+-----------+------+------+-------------+----------+-------------+
| ID                                   | Status    | Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+-----------+------+------+-------------+----------+-------------+
| 48e85f60-72a0-41f3-bbaf-63960d61d67a | available | volC | 2    | tripleo     | false    |             |
| 66078ab5-a1c8-4920-86bc-4e9e79d51bb4 | available | 1    | 2    | tripleo     | false    |             |
| 7ed0860e-46b2-400d-ba92-cf3422eee35c | available | volA | 2    | tripleo     | false    |             |
| 95116b3a-6aed-4b80-8b97-05831f361389 | available | volB | 2    | tripleo     | false    |             |
| cd5126a8-3bd5-47a6-9b9f-67ff33fb5ba8 | available | volF | 2    | tripleo     | false    |             |
| dcb1a9d7-cd1e-4829-afc7-910763339c79 | available | volE | 2    | tripleo     | false    |             |
| fe9e40e6-9b3b-48b7-b11b-5229628bfc5a | available | volD | 2    | tripleo     | false    |             |
+--------------------------------------+-----------+------+------+-------------+----------+-------------+ 

Default depth is set to 5. 

[root@controller-2 /]# grep -irn rbd_max_clone_dept /etc/cinder/cinder.conf 
3126:#rbd_max_clone_depth = 5


Odd I didn't see a flatten command on c-vol log yet, despite having 5 clones.
Lets try another clone


(overcloud) [stack@undercloud-0 ~]$ cinder create 2 --source-volid cd5126a8-3bd5-47a6-9b9f-67ff33fb5ba8 --name volG
+--------------------------------+--------------------------------------+
| Property                       | Value                                |
+--------------------------------+--------------------------------------+
| attachments                    | []                                   |
| availability_zone              | nova                                 |
| bootable                       | false                                |
| consistencygroup_id            | None                                 |
| created_at                     | 2021-03-29T10:43:33.000000           |
| description                    | None                                 |
| encrypted                      | False                                |
| id                             | 511f11b1-1093-4c5f-ae05-1e8a4a8f187a |
| metadata                       | {}                                   |
| migration_status               | None                                 |
| multiattach                    | False                                |
| name                           | volG                                 |
| os-vol-host-attr:host          | None                                 |
| os-vol-mig-status-attr:migstat | None                                 |
| os-vol-mig-status-attr:name_id | None                                 |
| os-vol-tenant-attr:tenant_id   | ee7b2b94ecd44b3695368c0342841fa6     |
| replication_status             | None                                 |
| size                           | 2                                    |
| snapshot_id                    | None                                 |
| source_volid                   | cd5126a8-3bd5-47a6-9b9f-67ff33fb5ba8 |
| status                         | creating                             |
| updated_at                     | None                                 |
| user_id                        | c909ee22438e4eb7bc72a4b37da41b7b     |
| volume_type                    | tripleo                              |
+--------------------------------+--------------------------------------+

OK only now when depth = 6 did we notice flatting occurring:
This is the bug it should have happned on depth 5. 

2021-03-29 10:43:34.738 49 DEBUG cinder.volume.drivers.rbd [req-7addd9a2-bfe6-4eb7-aa13-7b36d3771140 c909ee22438e4eb7bc72a4b37da41b7b ee7b2b94ecd44b3695368c0342841fa6 - default default] cloning 'volume-cd5126a8-3bd5-47a6-9b9f-67ff33fb5ba8' to 'volume-511f11b1-1093-4c5f-ae05-1e8a4a8f187a' create_cloned_volume /usr/lib/python3.6/site-packages/cinder/volume/drivers/rbd.py:707
2021-03-29 10:43:35.126 49 DEBUG cinder.volume.drivers.rbd [req-7addd9a2-bfe6-4eb7-aa13-7b36d3771140 c909ee22438e4eb7bc72a4b37da41b7b ee7b2b94ecd44b3695368c0342841fa6 - default default] Volume volume-7ed0860e-46b2-400d-ba92-cf3422eee35c is not a clone. _get_clone_info /usr/lib/python3.6/site-packages/cinder/volume/drivers/rbd.py:1061
2021-03-29 10:43:35.130 49 INFO cinder.volume.drivers.rbd [req-7addd9a2-bfe6-4eb7-aa13-7b36d3771140 c909ee22438e4eb7bc72a4b37da41b7b ee7b2b94ecd44b3695368c0342841fa6 - default default] maximum clone depth (5) has been reached - flattening dest volume
2021-03-29 10:43:35.230 49 DEBUG cinder.volume.drivers.rbd [req-7addd9a2-bfe6-4eb7-aa13-7b36d3771140 c909ee22438e4eb7bc72a4b37da41b7b ee7b2b94ecd44b3695368c0342841fa6 - default default] flattening dest volume volume-511f11b1-1093-4c5f-ae05-1e8a4a8f187a create_cloned_volume /usr/lib/python3.6/site-packages/cinder/volume/drivers/rbd.py:737



[root@controller-2 /]#  rbd -p volumes ls -l
NAME                                                                                               SIZE  PARENT                                                                                                     FMT PROT LOCK 
volume-48e85f60-72a0-41f3-bbaf-63960d61d67a                                                        2 GiB volumes/volume-95116b3a-6aed-4b80-8b97-05831f361389   2           
volume-48e85f60-72a0-41f3-bbaf-63960d61d67a 2 GiB volumes/volume-95116b3a-6aed-4b80-8b97-05831f361389   2 yes       
volume-511f11b1-1093-4c5f-ae05-1e8a4a8f187a                                                        2 GiB                                                                                                              2           
volume-66078ab5-a1c8-4920-86bc-4e9e79d51bb4                                                        2 GiB                                                                                                              2           
volume-66078ab5-a1c8-4920-86bc-4e9e79d51bb4@snapshot-61366e9e-ff23-41d7-87bf-3361f37b4645          1 GiB                                                                                                              2 yes       
volume-7ed0860e-46b2-400d-ba92-cf3422eee35c                                                        2 GiB                                                                                                              2           
volume-7ed0860e-46b2-400d-ba92-cf3422eee35c 2 GiB                                                                                                              2 yes       
volume-95116b3a-6aed-4b80-8b97-05831f361389                                                        2 GiB volumes/volume-7ed0860e-46b2-400d-ba92-cf3422eee35c   2           
volume-95116b3a-6aed-4b80-8b97-05831f361389 2 GiB volumes/volume-7ed0860e-46b2-400d-ba92-cf3422eee35c   2 yes       
volume-cd5126a8-3bd5-47a6-9b9f-67ff33fb5ba8                                                        2 GiB volumes/volume-dcb1a9d7-cd1e-4829-afc7-910763339c79   2           
volume-dcb1a9d7-cd1e-4829-afc7-910763339c79                                                        2 GiB volumes/volume-fe9e40e6-9b3b-48b7-b11b-5229628bfc5a   2           
volume-dcb1a9d7-cd1e-4829-afc7-910763339c79 2 GiB volumes/volume-fe9e40e6-9b3b-48b7-b11b-5229628bfc5a   2 yes       
volume-fe9e40e6-9b3b-48b7-b11b-5229628bfc5a                                                        2 GiB volumes/volume-48e85f60-72a0-41f3-bbaf-63960d61d67a   2           
volume-fe9e40e6-9b3b-48b7-b11b-5229628bfc5a 2 GiB volumes/volume-48e85f60-72a0-41f3-bbaf-63960d61d67a   2 yes




Actual results: 
Flatten occurs on on x+1 depth.


Expected results:
This used to working fine on 16.1, see another verification where flatten worked as expected on depth 5. 
https://bugzilla.redhat.com/show_bug.cgi?id=1888011#c17



Additional info:
I'll recreate this and confirm whether it's still an issue or was a fluke. 
If it still happens, it might be related to recent work done on RBD clone v2.

Comment 1 Tzach Shefi 2021-04-12 07:28:00 UTC
Created attachment 1771308 [details]
CInder c-vol log

Comment 2 Eric Harney 2021-04-12 14:51:26 UTC
Here's what's happening here:

_get_clone_depth() returns 0 for a volume w/ no parent.  1 for the first cloned volume, etc.

Depth | Volume
0 |a
1 |+> b
2 |   +> c
3 |      +> d
4 |         +> e
5 |            +> f



When cloning f, we call get_clone_depth(e) which returns 4
When cloning g, we call get_clone_depth(f) which returns 5 and passes
"depth >= self.configuration.rbd_max_clone_depth", triggering the flatten call in the driver.

This means that the total depth of chain is 6 instead of 5 for f, which is maybe not what would be expected based on the option description.

But, it does match this description in the comments for create_cloned_volume() in the RBD driver:
"If a clone is made of another clone and that clone has rbd_max_clone_depth clones behind it, the dest volume will be flattened."

We could put some thought into how to best clean this up but for now I'm marking this bug as low severity since this difference is negligible for most users.

Comment 3 Luigi Toscano 2021-04-12 15:26:21 UTC
(In reply to Eric Harney from comment #2)

> But, it does match this description in the comments for
> create_cloned_volume() in the RBD driver:
> "If a clone is made of another clone and that clone has rbd_max_clone_depth
> clones behind it, the dest volume will be flattened."
> 
> We could put some thought into how to best clean this up but for now I'm
> marking this bug as low severity since this difference is negligible for
> most users.

Fix the documentation of the option from the current value to something else?

    cfg.IntOpt('rbd_max_clone_depth',
               default=5,
               help='Maximum number of nested volume clones that are '
                    'taken before a flatten occurs. Set to 0 to disable '
                    'cloning. Note: lowering this value will not affect '
                    'existing volumes whose clone depth exceeds the new '
                    'value.'),

Comment 4 Tzach Shefi 2021-04-13 07:47:55 UTC
Created attachment 1771549 [details]
c-vol.log  retest 16.2 rbd clone v1

As I'd already spent the time deploying a system,
below are the results of a 16.2 clone v1.

A consistent result, flatten only occurs when we clone volF to volG.
Max_depth was left at default of 5. 
 

Create volA
(overcloud) [stack@undercloud-0 ~]$ cinder create 2 --name volA
+--------------------------------+--------------------------------------+
| Property                       | Value                                |
+--------------------------------+--------------------------------------+
| attachments                    | []                                   |
| availability_zone              | nova                                 |
| bootable                       | false                                |
| consistencygroup_id            | None                                 |
| created_at                     | 2021-04-13T06:58:37.000000           |
| description                    | None                                 |
| encrypted                      | False                                |
| id                             | 0a350775-48a8-4cc2-906f-e3e886bd6562 |
| metadata                       | {}                                   |
| migration_status               | None                                 |
| multiattach                    | False                                |
| name                           | volA                                 |
| os-vol-host-attr:host          | None                                 |
| os-vol-mig-status-attr:migstat | None                                 |
| os-vol-mig-status-attr:name_id | None                                 |
| os-vol-tenant-attr:tenant_id   | c9fa0f4eba444d34a078a09412992436     |
| replication_status             | None                                 |
| size                           | 2                                    |
| snapshot_id                    | None                                 |
| source_volid                   | None                                 |
| status                         | creating                             |
| updated_at                     | None                                 |
| user_id                        | 908708b360fc4eeeabf08b3fc741eb36     |
| volume_type                    | tripleo                              |
+--------------------------------+--------------------------------------+

Filled volA with random data.

Clone volA to volB
(overcloud) [stack@undercloud-0 ~]$ cinder create 2 --source-volid 0a350775-48a8-4cc2-906f-e3e886bd6562 --name volB
+--------------------------------+--------------------------------------+
| Property                       | Value                                |
+--------------------------------+--------------------------------------+
| attachments                    | []                                   |
| availability_zone              | nova                                 |
| bootable                       | false                                |
| consistencygroup_id            | None                                 |
| created_at                     | 2021-04-13T07:21:42.000000           |
| description                    | None                                 |
| encrypted                      | False                                |
| id                             | b218cf8a-ca47-4393-8ae9-1535876f61ee |
| metadata                       | {}                                   |
| migration_status               | None                                 |
| multiattach                    | False                                |
| name                           | volB                                 |
| os-vol-host-attr:host          | None                                 |
| os-vol-mig-status-attr:migstat | None                                 |
| os-vol-mig-status-attr:name_id | None                                 |
| os-vol-tenant-attr:tenant_id   | c9fa0f4eba444d34a078a09412992436     |
| replication_status             | None                                 |
| size                           | 2                                    |
| snapshot_id                    | None                                 |
| source_volid                   | 0a350775-48a8-4cc2-906f-e3e886bd6562 |
| status                         | creating                             |
| updated_at                     | None                                 |
| user_id                        | 908708b360fc4eeeabf08b3fc741eb36     |
| volume_type                    | tripleo                              |
+--------------------------------+--------------------------------------+

Clone volB to volC
(overcloud) [stack@undercloud-0 ~]$ cinder create 2 --source-volid b218cf8a-ca47-4393-8ae9-1535876f61ee --name volC
+--------------------------------+--------------------------------------+
| Property                       | Value                                |
+--------------------------------+--------------------------------------+
| attachments                    | []                                   |
| availability_zone              | nova                                 |
| bootable                       | false                                |
| consistencygroup_id            | None                                 |
| created_at                     | 2021-04-13T07:22:27.000000           |
| description                    | None                                 |
| encrypted                      | False                                |
| id                             | 3aaba2c8-47ad-437b-9d35-62fbfdfbb601 |
| metadata                       | {}                                   |
| migration_status               | None                                 |
| multiattach                    | False                                |
| name                           | volC                                 |
| os-vol-host-attr:host          | None                                 |
| os-vol-mig-status-attr:migstat | None                                 |
| os-vol-mig-status-attr:name_id | None                                 |
| os-vol-tenant-attr:tenant_id   | c9fa0f4eba444d34a078a09412992436     |
| replication_status             | None                                 |
| size                           | 2                                    |
| snapshot_id                    | None                                 |
| source_volid                   | b218cf8a-ca47-4393-8ae9-1535876f61ee |
| status                         | creating                             |
| updated_at                     | None                                 |
| user_id                        | 908708b360fc4eeeabf08b3fc741eb36     |
| volume_type                    | tripleo                              |
+--------------------------------+--------------------------------------+

VolC to volD
(overcloud) [stack@undercloud-0 ~]$ cinder create 2 --source-volid 3aaba2c8-47ad-437b-9d35-62fbfdfbb601 --name volD
+--------------------------------+--------------------------------------+
| Property                       | Value                                |
+--------------------------------+--------------------------------------+
| attachments                    | []                                   |
| availability_zone              | nova                                 |
| bootable                       | false                                |
| consistencygroup_id            | None                                 |
| created_at                     | 2021-04-13T07:23:59.000000           |
| description                    | None                                 |
| encrypted                      | False                                |
| id                             | 478dfac3-12b7-4a0a-8a3e-7c221f6bc40d |
| metadata                       | {}                                   |
| migration_status               | None                                 |
| multiattach                    | False                                |
| name                           | volD                                 |
| os-vol-host-attr:host          | None                                 |
| os-vol-mig-status-attr:migstat | None                                 |
| os-vol-mig-status-attr:name_id | None                                 |
| os-vol-tenant-attr:tenant_id   | c9fa0f4eba444d34a078a09412992436     |
| replication_status             | None                                 |
| size                           | 2                                    |
| snapshot_id                    | None                                 |
| source_volid                   | 3aaba2c8-47ad-437b-9d35-62fbfdfbb601 |
| status                         | creating                             |
| updated_at                     | None                                 |
| user_id                        | 908708b360fc4eeeabf08b3fc741eb36     |
| volume_type                    | tripleo                              |
+--------------------------------+--------------------------------------+

VolD to volE
(overcloud) [stack@undercloud-0 ~]$ cinder create 2 --source-volid 478dfac3-12b7-4a0a-8a3e-7c221f6bc40d --name volE
+--------------------------------+--------------------------------------+
| Property                       | Value                                |
+--------------------------------+--------------------------------------+
| attachments                    | []                                   |
| availability_zone              | nova                                 |
| bootable                       | false                                |
| consistencygroup_id            | None                                 |
| created_at                     | 2021-04-13T07:25:11.000000           |
| description                    | None                                 |
| encrypted                      | False                                |
| id                             | ccdef36b-a837-4eaf-8d23-033eb8b5960c |
| metadata                       | {}                                   |
| migration_status               | None                                 |
| multiattach                    | False                                |
| name                           | volE                                 |
| os-vol-host-attr:host          | None                                 |
| os-vol-mig-status-attr:migstat | None                                 |
| os-vol-mig-status-attr:name_id | None                                 |
| os-vol-tenant-attr:tenant_id   | c9fa0f4eba444d34a078a09412992436     |
| replication_status             | None                                 |
| size                           | 2                                    |
| snapshot_id                    | None                                 |
| source_volid                   | 478dfac3-12b7-4a0a-8a3e-7c221f6bc40d |
| status                         | creating                             |
| updated_at                     | None                                 |
| user_id                        | 908708b360fc4eeeabf08b3fc741eb36     |
| volume_type                    | tripleo                              |
+--------------------------------+--------------------------------------+

VolE to volF
(overcloud) [stack@undercloud-0 ~]$ cinder create 2 --source-volid ccdef36b-a837-4eaf-8d23-033eb8b5960c --name volF
+--------------------------------+--------------------------------------+
| Property                       | Value                                |
+--------------------------------+--------------------------------------+
| attachments                    | []                                   |
| availability_zone              | nova                                 |
| bootable                       | false                                |
| consistencygroup_id            | None                                 |
| created_at                     | 2021-04-13T07:26:07.000000           |
| description                    | None                                 |
| encrypted                      | False                                |
| id                             | a71290e0-3af3-4a48-89ce-459f995d039f |
| metadata                       | {}                                   |
| migration_status               | None                                 |
| multiattach                    | False                                |
| name                           | volF                                 |
| os-vol-host-attr:host          | None                                 |
| os-vol-mig-status-attr:migstat | None                                 |
| os-vol-mig-status-attr:name_id | None                                 |
| os-vol-tenant-attr:tenant_id   | c9fa0f4eba444d34a078a09412992436     |
| replication_status             | None                                 |
| size                           | 2                                    |
| snapshot_id                    | None                                 |
| source_volid                   | ccdef36b-a837-4eaf-8d23-033eb8b5960c |
| status                         | creating                             |
| updated_at                     | None                                 |
| user_id                        | 908708b360fc4eeeabf08b3fc741eb36     |
| volume_type                    | tripleo                              |
+--------------------------------+--------------------------------------+

VolF to volG  
(overcloud) [stack@undercloud-0 ~]$ cinder create 2 --source-volid a71290e0-3af3-4a48-89ce-459f995d039f --name volG
+--------------------------------+--------------------------------------+
| Property                       | Value                                |
+--------------------------------+--------------------------------------+
| attachments                    | []                                   |
| availability_zone              | nova                                 |
| bootable                       | false                                |
| consistencygroup_id            | None                                 |
| created_at                     | 2021-04-13T07:26:52.000000           |
| description                    | None                                 |
| encrypted                      | False                                |
| id                             | def3bcc1-cb19-4a0a-8d7a-d88836990afe |
| metadata                       | {}                                   |
| migration_status               | None                                 |
| multiattach                    | False                                |
| name                           | volG                                 |
| os-vol-host-attr:host          | None                                 |
| os-vol-mig-status-attr:migstat | None                                 |
| os-vol-mig-status-attr:name_id | None                                 |
| os-vol-tenant-attr:tenant_id   | c9fa0f4eba444d34a078a09412992436     |
| replication_status             | None                                 |
| size                           | 2                                    |
| snapshot_id                    | None                                 |
| source_volid                   | a71290e0-3af3-4a48-89ce-459f995d039f |
| status                         | creating                             |
| updated_at                     | None                                 |
| user_id                        | 908708b360fc4eeeabf08b3fc741eb36     |
| volume_type                    | tripleo                              |
+--------------------------------+--------------------------------------+

And only now we get the flattening, max-depth was left at default of 5 


[root@controller-2 ~]# tail -f /var/log/containers/cinder/cinder-volume.log | grep flatten
2021-04-13 07:26:54.411 46 INFO cinder.volume.drivers.rbd [req-c44ac234-b80d-45c5-9f84-795a122c490b 908708b360fc4eeeabf08b3fc741eb36 c9fa0f4eba444d34a078a09412992436 - default default] maximum clone depth (5) has been reached - flattening dest volume
2021-04-13 07:26:54.505 46 DEBUG cinder.volume.drivers.rbd [req-c44ac234-b80d-45c5-9f84-795a122c490b 908708b360fc4eeeabf08b3fc741eb36 c9fa0f4eba444d34a078a09412992436 - default default] flattening dest volume volume-def3bcc1-cb19-4a0a-8d7a-d88836990afe create_cloned_volume /usr/lib/python3.6/site-packages/cinder/volume/drivers/rbd.py:737


What still bothers me is why on 16.1:
https://bugzilla.redhat.com/show_bug.cgi?id=1888011#c17
I noticed flattening when I cloned volE to volF. 

Either 16.1 and 16.2 don't work the same, 
Or I had a problem when I tested 16.1. 

To put my mind as rest, I'll retest 16.1 again and report back.

Comment 5 Tzach Shefi 2021-04-13 10:41:32 UTC
Per 16.1 it works the same as 16.2,
I just retested it flattening happens during cloning VolF to volG

Why on bz1888011 it happened on VolE to volF, 
Maybe in that case VolA was created from an image, 
that's the only logical explanation I can think of.


Note You need to log in before you can comment on or make changes to this bug.