Bug 1777906

Same issue as reported in bug 1565460.

*** Bug 1565460 has been marked as a duplicate of this bug. ***

The customer said that they are not yet affected by https://access.redhat.com/solutions/4619831, but given that the described issue may be a direct result of this bug, is there an ETA for this specific BZ?

I see that this wasn't merged upstream, yet:
https://review.opendev.org/#/c/696676/

Can we push for this to be merged and then backport it to OSP 13?

Thanks,

Andreas

Hi,

I'd still like to push for some news.

THanks,

Andreas

Hi Andreas,

The patch is currently under review upstream and has a +2 on it.
When it merges (can't propose an ETA), we will backport it to queens.

Thanks and Regards
Rajat Dhasmana

smykhail: duplicate bug has been closed
akaris: patch has been merged upstream and currently in the upstream backport process to stable branches

Verified on:
openstack-cinder-12.0.10-10.el7ost.noarch

Create a volume:
(overcloud) [stack@undercloud-0 ~]$ cinder create 4 --name Tshefi
+--------------------------------+--------------------------------------+
| Property                       | Value                                |
+--------------------------------+--------------------------------------+
| attachments                    | []                                   |
| availability_zone              | nova                                 |
| bootable                       | false                                |
| consistencygroup_id            | None                                 |
| created_at                     | 2020-06-09T11:32:53.000000           |
| description                    | None                                 |
| encrypted                      | False                                |
| id                             | b760df67-d786-4a00-b1a7-c3b53f57b01a |
| metadata                       | {}                                   |
| migration_status               | None                                 |
| multiattach                    | False                                |
| name                           | Tshefi                               |
| os-vol-host-attr:host          | controller-0@nfs#nfs                 |
| os-vol-mig-status-attr:migstat | None                                 |
| os-vol-mig-status-attr:name_id | None                                 |
| os-vol-tenant-attr:tenant_id   | db6d21ea62474d62811b2354371875ce     |
| replication_status             | None                                 |
| size                           | 4                                    |
| snapshot_id                    | None                                 |
| source_volid                   | None                                 |
| status                         | creating                             |
| updated_at                     | 2020-06-09T11:32:53.000000           |
| user_id                        | 9449585f76de4556b7c31d9d001f4618     |
| volume_type                    | tripleo                              |
+--------------------------------+--------------------------------------+


2. Set metadata on volume:
(overcloud) [stack@undercloud-0 ~]$ cinder metadata b760df67-d786-4a00-b1a7-c3b53f57b01a set a=b


3. Validate metadata set:
(overcloud) [stack@undercloud-0 ~]$ cinder show b760df67-d786-4a00-b1a7-c3b53f57b01a | grep meta
| metadata                       | a : b                                |

4. Manipulate DB, set a "fake" delete date, notice we didn't actuality delete the volume yet:
MariaDB [cinder]> update volumes set deleted_at='2020-03-31 07:29:14' where id='b760df67-d786-4a00-b1a7-c3b53f57b01a';

5. Now lets try to purge record db using:
cinder-manage.log reports the original problem reproduced:

()[root@controller-0 cinder]# cinder-manage db purge 0
Option "logdir" from group "DEFAULT" is deprecated. Use option "log-dir" from group "DEFAULT".
Purge command failed, check cinder-manage logs for more details.

2020-06-09 12:36:47.501 7421 ERROR cinder.db.sqlalchemy.api [req-54759fe4-6136-49dd-8725-be59ab46f6b6 - - - - -] DBError detected when purging from volumes: (pymysql.err.IntegrityError) (1451, u'Cannot delete or update a parent row: a foreign key constraint fails (`cinder`.`volume_metadata`, CONSTRAINT `volume_metadata_ibfk_1` FOREIGN KEY (`volume_id`) REFERENCES `volumes` (`id`))') [SQL: u'DELETE FROM volumes WHERE volumes.deleted_at < %(deleted_at_1)s'] [parameters: {u'deleted_at_1': datetime.datetime(2020, 6, 9, 12, 36, 47, 492991)}] (Background on this error at: http://sqlalche.me/e/gkpj).: DBReferenceError: (pymysql.err.IntegrityError) (1451, u'Cannot delete or update a parent row: a foreign key constraint fails (`cinder`.`volume_metadata`, CONSTRAINT `volume_metadata_ibfk_1` FOREIGN KEY (`volume_id`) REFERENCES `volumes` (`id`))') [SQL: u'DELETE FROM volumes WHERE volumes.deleted_at < %(deleted_at_1)s'] [parameters: {u'deleted_at_1': datetime.datetime(2020, 6, 9, 12, 36, 47, 492991)}] (Background on this error at: http://sqlalche.me/e/gkpj)

And well unsurprisingly as purge failed we retain the volume's DB record:
| 2020-06-09 11:32:53 | 2020-06-09 11:32:58 | 2020-03-31 07:29:14 |       0 | b760df67-d786-4a00-b1a7-c3b53f57b01a | NULL   | 9449585f76de4556b7c31d9d001f4618 | db6d21ea62474d62811b2354371875ce | controller-0@nfs#nfs |    4 | nova              | available | detached      | 2020-06-09 11:32:53 | 2020-06-09 11:32:58 | NULL          | Tshefi       | NULL                | 10.35.160.111:/export/ins_cinder | NULL          | NULL        | ef58deca-4022-47c2-b56e-8d2a5e58f1cc | NULL         |        0 | NULL              | NULL     | NULL              | NULL             | NULL               | NULL                        | NULL                    | NULL                | NULL        |           0 | NULL            | NULL         | NULL     | 2d22950a-7c10-4344-8ec0-6ed789b65369 |              0 |


6. This time delete the volume for real
(overcloud) [stack@undercloud-0 ~]$ cinder delete b760df67-d786-4a00-b1a7-c3b53f57b01a
Request to delete volume b760df67-d786-4a00-b1a7-c3b53f57b01a has been accepted.

7. Now re-purge the DB again:
()[root@controller-0 cinder]# cinder-manage db purge 0
Option "logdir" from group "DEFAULT" is deprecated. Use option "log-dir" from group "DEFAULT".

This time around the purge prunes the DB as expected. 

2020-06-09 12:42:22.126 7497 INFO cinder.db.sqlalchemy.api [req-fe9d5b06-54e4-4592-8560-63c8ed5f0ba4 - - - - -] Deleted 1 rows from table=volumes
2020-06-09 12:42:22.127 7497 INFO cinder.db.sqlalchemy.api [req-fe9d5b06-54e4-4592-8560-63c8ed5f0ba4 - - - - -] Purging deleted rows older than age=0 days from table=volume_types


As expected volume is purged from DB, despite having the metadata set. 


MariaDB [cinder]> MariaDB [cinder]> select * from volumes where id='b760df67-d786-4a00-b1a7-c3b53f57b01a';
Empty set (0.00 sec)

..

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:2722

The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days