Bug 1565460 - cinder-manage db purge fails
Summary: cinder-manage db purge fails
Keywords:
Status: CLOSED DUPLICATE of bug 1777906
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-cinder
Version: 10.0 (Newton)
Hardware: All
OS: All
high
high
Target Milestone: ---
: ---
Assignee: Rajat Dhasmana
QA Contact: Avi Avraham
RHOS Documentation Team
URL:
Whiteboard:
Depends On:
Blocks: 1777906 1794693 1796144
TreeView+ depends on / blocked
 
Reported: 2018-04-10 04:47 UTC by Nilesh
Modified: 2020-01-29 17:28 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-12-13 15:58:48 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 696676 0 'None' MERGED Make volume soft delete more thorough 2021-01-29 02:40:46 UTC

Description Nilesh 2018-04-10 04:47:00 UTC
# cinder-manage db purge 300
Option "logdir" from group "DEFAULT" is deprecated. Use option "log-dir" from group "DEFAULT".
Purge command failed, check cinder-manage logs for more details.

# tail /var/log/cinder/cinder-manage.log
2018-03-29 15:39:37.255 454259 ERROR cinder.db.sqlalchemy.api [req-a5a8a546-c6d5-4646-b202-3bf118711594 - - - - -] DBError detected when purging from volumes: (pymysql.err.IntegrityError) (1451, u'Cannot delete or update a parent row: a foreign key constraint fails (`cinder`.`volume_attachment`, CONSTRAINT `volume_attachment_ibfk_1` FOREIGN KEY (`volume_id`) REFERENCES `volumes` (`id`))') [SQL: u'DELETE FROM volumes WHERE volumes.deleted_at < %(deleted_at_1)s'] [parameters: {u'deleted_at_1': datetime.datetime(2017, 6, 2, 13, 39, 37, 226381)}].

The bug is "fixed": https://bugs.launchpad.net/cinder/+bug/1542169
unfortunately the bugfix only fixed the error log message and not the actual problem.


* ERROR
	~~~
	2018-03-29 15:39:30.500 454259 INFO cinder.db.sqlalchemy.api [req-a5a8a546-c6d5-4646-b202-3bf118711594 - - - - -] Purging deleted rows older than age=300 days from table=volume_glance_metadata
	2018-03-29 15:39:30.987 454259 INFO cinder.db.sqlalchemy.api [req-a5a8a546-c6d5-4646-b202-3bf118711594 - - - - -] Purging deleted rows older than age=300 days from table=snapshot_metadata
	2018-03-29 15:39:31.572 454259 INFO cinder.db.sqlalchemy.api [req-a5a8a546-c6d5-4646-b202-3bf118711594 - - - - -] Purging deleted rows older than age=300 days from table=attachment_specs
	2018-03-29 15:39:32.048 454259 INFO cinder.db.sqlalchemy.api [req-a5a8a546-c6d5-4646-b202-3bf118711594 - - - - -] Purging deleted rows older than age=300 days from table=volume_type_projects
	2018-03-29 15:39:32.517 454259 INFO cinder.db.sqlalchemy.api [req-a5a8a546-c6d5-4646-b202-3bf118711594 - - - - -] Purging deleted rows older than age=300 days from table=volume_type_extra_specs
	2018-03-29 15:39:33.101 454259 INFO cinder.db.sqlalchemy.api [req-a5a8a546-c6d5-4646-b202-3bf118711594 - - - - -] Purging deleted rows older than age=300 days from table=volume_metadata
	2018-03-29 15:39:33.578 454259 INFO cinder.db.sqlalchemy.api [req-a5a8a546-c6d5-4646-b202-3bf118711594 - - - - -] Purging deleted rows older than age=300 days from table=volume_attachment
	2018-03-29 15:39:34.170 454259 INFO cinder.db.sqlalchemy.api [req-a5a8a546-c6d5-4646-b202-3bf118711594 - - - - -] Purging deleted rows older than age=300 days from table=volume_admin_metadata
	2018-03-29 15:39:34.644 454259 INFO cinder.db.sqlalchemy.api [req-a5a8a546-c6d5-4646-b202-3bf118711594 - - - - -] Purging deleted rows older than age=300 days from table=transfers
	2018-03-29 15:39:35.110 454259 INFO cinder.db.sqlalchemy.api [req-a5a8a546-c6d5-4646-b202-3bf118711594 - - - - -] Purging deleted rows older than age=300 days from table=snapshots
	2018-03-29 15:39:35.684 454259 INFO cinder.db.sqlalchemy.api [req-a5a8a546-c6d5-4646-b202-3bf118711594 - - - - -] Purging deleted rows older than age=300 days from table=group_volume_type_mapping
	2018-03-29 15:39:36.187 454259 INFO cinder.db.sqlalchemy.api [req-a5a8a546-c6d5-4646-b202-3bf118711594 - - - - -] Purging deleted rows older than age=300 days from table=workers
	2018-03-29 15:39:36.766 454259 INFO cinder.db.sqlalchemy.api [req-a5a8a546-c6d5-4646-b202-3bf118711594 - - - - -] Purging deleted rows older than age=300 days from table=volumes
	2018-03-29 15:39:37.255 454259 ERROR cinder.db.sqlalchemy.api [req-a5a8a546-c6d5-4646-b202-3bf118711594 - - - - -] DBError detected when purging from volumes: (pymysql.err.IntegrityError) (1451, u'Cannot delete or update a parent row: a foreign key constraint fails (`cinder`.`volume_attachment`, CONSTRAINT `volume_attachment_ibfk_1` FOREIGN KEY (`volume_id`) REFERENCES `volumes` (`id`))') [SQL: u'DELETE FROM volumes WHERE volumes.deleted_at < %(deleted_at_1)s'] [parameters: {u'deleted_at_1': datetime.datetime(2017, 6, 2, 13, 39, 37, 226381)}].
	(END)
	~~~

Comment 3 Michael Bayer 2018-04-10 15:34:28 UTC
the issue as described so far seems to be that you are unsatisfied with the use case not being covered from https://bugs.launchpad.net/cinder/+bug/1542169, and you would prefer that they implement option "2. Explicitly delete attachment records" as stated at https://bugs.launchpad.net/cinder/+bug/1542169/comments/1.  This is not a mariadb issue.

Comment 4 Eric Harney 2018-04-10 15:40:03 UTC
It looks like this is or was a bug in Cinder:  the fix for https://bugs.launchpad.net/cinder/+bug/1542169 was not actually a fix, it just generated a better error message.

Comment 12 Eric Harney 2019-04-17 13:27:02 UTC
I think we could fix this by adjusting

https://git.openstack.org/cgit/openstack/cinder/tree/cinder/db/sqlalchemy/api.py?h=14.0.0#n6636

to skip any volumes that still have attachment records.  (Since they aren't supposed to be in that state and it isn't clear what to do with them.)

Issuing a warning for the admin in that case might also be helpful, but not having "cinder-manage" fail would be a good improvement in itself.

Comment 13 Eric Harney 2019-12-13 15:58:48 UTC

*** This bug has been marked as a duplicate of bug 1777906 ***


Note You need to log in before you can comment on or make changes to this bug.