Bug 1703091 - Since we're using nova_cell0 instead of nova database, nova-manage db archive_deleted_rows is no longer archiving deleted rows
Summary: Since we're using nova_cell0 instead of nova database, nova-manage db archive...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-nova
Version: 13.0 (Queens)
Hardware: x86_64
OS: Linux
medium
medium
Target Milestone: Upstream M3
: 16.0 (Train on RHEL 8.1)
Assignee: melanie witt
QA Contact: Paras Babbar
URL:
Whiteboard:
Depends On:
Blocks: 1748990 1749087 1752711 1769458 1778905
TreeView+ depends on / blocked
 
Reported: 2019-04-25 13:39 UTC by David Hill
Modified: 2023-10-06 18:19 UTC (History)
14 users (show)

Fixed In Version: openstack-nova-20.0.1-0.20191025043858.390db63.el8ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1748990 1749087 1752711 (view as bug list)
Environment:
Last Closed: 2020-02-06 14:40:18 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1719487 0 None None None 2019-04-25 15:25:42 UTC
OpenStack gerrit 507486 0 'None' MERGED nova-manage db archive_deleted_rows is not multi-cell aware 2021-02-03 14:46:22 UTC
Red Hat Issue Tracker OSP-29358 0 None None None 2023-10-06 18:19:57 UTC
Red Hat Knowledge Base (Solution) 4088311 0 None None None 2019-04-25 15:10:55 UTC
Red Hat Product Errata RHEA-2020:0283 0 None None None 2020-02-06 14:41:26 UTC

Description David Hill 2019-04-25 13:39:53 UTC
Description of problem:
Since we're using nova_cell0 instead of nova database, nova-manage db archive_deleted_rows is no longer archiving deleted rows

nova-manage db archive_deleted_rows --verbose --max_rows 1000000

Version-Release number of selected component (if applicable):
# rpm -qa|grep nova
python-nova-17.0.5-3.d7864fbgit.el7ost.noarch
openstack-nova-api-17.0.5-3.d7864fbgit.el7ost.noarch
puppet-nova-12.4.0-6.el7ost.noarch
openstack-nova-common-17.0.5-3.d7864fbgit.el7ost.noarch
python2-novaclient-10.1.0-1.el7ost.noarch

How reproducible:
Always

Steps to Reproduce:
1. Get 705k deleted instances
2. run nova-manage db archive_deleted_rows
3.

Actual results:
Nothing is archived

Expected results:
All rows should be achived up to --max_rows

Additional info:

Comment 1 David Hill 2019-04-25 13:54:45 UTC
MariaDB [nova_cell0]> select count(*) from instances where deleted_at is not null;
+----------+
| count(*) |
+----------+
|   708405 |
+----------+

Comment 2 David Hill 2019-04-25 14:40:09 UTC
We do have a workaround on this and it's to use --config-file nova_cell0.conf where we changed the connection string to use nova_cell0 instead of nova.   This is still a bug in my opinion if customers have a big failure rate, that will consume a lot of disk space that could be avoided with the cleanup cronjobs.

Comment 5 chrisbro@redhat.com 2019-07-03 22:13:19 UTC
Hi Team,

Can I get an update with this as this impacting a very important customer?

Comment 6 David Hill 2019-07-04 12:55:49 UTC
Hey Chris,
  
    I created another BZ for another issue but this helps cleaning up common tables but some of them are not cleaned up which I feel like it's another issue.
We do have a workaround which is to set the connection string to nova_cell0 instead of nova and call that configuration using nova-manage.

Thank you very much,
David Hill

Comment 8 melanie witt 2019-08-29 00:48:29 UTC
Upstream patch has merged: https://review.opendev.org/507486

Comment 15 errata-xmlrpc 2020-02-06 14:40:18 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2020:0283


Note You need to log in before you can comment on or make changes to this bug.