Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1427765

Summary: Database Size Management for Red Hat Enterprise Linux OpenStack Platform Glance
Product: Red Hat OpenStack Reporter: Nilesh <nchandek>
Component: openstack-glanceAssignee: Cyril Roelandt <cyril>
Status: CLOSED WONTFIX QA Contact: Avi Avraham <aavraham>
Severity: medium Docs Contact:
Priority: medium    
Version: 9.0 (Mitaka)CC: aavraham, cyril, egafford, eglynn, fpercoco, nchandek, pgrist, scohen, srevivo, tvignaud
Target Milestone: ---   
Target Release: 12.0 (Pike)   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-11-08 18:15:11 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1399026    

Description Nilesh 2017-03-01 07:03:34 UTC
Description of problem:

In order to automate this command, it should run within the scope of a cron job on at least an hourly basis. And this should automatically build during Director installation.

Red Hat Enterprise Linux OpenStack Platform uses a MariaDB database in the OpenStack control plane for data persistence. This database acts as a system backend, storing configuration and historical data for certain OpenStack services, including Compute, Identity Service, and Block Storage.


Version-Release number of selected component (if applicable):

OSP 7
OSP 8
OSP 9 

++++++++++++++++++++++++++++
Image Service (Glance)
++++++++++++++++++++++++++++

~~~~~~~~~~
Glance also makes use of “soft-deleted” rows, however currently these can only be removed using manual SQL commands or custom scripting. 
~~~~~~~~~~

Comment 1 Red Hat Bugzilla Rules Engine 2017-03-01 14:30:15 UTC
This bugzilla has been removed from the release and needs to be reviewed and Triaged for another Target Release.

Comment 2 Nilesh 2017-03-02 06:08:43 UTC
Elise, Thanks for the confirmation.

Comment 3 Cyril Roelandt 2017-03-03 15:13:13 UTC
(In reply to Nilesh from comment #0)
> Description of problem:
> 
> In order to automate this command


Which command? I have no idea what this bug is about.

Comment 4 Nilesh 2017-03-06 07:45:59 UTC
Hello, 

There is no command as of now for Glance to automate this feature. 


As en example have a look below for Keystone and Nova :- 

++++++++++++++++++++++++++++
Identity Service (Keystone)
++++++++++++++++++++++++++++

~~~~~~~~~~
/usr/bin/keystone-manage token_flush

Example: 0 */1 * * * /usr/bin/keystone-manage token_flush >/dev/null 2>&1
~~~~~~~~~~

++++++++++++++++++++++++++++
Compute Service (Nova)
++++++++++++++++++++++++++++

~~~~~~~~~~
/usr/bin/nova-manage db archive_deleted_rows

Example: 0 */12 * * * /usr/bin/nova-manage db archive_deleted_rows >/dev/null 2>&1
~~~~~~~~~~


In the same manner thinking about Glance, if we can achieve this as Customer is specifically asking for Glance. 



++++++++++++++++++++++++++++
Image Service (Glance)
++++++++++++++++++++++++++++

~~~~~~~~~~
Glance also makes use of “soft-deleted” rows, however currently these can only be removed using manual SQL commands or custom scripting. 
~~~~~~~~~~

Comment 5 Cyril Roelandt 2017-03-06 11:01:15 UTC
So, isn't this the functionality provided by "glance-manage db purge"?

Comment 6 Nilesh 2017-03-06 11:33:40 UTC
I am not very much sure, but Yes, its provided by "glance-manage db purge"


we want a way to automate this command, it should run within the scope of a cron job on at least an hourly basis. And this should automatically build during Director installation.

Comment 7 Avi Avraham 2017-03-06 12:49:52 UTC
(In reply to Nilesh from comment #6)
> I am not very much sure, but Yes, its provided by "glance-manage db purge"
> 
> 
> we want a way to automate this command, it should run within the scope of a
> cron job on at least an hourly basis. And this should automatically build
> during Director installation.

This feature was recently added to glance in RHOS11. The following command will delete any rows from glance DB of images records that was deleted from the system. 
# glance-manage db purge --age_in_days 0

Comment 8 Nilesh 2017-03-06 14:28:59 UTC
Can we have the same functionality on OSP9 as well.

Comment 9 Cyril Roelandt 2017-03-07 19:41:28 UTC
So Avi says "glance-manage db purge" is new in OSP11, but I think it was already there in OSP9 (Mitaka): https://github.com/openstack/glance/blob/stable/mitaka/glance/cmd/manage.py#L154 . What is new in OSP11 is the ability to purge rows created less than a day ago.

@Nilesh: Could you try this feature and tell us whether it is what you're looking for? If so, we could ask the triple-o team if it would make sense to add a call to this command in the crontab.

Comment 10 Nilesh 2017-03-08 06:31:12 UTC
hello Cyril, 

Could you please share exact command to purge so I can try on my packstack setup to test it.

Comment 11 Cyril Roelandt 2017-03-08 19:18:39 UTC
Sure, you could try:

glance-manage db purge --age_in_days 1

Comment 12 Avi Avraham 2017-03-09 06:58:56 UTC
I recommend to to run the following command 
# glance-manage db purge --age_in_days 0
running "--age_in_days 1" will delete rows created 24 hours ago so it is not logical to run hourly .

Comment 13 Nilesh 2017-03-09 07:10:01 UTC
Hello Avi, 

Running "glance-manage --debug db purge --age_in_days 0"

Got bellow output :- 

Option "verbose" from group "DEFAULT" is deprecated for removal.  Its value may be silently ignored in the future.
Must supply a positive, non-zero value for age.
You have new mail in /var/spool/mail/root


And running "glance-manage --debug db purge --age_in_days 1"

Got :- 

Option "verbose" from group "DEFAULT" is deprecated for removal.  Its value may be silently ignored in the future.
/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py:1171: OsloDBDeprecationWarning: EngineFacade is deprecated; please use oslo_db.sqlalchemy.enginefacade
  expire_on_commit=expire_on_commit, _conf=conf)



Can we automate this command, it should run within the scope of a cron job on at least an hourly basis. And this should automatically build during Director installation.

Master bug already open for other components as well. 

https://bugzilla.redhat.com/show_bug.cgi?id=1399026

https://bugzilla.redhat.com/show_bug.cgi?id=1427766

Comment 14 Cyril Roelandt 2017-03-09 14:00:36 UTC
@Avi: I think that "0" can only be passed in OSP11 :)

@Nilesh: why "at least on an hourly basis"? Why wouldn't it be OK to run this once a day? Anyway, I think this should be retargeted to triple-o, since this is where the cron entry should be added. WDYT?

Comment 15 Nilesh 2017-03-09 14:08:39 UTC
@Cyril, Yeah agree with you,  It should be once a day.

Comment 16 Sean Cohen 2017-03-15 13:39:59 UTC
Avi,
Can we please test it in OSP12 and report actual status in this release?
Sean

Comment 17 Avi Avraham 2017-03-20 07:38:42 UTC
(In reply to Sean Cohen from comment #16)
> Avi,
> Can we please test it in OSP12 and report actual status in this release?
> Sean

Just to be clear. Do I need to test if the feature works on RHOS12 or if it's can be run as schedule task ?

Comment 18 Cyril Roelandt 2017-03-20 11:49:04 UTC
I agree with Avi. This feature is currently being tested in OSP11. Adding a cron entry is a job for triple-o, or should be done by the user. There is not much to test here: we know that "cron" works well, and the "db purge" feature is already being taken care of.

Comment 19 Nilesh 2017-03-20 13:10:07 UTC
Thanks @Avi and @Cyril, Customer is curious to add this cron automatically during director installation.

Comment 21 Cyril Roelandt 2017-04-06 13:55:00 UTC
We could convert this to an OOO bug that would be assigned to Erno, but I still think it is not a really good idea. Most users probably do not want that. Nilesh's customer should ask their own sysadmins to add the cron entry. Or maybe this could somehow be disabled by default?

Comment 22 Paul Grist 2017-04-06 14:21:25 UTC
Good points.  I think the full context in the the original bug filed on multiple components https://bugzilla.redhat.com/show_bug.cgi?id=1399026 

I like the disabled by default idea.