Description of problem:
Same as referenced in https://bugzilla.redhat.com/show_bug.cgi?id=1173970 , but this BZ is specific to undercloud node.
Steps to Reproduce:
1. Install an undercloud
2. Do some openstack-related stuff and see your MySQL database growing indefinitely.
Running the token_flush manually reduces the size of the table by flushing all expired tokens.
There should be a cron job that flushes expired tokens every minute:
*/1 * * * * /usr/bin/keystone-manage token_flush >/dev/null 2>&1
*** Bug 1293274 has been marked as a duplicate of this bug. ***
This bug did not make the OSP 8.0 release. It is being deferred to OSP 10.
I'd even consider changing the component to openstack-keystone and having the rpm drop the cron-job into /etc/cron.d as a default.
Or is there a reason a customer would _not_ want the tokens flushed?
*** Bug 1328180 has been marked as a duplicate of this bug. ***
(In reply to David Juran from comment #6)
> I'd even consider changing the component to openstack-keystone and having
> the rpm drop the cron-job into /etc/cron.d as a default.
> Or is there a reason a customer would _not_ want the tokens flushed?
I've asked for that for a long time and the consistent answer has been that the keystone folks won't do that in the rpm. They have a few reasons:
* they're trying to move to a model where they don't need the flush (fernet tokens, iirc) so having it is extra if that option is in use
* The flushing can be db intensive and the operator really needs to determine how often they want/need the flush. Some people might prefer 1 really large flush weekly at a (for them) off-peak time. Others might prefer an extremely aggressive flush ever hour or 2.
I think this is an urgent blocker, I changed the severity accordingly. We keep getting hit by this issue and it's easy for a user to not be aware of this problem, even if the user is a very knowledgeable one.
*** Bug 1347359 has been marked as a duplicate of this bug. ***
The default is set to "purge tokens every 24h". Do you want to reduce the frequency to 1 min? Or something else?
As far as I can recall, the frequency of once every 24 hours is what was decided on, and it can stay the way it is. The problem is that we don't see this cron job created at all on the undercloud. Need to check it on the overcloud as well. What is responsible for creating it?
Puppet is responsible of creating the crontab. Could you verify OSP10 have the cron exist on both undercloud and overcloud? I checked the puppet code and it looks good to me.
to check if the cron job is defined, as root you can run:
sudo -u keystone crontab -l
should see something similar output to:
[root@instack ~]# sudo -u keystone crontab -l
# HEADER: This file was autogenerated at 2016-08-15 12:10:54 +0000 by puppet.
# HEADER: While it can still be managed manually, it is definitely not recommended.
# HEADER: Note particularly that the comments starting with 'Puppet Name' should
# HEADER: not be deleted, as doing so could cause duplicate cron jobs.
# Puppet Name: keystone-manage token_flush
1 0 * * * keystone-manage token_flush >>/dev/null 2>&1
you can also check the /var/spool/cron/keystone file
Verified that it's fixed for the undercloud in OSP10 (puddle 2016-09-20.1).
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.