Bug 1476762 - keystone-manage token_flush cron job fails with: expr: division by zero sleep: missing operand
keystone-manage token_flush cron job fails with: expr: division by zero sleep...
Status: CLOSED ERRATA
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates (Show other bugs)
11.0 (Ocata)
Unspecified Unspecified
urgent Severity urgent
: z2
: 11.0 (Ocata)
Assigned To: Emilien Macchi
Prasanth Anbalagan
: Triaged, ZStream
Depends On:
Blocks: 1473713 1478993
  Show dependency treegraph
 
Reported: 2017-07-31 08:07 EDT by Marius Cornea
Modified: 2017-09-13 17:43 EDT (History)
10 users (show)

See Also:
Fixed In Version: openstack-tripleo-heat-templates-6.2.0-3.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1478993 (view as bug list)
Environment:
Last Closed: 2017-09-13 17:43:17 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
OpenStack gerrit 492023 None None None 2017-08-30 11:30 EDT
OpenStack gerrit 493304 None None None 2017-08-30 11:34 EDT

  None (edit)
Description Marius Cornea 2017-07-31 08:07:50 EDT
Description of problem:
keystone-manage token_flush cron job fails with: expr: division by zero sleep: missing operand:

[root@controller-0 heat-admin]# cat /var/spool/cron/keystone 
# HEADER: This file was autogenerated at 2017-07-31 11:50:18 +0000 by puppet.
# HEADER: While it can still be managed manually, it is definitely not recommended.
# HEADER: Note particularly that the comments starting with 'Puppet Name' should
# HEADER: not be deleted, as doing so could cause duplicate cron jobs.
# Puppet Name: keystone-manage token_flush
PATH=/bin:/usr/bin:/usr/sbin SHELL=/bin/sh
1 * * * * sleep `expr ${RANDOM} \% 0`; keystone-manage token_flush >>/var/log/keystone/keystone-tokenflush.log 2>&1
# Puppet Name: cinder-manage db purge
PATH=/bin:/usr/bin:/usr/sbin SHELL=/bin/sh
1 0 * * * cinder-manage db purge 0 >>/var/log/cinder/cinder-rowsflush.log 2>&1

Version-Release number of selected component (if applicable):
openstack-tripleo-heat-templates-6.1.0-2.el7ost.noarch

How reproducible:
100%

Steps to Reproduce:
1. Deploy OSP11
2. Check /var/spool/cron/keystone

Actual results:
# Puppet Name: keystone-manage token_flush
PATH=/bin:/usr/bin:/usr/sbin SHELL=/bin/sh
1 * * * * sleep `expr ${RANDOM} \% 0`; keystone-manage token_flush >>/var/log/keystone/keystone-tokenflush.log 2>&1

Expected results:
No errors.

Additional info:

[root@controller-0 heat-admin]# expr ${RANDOM} \% 0
expr: division by zero
Comment 1 Nathan Kinder 2017-08-03 17:10:56 EDT
I think this may be a problem related to the tripleo-heat-template related to the token flush cronjob (though I am admittedly not an expert on puppet or t-h-t).

The puppet-keystone code is only supposed to add a sleep to the token flush cronjob is maxdelay is a positive integer.  The code that checks for this is here:

    https://github.com/openstack/puppet-keystone/blob/b479685940e27dea5fa86155e3a2ba6878865abc/manifests/cron/token_flush.pp#L70-L74

As you can see in the above code, a maxdelay of 0 should result in $sleep being set to an empty string.  The crontab shown in this bug clearly shows that maxdelay must have been 0, yet $sleep was set.

If you look at the t-h-t code for this, it uses a maxdelay default of 0, but the type is 'string':

    https://github.com/openstack/tripleo-heat-templates/blob/617e5f1022da5309328a1d9c96399c1fedd257ec/puppet/services/keystone.yaml#L180-L184

My theory is that the type is incorrect in t-h-t.  Because it is a string, the evaluation of "if $maxdelay == 0" in puppet-keystone ends up being false since we are comparing the number 0 to the string "0".  I think that the type for KeystoneCronTokenFlushMaxDelay in t-h-t should be 'number'.  I don't have an install currently to personally test this theory, so it would be great if you could try it out to prove/disprove it.
Comment 2 Nathan Kinder 2017-08-30 12:05:01 EDT
The fix to tripleo-heat-templates has merged upstream for stable/ocata:

  https://review.openstack.org/#/c/492023/
  
There is a related change to puppet-keystone to perform better validation which is still under review, but that isn't really needed to fix this issue.  We can consider this fixed with just the t-h-t patch.  For reference, the puppet-keystone review is here:

  https://review.openstack.org/#/c/493304
Comment 3 Nathan Kinder 2017-08-30 12:17:18 EDT
This fix did not make it into the upstream 6.2.0 release of tripleo-heat-templates, which is the latest 6.x release.  We should cherry-pick this downstream for now:

  https://github.com/openstack/tripleo-heat-templates/commit/564e3feeb9c518851afde9ed778360e4032e9213
Comment 5 Prasanth Anbalagan 2017-08-31 12:52:52 EDT
Verified as follows - Sleep is not added to the keystone token-flush cron job when maxdelay is 0.

*****
LOGS
*****
[stack@undercloud-0 ~]$ rpm -qa openstack-tripleo-heat-templates
openstack-tripleo-heat-templates-6.2.0-3.el7ost.noarch

[heat-admin@controller-0 ~]$ sudo  cat /var/spool/cron/keystone 
# HEADER: This file was autogenerated at 2017-08-31 13:54:59 +0000 by puppet.
# HEADER: While it can still be managed manually, it is definitely not recommended.
# HEADER: Note particularly that the comments starting with 'Puppet Name' should
# HEADER: not be deleted, as doing so could cause duplicate cron jobs.
# Puppet Name: keystone-manage token_flush
PATH=/bin:/usr/bin:/usr/sbin SHELL=/bin/sh
1 * * * * keystone-manage token_flush >>/var/log/keystone/keystone-tokenflush.log 2>&1
# Puppet Name: cinder-manage db purge
PATH=/bin:/usr/bin:/usr/sbin SHELL=/bin/sh
1 0 * * * cinder-manage db purge 0 >>/var/log/cinder/cinder-rowsflush.log 2>&1
[heat-admin@controller-0 ~]$
Comment 7 errata-xmlrpc 2017-09-13 17:43:17 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2721

Note You need to log in before you can comment on or make changes to this bug.