Bug 883492 - Cron cartridge does not prevent subsequent runs of particular frequency if a job is already running for that frequency
Summary: Cron cartridge does not prevent subsequent runs of particular frequency if a ...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: OKD
Classification: Red Hat
Component: Containers
Version: 2.x
Hardware: Unspecified
OS: Linux
medium
medium
Target Milestone: ---
: ---
Assignee: Paul Morie
QA Contact: libra bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2012-12-04 17:39 UTC by Paul Morie
Modified: 2015-05-14 23:03 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2012-12-19 19:26:32 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Paul Morie 2012-12-04 17:39:55 UTC
Description of problem:
The cron cartridge does not implement any type of locking.  Therefore, if a job for frequency X is already running and it's time to start another job, the second job will be started.  The solution is to implement locking for each frequency, and skip runs where the lock cannot be obtained.

Version-Release number of selected component (if applicable):
2.0.20

How reproducible:
Always

Steps to Reproduce:
1. Create an application and embed the cron cartridge.
2. Create a minutely job that runs for 2 minutes and marks its end in a log file in the application's dir (sample code below).

  
Actual results:

End of job should be marked in log file.


Expected results:

End of job is not marked in log file; subsequent runs overwrite log file.

Additional info:

Comment 1 Paul Morie 2012-12-04 17:43:12 UTC
Test job code (designed to work with a ruby app):

==========

#!/bin/bash

date >> $OPENSHIFT_RUBY_LOG_DIR/test.log
sleep 120
echo 'test job complete!' >> $OPENSHIFT_RUBY_LOG_DIR/test.log

==========

This should go in the app's .openshift/cron/minutely and have the executable bit set.

Comment 2 Paul Morie 2012-12-04 18:53:51 UTC
I've introduced locking that will result in a skipped run for any frequency if a job for that frequency is already running.  A skipped run will result in a message like so in /var/log/messages:

:SKIPPED: minutely cron run for openshift user '53e8fde8be0742d1bb42cad10da24345'

Comment 3 Paul Morie 2012-12-05 14:19:30 UTC
Fix is present in devenv AMI 2556.

Comment 4 Jianwei Hou 2012-12-06 02:44:34 UTC
Verified on devenv_2558

Steps:
1. Create one app and embed cron cartridge
2. Create a minutely job that runs for 2 minutes and marks its end in a log file in the application's dir (sample code below).

#!/bin/bash

date >> $OPENSHIFT_PHP_LOG_DIR/test.log
sleep 120
echo 'test job complete!' >> $OPENSHIFT_PHP_LOG_DIR/test.log

3. Git push

Check locking is enabled for cron cartridge
On app's gear:
[php1-2558t.dev.rhcloud.com logs]\> tailf test.log 
Wed Dec  5 21:39:03 EST 2012
test job complete!

On instance, a skipped run is logged
Dec  5 21:43:02 ip-10-151-4-201 root[31265]: user-cron-jobs Wed Dec  5 21:43:02 EST 2012: :START: minutely cron run for openshift user 'ef037b3bb9494ffca77423bfdae2fc84'
Dec  5 21:43:02 ip-10-151-4-201 root[31275]: user-cron-jobs Wed Dec  5 21:43:02 EST 2012: :SKIPPED: minutely cron run for openshift user 'ef037b3bb9494ffca77423bfdae2fc84'


Note You need to log in before you can comment on or make changes to this bug.