Bug 747759

Summary: @reboot no longer works in user crontabs
Product: [Fedora] Fedora Reporter: Elliott Forney <elliott.forney>
Component: cronieAssignee: Marcela Mašláňová <mmaslano>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 15CC: mmaslano, pertusus, tmraz
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-04-24 23:38:57 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Elliott Forney 2011-10-20 23:42:53 UTC
Description of problem:

Cronjobs that use the @reboot extension (man -s 5 crontab) no longer work in fedora 15.

Version-Release number of selected component (if applicable):

cronie-1.4.8-2.fc15.x86_64

How reproducible:

Always.

Steps to Reproduce:
1. crontab -e
2. @reboot /bin/touch /tmp/foo
3. shutdown -r now
4. ls -ld /tmp/foo
  
Actual results:

Job was not run.

Expected results:

Job should be run after crond starts.

Additional info:

This works find in Fedora 14 but is broken in Fedora 15.

Comment 1 Elliott Forney 2011-10-20 23:43:23 UTC
Looks like crond is run with -n in Fedora 15 but not in 14.  I wonder if this is related?

Comment 2 Tomas Mraz 2011-10-21 08:33:51 UTC
What do you see in /var/log/cron when you reboot the computer?

Comment 3 Elliott Forney 2011-10-21 10:19:45 UTC
Nothing, until the next cron job runs.

Comment 4 Tomas Mraz 2011-10-21 10:59:19 UTC
I cannot reproduce the problem on Fedora 16 beta. Are you sure that the @reboot job is correctly set in a crontab?

Can you attach the crontab you have the @reboot job in?

Comment 5 Tomas Mraz 2011-10-21 14:08:03 UTC
I cannot reproduce it on Fedora 15 either.

Comment 6 Elliott Forney 2011-10-21 20:59:20 UTC
Here is the exact entry I am putting in my crontab:
@reboot /bin/touch /tmp/foo

So, it looks like it works fine for local users but fails for users with accounts served via NIS.

Could it be that crond is starting before NIS?  I'm not quite sure how to tell... I'm still getting used to systemd.

Comment 7 Tomas Mraz 2011-10-24 06:37:35 UTC
Yes, that's possible. Do you see any ORPHAN messages in /var/log/cron?

Comment 8 Elliott Forney 2011-10-25 16:48:52 UTC
I do not see any ORPHAN messages.

I did, however, try creating a crontab in /var/spool/cron with a non-existant UID and GID and also don't see any ORPHAN messages during reboot, regardless of whether it is an @reboot or ordinary cron entry.

I do see ORPHAN messages when cron is manually restarted though.  Looks like ORPHAN messages are only generated when the crontab is manually reloaded, not at reboot?

More likely, crond is also starting before syslog.

I was able to generate a systemd dependency graph with

systemctl --order dot | dot -Tsvg > /tmp/systemd.svg

but it is too complex to really read.  I wonder if there is an easy way to just tell if one service starts before another... systemd is pretty cool but I kinda miss rc scripts right now.

Comment 9 Tomas Mraz 2011-10-25 17:22:36 UTC
Hmm that looks like the messages from startup of crond on reboot are lost or more probably redirected to /var/log/messages. Unfortunately this logging problem is caused by migration to systemd and I do not think the solution is in crond. Either the problem is in systemd itself or in the rsyslog.

Comment 10 Tomas Mraz 2011-10-25 17:27:47 UTC
Also to me it seems the crond service should be put at the latest end of the boot process but due to the parallel nature of the systemd boot it would be really hard to ensure that it is always started after the nis/ldap/ipa or whatever else network account service is started. The problem with orphaned regular jobs was solved by the orphan adoptions, but the @reboot jobs cannot be run this way because the orphans can be adopted in some cases very late when running eventual @reboot jobs would not be desirable.

The only solution I see for now is to document this problem at the same place the special @reboot keyword is documented.

Comment 11 Elliott Forney 2011-10-25 17:36:42 UTC
It may be a good idea to document this problem in the man page but can't we also start crond later in the boot process?  That's why systemd has targets right?

Is there a systemd guru we can run this by?

Comment 12 Elliott Forney 2011-10-25 17:50:24 UTC
After modifying the following in 

[Unit]
Description=Command Scheduler
After=syslog.target network.target

I now see the ORPHANED messages at reboot.  Not sure why this is, the After=syslog.target was already in there, so why do we need After=network.target to log messages?

This does not, however, fix my NIS problem.  Unfortunately ypbind is still started with SysV scripts so I'm not sure how to force crond to start after it.

And what happens if it takes a second for ypbind to start working?

Comment 13 Elliott Forney 2011-10-25 17:51:51 UTC
Oh, the file I am playing with is /lib/systemd/system/crond.service which does belong to the cronie package.

Comment 14 Tomas Mraz 2011-10-25 18:10:13 UTC
It does belong to the cronie package. I see it in the spec file of the F15 package and I verified it on my F16 install.

I'd just say do not use @reboot for non-local users at least until someone comes up with some clever way how to distinguish orphans from not-yet-running network account services and orphans of other kind (network down on reboot because a laptop is started disconnected and then connected a hour or two later).

Comment 15 Tomas Mraz 2011-10-25 18:12:57 UTC
Of course we could say if orphan comes up less than ??? minutes after the startup then run @reboot jobs for him. But this seems to be too vague to me.

Comment 16 Elliott Forney 2011-10-25 18:58:06 UTC
Thanks, I agree.

I might mention again, however, that it works fine for me in Fedora 14, even if there are no guarantees that network accounts will be available.  I'm pretty sure that the integration of systemd started the problems and that starting cron after NIS would fix the particular problem that I am seeing.

Comment 17 Elliott Forney 2011-10-25 19:00:21 UTC
Is ypbind in systemd in Fedora 16 or is it still a legacy script?

Comment 18 Tomas Mraz 2011-10-25 19:22:33 UTC
It was migrated to a native systemd unit in F16.

We could probably add After=ypbind.service sssd.service to the crond unit file as a workaround. Does it help for you?

Comment 19 Elliott Forney 2011-10-25 19:32:31 UTC
Yes, actually, that solves the problem for me.  Thanks!

Comment 20 Elliott Forney 2011-10-25 19:56:11 UTC
In fact, just ypbind.service seems to be enough.  Since After=network.service solves the problem of not seeing the ORPHAN messages at reboot I would recommend

After=syslog.target network.service ypbind.service

Although I still don't get why syslog.target isn't enough.

Comment 21 Fedora Update System 2011-10-25 20:47:37 UTC
cronie-1.4.8-4.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/FEDORA-2011-14523

Comment 22 Fedora Update System 2011-10-25 21:00:04 UTC
cronie-1.4.8-9.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/cronie-1.4.8-9.fc16

Comment 23 Fedora Update System 2011-10-26 19:04:30 UTC
Package cronie-1.4.8-9.fc16:
* should fix your issue,
* was pushed to the Fedora 16 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing cronie-1.4.8-9.fc16'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2011-14947
then log in and leave karma (feedback).

Comment 24 Elliott Forney 2011-11-10 21:19:27 UTC
This solved the problem for us.  I'm not sure, however, that this will solve the issue for other nameservices, e.g. ldap.  Thanks!

Comment 25 Fedora Update System 2012-01-11 06:03:35 UTC
cronie-1.4.8-4.fc15 has been pushed to the Fedora 15 stable repository.  If problems still persist, please make note of it in this bug report.