Bug 735802

Summary: crond doesn't run after distro-sync from F14 to F15
Product: [Fedora] Fedora Reporter: Ian Donaldson <iand>
Component: cronieAssignee: Marcela Mašláňová <mmaslano>
Status: CLOSED CURRENTRELEASE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 15CC: 98036119lmak, guilde.nt, louis, mmaslano, mschmidt, pertusus, plgs, tmraz
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-10-17 15:12:29 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Ian Donaldson 2011-09-05 13:30:28 UTC
Description of problem:

Just upgraded from FC14 to FC15 using

   yum --releasever=15 --disableplugin=presto distro-sync
   grub-install ... etc

and rebooted; crond no longer running.

Version-Release number of selected component (if applicable):

# rpm -qa |grep cron
cronie-1.4.8-2.fc15.x86_64
cronie-anacron-1.4.8-2.fc15.x86_64
crontabs-1.11-2.20101115git.fc15.noarch

# rpm -qa |grep systemd
systemd-26-9.fc15.x86_64
systemd-sysv-26-9.fc15.x86_64
systemd-units-26-9.fc15.x86_64

How reproducible:

100%.  Did same thing on 3 systems I upgraded.  All desktops.

Steps to Reproduce:
1.  see above
2.
3.
  
Actual results:

ps -ef|grep crond 
shows nothing

Expected results:

crond running

/var/log/cron shows no activity post the upgrade


Additional info:

running selinux in targeted mode.

I'm new to systemd; didn't run it before the upgrade; nor
did I install it.

This:

    systemctl list-units |grep cron

shows now output.

However this:

   systemctl show  cronie.service

shows hundreds of lines which (to me) look normal

rpm -qil cronie|grep systemd
/lib/systemd/system/crond.service

# cat /lib/systemd/system/crond.service
[Unit]
Description=Command Scheduler
After=syslog.target

[Service]
EnvironmentFile=/etc/sysconfig/crond
ExecStart=/usr/sbin/crond -n $CRONDARGS

[Install]
WantedBy=multi-user.target

# ls -lL /etc/rc*.d/*crond*
ls: cannot access /etc/rc0.d/K60crond: No such file or directory
ls: cannot access /etc/rc1.d/K60crond: No such file or directory
ls: cannot access /etc/rc2.d/S90crond: No such file or directory
ls: cannot access /etc/rc3.d/S90crond: No such file or directory
ls: cannot access /etc/rc4.d/S90crond: No such file or directory
ls: cannot access /etc/rc5.d/S90crond: No such file or directory
ls: cannot access /etc/rc6.d/K60crond: No such file or directory
# ls -l /etc/rc*.d/*crond*
lrwxrwxrwx. 1 root root 15 Aug  6 12:21 /etc/rc0.d/K60crond -> ../init.d/crond
lrwxrwxrwx. 1 root root 15 Aug  6 12:21 /etc/rc1.d/K60crond -> ../init.d/crond
lrwxrwxrwx. 1 root root 15 Aug  6 12:21 /etc/rc2.d/S90crond -> ../init.d/crond
lrwxrwxrwx. 1 root root 15 Aug  6 12:21 /etc/rc3.d/S90crond -> ../init.d/crond
lrwxrwxrwx. 1 root root 15 Aug  6 12:21 /etc/rc4.d/S90crond -> ../init.d/crond
lrwxrwxrwx. 1 root root 15 Aug  6 12:21 /etc/rc5.d/S90crond -> ../init.d/crond
lrwxrwxrwx. 1 root root 15 Aug  6 12:21 /etc/rc6.d/K60crond -> ../init.d/crond
# ls -l /etc/init.d/*crond*
ls: cannot access /etc/init.d/*crond*: No such file or directory
# ls -l /usr/sbin/crond
-rwxr-xr-x. 1 root root 59840 Jun 29 22:55 /usr/sbin/crond


I'm not sure if there should be an init script for crond under
systemd.  There isn't one in the cronie package on FC15.


No idea why systemd isn't starting cron.

As an aside I also noticed dhcpd wasn't running after the upgrade but
found dhcp-sysvinit package which provided the necessary init scripts.
I can't find an equivalent one for cronie.

Comment 1 Tomas Mraz 2011-09-05 13:54:45 UTC
Crond is now started through the systemd cronie.service unit file.

Does the crond start after reboot?

Comment 2 Ian Donaldson 2011-09-05 22:24:21 UTC
No.

lastcom |grep cron 

shows it dumped core a lot during the yum distro-sync but that
was the last time it ran.

Comment 3 Ian Donaldson 2011-09-11 03:53:42 UTC
Ok after researching how to drive systemd, I have found a workaround,
but clearly there is an issue with the distro-sync related to cronie
upgrade that needs addressing.

The workaround was to reinstall cronie.

ie:
    rpm -e --nodeps cronie
    yum install cronie
    systemctl daemon-reload
    systemctl start crond

and after reboot things will be ok also (tested to be sure)



Prior, I noted this:


# rpm -q --verify cronie
missing   c /etc/pam.d/crond

# systemctl show cronie.service|grep Want
   (nothing wants crond)

# ll /etc/systemd/system/multi-user.target.wants/crond.service /lib/systemd/system/multi-user.target/crond.service
/bin/ls: cannot access /etc/systemd/system/multi-user.target.wants/crond.service: No such file or directory
/bin/ls: cannot access /lib/systemd/system/multi-user.target/crond.service: Not a directory


and after the reinstall of cronie, I see this:


# ll /etc/systemd/system/multi-user.target.wants/crond.service /lib/systemd/system/multi-user.target/crond.service
/bin/ls: cannot access /lib/systemd/system/multi-user.target/crond.service: Not a directory
-rw-r--r--. 1 root root 184 Jun 29 22:54 /etc/systemd/system/multi-user.target.wants/crond.service

# systemctl daemon-reload 

# systemctl show crond.service |grep Want
WantedBy=multi-user.target


So clearly the upgrade process failed to install /etc/systemd/system/multi-user.target.wants/crond.service symlink for some reason, which was
the main issue.  

Not sure why the pam.d entry was missing or if that matters a lot.

Comment 4 Ian Donaldson 2011-09-11 04:14:11 UTC
chkconfig crond on
systemctl daemon-reload
systemctl start cron.service

would have also had the same affect without having to reinstall cronie
as it recreates the systemd symlink above.

Comment 5 Marcela Mašláňová 2011-09-13 09:10:36 UTC
In postinstall scripts are called commands which should run cronie after update, but they probably don't work with upgrade of whole system. Imho upgrade from sysvinit scripts to systemd can't work well.

Comment 6 Michal Schmidt 2011-09-19 09:22:51 UTC
The %triggerun should take care of that. Unfortunately, multiple %triggerun scripts for the same package do not work as one would expect. See bug 702378.

Comment 7 Marcela Mašláňová 2011-10-17 12:05:35 UTC
Thanks, I wasn't aware that more triggerun doesn't work. I'll try to rewrite it somehow.

Comment 8 Fedora Update System 2011-10-17 15:04:08 UTC
cronie-1.4.8-3.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/cronie-1.4.8-3.fc15

Comment 9 Fedora Update System 2011-10-17 15:04:17 UTC
cronie-1.4.8-8.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/cronie-1.4.8-8.fc16

Comment 10 Marcela Mašláňová 2011-10-17 15:12:29 UTC
I hope it's fixed now. Upgrade from F-14 or F-15 should be now successful.

Comment 11 Fedora Update System 2011-10-25 03:21:49 UTC
cronie-1.4.8-8.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 12 guilde.nt 2011-11-26 11:01:11 UTC
cronie-1.4.8-10.fc16.i686
kernel-PAE-3.1.2-1.fc16.i686

$ ps -efH
...
root       814     1  0 11:33 ?        00:00:00   /usr/sbin/crond -n
root      1921   814  0 11:45 ?        00:00:00     /USR/SBIN/CROND -n
root      1922  1921  0 11:45 ?        00:00:00       /USR/SBIN/CROND -n
...

$ systemctl status crond.service
crond.service - Command Scheduler
	  Loaded: loaded (/lib/systemd/system/crond.service; enabled)
	  Active: active (running) since Sat, 26 Nov 2011 11:33:11 +0100; 24min ago
	Main PID: 814 (crond)
	  CGroup: name=systemd:/system/crond.service
		  â 814 /usr/sbin/crond -n

  However, it does not schedule the user commands (this was not checked for root).
  What does '/USR/SBIN/CROND -n' mean, please ? There is no /USR directory on the system.
  Thank you.

Comment 13 Marcela Mašláňová 2011-11-28 07:57:35 UTC
(In reply to comment #12)
> cronie-1.4.8-10.fc16.i686
> kernel-PAE-3.1.2-1.fc16.i686
> 
> $ ps -efH
> ...
> root       814     1  0 11:33 ?        00:00:00   /usr/sbin/crond -n
> root      1921   814  0 11:45 ?        00:00:00     /USR/SBIN/CROND -n
> root      1922  1921  0 11:45 ?        00:00:00       /USR/SBIN/CROND -n
> ...
> 
> $ systemctl status crond.service
> crond.service - Command Scheduler
>    Loaded: loaded (/lib/systemd/system/crond.service; enabled)
>    Active: active (running) since Sat, 26 Nov 2011 11:33:11 +0100; 24min ago
>  Main PID: 814 (crond)
>    CGroup: name=systemd:/system/crond.service
>     â 814 /usr/sbin/crond -n
> 
>   However, it does not schedule the user commands (this was not checked for
> root).
Could you elaborate? What doesn't work?

>   What does '/USR/SBIN/CROND -n' mean, please ? There is no /USR directory on
> the system.
>   Thank you.

The capital letters will be fixed in next release of cronie.

Comment 14 Tomas Mraz 2011-11-28 08:05:19 UTC
Actually not the capital letters but there will not be the full path in the logs, just the 'crond/CROND'. The capitalization of the letters means that this is a job-running process.

Comment 15 guilde.nt 2011-12-03 15:51:10 UTC
(In reply to comment #13)

> Could you elaborate? What doesn't work?

  'crond' is set to schedule a job every 15 min ; the job results are written to a file. After a few days, the process table contains hundreds of entries like these :

root      1921   814  0 11:45 ?        00:00:00     /USR/SBIN/CROND -n
root      1922  1921  0 11:45 ?        00:00:00       /USR/SBIN/CROND -n

but the timestamp shows that the file had not been modified. Sending a SIGHUP to 'crond' sets the scheduling of the job in order ; the process table entries shown above need to be discarded manually.
  I don't know at which version this problem occurred for the first time.
  Thank you.

Comment 16 Tomas Mraz 2011-12-05 08:02:19 UTC
Please open a new bug report as this is a completely different issue.

Comment 17 Patrick Sefton 2012-01-05 01:00:16 UTC
Just noting this bug is still (5 Jan 2012) present in F15 stable and affected me when I upgraded F14->F15 this week.  I assume this is because F15 cronie is still at cronie.1.4.8-2.fc15 (ie the fix in comment #8 has not been pushed to F15 stable).
Thanks.

Comment 18 Marcela Mašláňová 2012-01-09 15:29:21 UTC
(In reply to comment #17)
> Just noting this bug is still (5 Jan 2012) present in F15 stable and affected
> me when I upgraded F14->F15 this week.  I assume this is because F15 cronie is
> still at cronie.1.4.8-2.fc15 (ie the fix in comment #8 has not been pushed to
> F15 stable).
> Thanks.
In stable is currently this package since December:
https://admin.fedoraproject.org/updates/FEDORA-2011-14523/cronie-1.4.8-4.fc15

Comment 19 Michal Schmidt 2012-01-09 16:18:35 UTC
No, it's not in stable. The Bodhi page shows "Status: testing".

Also:
$ koji latest-pkg dist-f15-updates cronie
Build                                     Tag                   Built by
----------------------------------------  --------------------  ----------------
cronie-1.4.8-2.fc15                       dist-f15-updates      tmraz

Comment 20 Marcela Mašláňová 2012-01-09 16:34:05 UTC
I'm sorry, it must be manually marked for stable, because no one gave me the last +1. So, in few days it should be there.

Comment 21 Fedora Update System 2012-01-11 06:03:28 UTC
cronie-1.4.8-4.fc15 has been pushed to the Fedora 15 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 22 Lancelot Mak 2012-01-17 01:08:52 UTC
cronie-anacron-1.4.8-10.fc16.x86_64
cronie-1.4.8-10.fc16.x86_64


preupgrade-cli to F16 from F14 directly (i.e. no F15 in between)

locate cronie.service => no output

output of systemctl dump
-> Unit cronie.service:
        Description: cronie.service
        Instance: n/a
        Unit Load State: error
        Unit Active State: inactive
        Inactive Exit Timestamp: n/a
        Active Enter Timestamp: n/a
        Active Exit Timestamp: n/a
        Inactive Enter Timestamp: n/a
        GC Check Good: no
        Need Daemon Reload: no
        Name: cronie.service
        Load Error Code: No such file or directory

Comment 23 Michal Schmidt 2012-01-17 09:59:17 UTC
(In reply to comment #22)
> locate cronie.service => no output

Because the unit is called crond.service

Comment 24 Louis van Dyk 2012-07-18 00:48:25 UTC
I realise no more patches are being done for F15, but I updated two systems from F14 to F15 this week and this bug is still present.

Comment 25 Michal Schmidt 2012-07-18 07:48:19 UTC
That's probably because the latest build in F14 updates was cronie-1.4.8-2.fc14, but the %triggerun scriptlet that's supposed to enable crond.service has the condition cronie < 1.4.7-2.

Not sure if any fix for this in any maintained branch makes sense at this point.

Since the packaging guidelines require that packages migrating from SysV scripts to systemd units apply their default enablement policy instead of migrating the previous enablement status, you pretty much have to check which services you want enabled after an upgrade anyway. Just enable crond.service while at it.

Comment 26 Marcela Mašláňová 2012-07-18 07:57:36 UTC
Hm, true. I forgot to bump it after update to 1.4.8. I'm not sure if change of triggers would help at this point. Most of the people would upgrade before fix would hit mirrors.