Bug 1067236 - Jobs won't run; crond states bad username
Summary: Jobs won't run; crond states bad username
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: cronie
Version: 20
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Marcela Mašláňová
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: 1145080
TreeView+ depends on / blocked
 
Reported: 2014-02-20 01:09 UTC by John Florian
Modified: 2017-03-22 14:20 UTC (History)
7 users (show)

Fixed In Version: cronie-1.4.12-1.fc21
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 1145080 (view as bug list)
Environment:
Last Closed: 2014-09-28 04:25:22 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description John Florian 2014-02-20 01:09:57 UTC
Description of problem:
I have cron jobs that must run as usernames that only exist in LDAP.  I see the "bad username" error because crond can be started before the LDAP server is reachable.  The LDAP server is distinct from the server with the cron job.  The issue is especially troublesome when both servers are VMs on the same host and booting up together.

Version-Release number of selected component (if applicable):
cronie-1.4.11-4.fc20.x86_64

How reproducible:
always

Steps to Reproduce:
1. create a cron job with a username that only exists in LDAP
2. stop crond and the LDAP server (or otherwise make it unreachable)
3. start crond
4. start the LDAP server (or make it available again)

Actual results:
Logs show "crond[PID]: (CRON) bad username (/etc/cron.d/JOBNAME)


Expected results:
Ideally crond would periodically retry to establish the username identities each time the job comes due again, if they weren't already successfully established.

Additional info:

Comment 1 Marcela Mašláňová 2014-03-20 16:06:40 UTC
That's strange. Bug with orphaned cronjobs was fixed by 1.4.8 release. The 1.4.11 release is also in RHEL-7, where this bug was verified as fixed.

Could you still reproduce it?

Comment 2 John Florian 2014-03-28 12:52:45 UTC
I can indeed still reproduce it with cronie-1.4.10-7.fc19.x86_64.  I just discovered another daily job that's not been run for the last 25 days or so because of this:

# ls -art /var/log/cron* | xargs grep bad
/var/log/cron-20140309:Mar  3 14:13:58 mdct-aos-master-f19 crond[300]: (CRON) bad username (/etc/cron.d/mdct-puppeteer-admin)
/var/log/cron-20140309:Mar  3 14:13:58 mdct-aos-master-f19 crond[300]: (CRON) bad username (/etc/cron.d/mdct-puppeteer-admin)
/var/log/cron-20140309:Mar  3 14:13:58 mdct-aos-master-f19 crond[300]: (CRON) bad username (/etc/cron.d/mdct-puppeteer-admin)
/var/log/cron-20140323:Mar 18 14:33:19 mdct-aos-master-f19 crond[303]: (CRON) bad username (/etc/cron.d/mdct-puppeteer-admin)
/var/log/cron-20140323:Mar 18 14:33:19 mdct-aos-master-f19 crond[303]: (CRON) bad username (/etc/cron.d/mdct-puppeteer-admin)
/var/log/cron-20140323:Mar 18 14:33:19 mdct-aos-master-f19 crond[303]: (CRON) bad username (/etc/cron.d/mdct-puppeteer-admin)

Comment 3 John Florian 2014-06-10 21:34:59 UTC
Is there anything I can do to help move this bug along?

As things stand, I've resorted to puppet pushing out a cron.daily job (as root) that restarts crond to minimize the damage this is causing on numerous systems.

Comment 4 Marcela Mašláňová 2014-09-04 11:41:39 UTC
I'm sorry I missed the needinfo. Anyway according our tests it's working. I do not know if we have different setting or what's the reason I can't reproduce it.

Comment 5 John Florian 2014-09-04 12:09:56 UTC
(In reply to Marcela Mašláňová from comment #4)
> I'm sorry I missed the needinfo. Anyway according our tests it's working. I
> do not know if we have different setting or what's the reason I can't
> reproduce it.

I would have to say then that your tests somehow do not cover the situation I described.  Perhaps you have an insufficient elapsed time between steps 3 & 4?  I still see this and require a cron.daily job to restart crond -- which is kludgey at best.

Comment 6 Fedora Admin XMLRPC Client 2014-09-04 12:15:13 UTC
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Comment 7 Terje Røsten 2014-09-16 20:51:36 UTC
I see something similar on RHEL7. 

Systems are using sssd for users auth, However it seems crond is started before sssd in init process and then find users crontabs to be orphans and disable all.

Restarting crond after startup is done fixes the problem. Is it possible
in systemd service speak to order crond to start after sssd has made users
visible on the systems?

Comment 8 Marcela Mašláňová 2014-09-17 08:51:34 UTC
Thanks, that explains the mystery.

Cronie is probably missing this in unit file:
After=nss-user-lookup.target

Lukas, could you verify it's a proper fix?

Comment 9 Jakub Hrozek 2014-09-17 08:59:34 UTC
We added:

Before=systemd-user-sessions.service nss-user-lookup.target
Wants=nss-user-lookup.target

Very recently to sssd, as a matter of fact I just pushed that build to stable (it's sssd-1.11.6-3).

Comment 10 Lukáš Nykrýn 2014-09-17 09:05:00 UTC
(In reply to Marcela Mašláňová from comment #8)
> Thanks, that explains the mystery.
> 
> Cronie is probably missing this in unit file:
> After=nss-user-lookup.target
> 
> Lukas, could you verify it's a proper fix?

Yep, looks sane.

Comment 11 Fedora Update System 2014-09-18 08:42:32 UTC
cronie-1.4.12-1.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/cronie-1.4.12-1.fc20

Comment 12 Fedora Update System 2014-09-18 09:17:16 UTC
cronie-1.4.12-1.fc21 has been submitted as an update for Fedora 21.
https://admin.fedoraproject.org/updates/cronie-1.4.12-1.fc21

Comment 13 Fedora Update System 2014-09-19 10:16:26 UTC
Package cronie-1.4.12-1.fc20:
* should fix your issue,
* was pushed to the Fedora 20 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing cronie-1.4.12-1.fc20'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2014-11067/cronie-1.4.12-1.fc20
then log in and leave karma (feedback).

Comment 14 John Florian 2014-09-19 12:43:27 UTC
Sadly cronie-1.4.12-1.fc20 does not resolve the issue for me.  I installed this (and all other) updates, rebooted and still see "bad username" errors from crond that will go away simply by restarting crond.

Would it really be so hard for cronie to just retest the validity of any failed user periodically?

Comment 15 Tomas Mraz 2014-09-19 12:48:05 UTC
There already is such functionality, but it is probably broken somehow for you. I need to find a reproducer on my system to debug this.

Comment 16 John Florian 2014-09-19 12:50:17 UTC
(In reply to Tomas Mraz from comment #15)
> There already is such functionality, but it is probably broken somehow for
> you. I need to find a reproducer on my system to debug this.

If there's anything (tcpdump, strace, etc) I can provide to help, please let me know.

Comment 17 John Florian 2014-09-19 13:00:43 UTC
Oh crap!  I just reread Jakub's comment #9 and realized I hadn't checked what release of sssd I had installed.  Sure enough -3 was still only in our testing repo mirror.  I upgraded from -2 to -3, rebooted and jobs seem to be firing now.

I'm so sorry for the bad info.  Granted, I've only seen this now working for a few minutes and a single test, but it *always* failed before so I'm now very hopeful that this bug is squashed for good.

Thanks to all!

Comment 18 Marcela Mašláňová 2014-09-22 10:42:31 UTC
Thanks all for comments. I'll clone it for RHEL.

Comment 19 Fedora Update System 2014-09-28 04:25:22 UTC
cronie-1.4.12-1.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 20 Fedora Update System 2014-10-08 19:15:01 UTC
cronie-1.4.12-1.fc21 has been pushed to the Fedora 21 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.