Bug 712611

Summary: 'crontab' loops on DNS if nscd not running
Product: [Fedora] Fedora Reporter: todd_lewis
Component: pamAssignee: Tomas Mraz <tmraz>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rawhideCC: mmaslano, pertusus, tmraz
Target Milestone: ---Keywords: FutureFeature
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: pam-1.1.5-1.fc15 Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-12-04 02:23:18 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
/etc/nsswitch.conf
none
/etc/pam.d/system-auth
none
/etc/pam.d/password-auth
none
/etc/pam.d/crond none

Description todd_lewis 2011-06-11 19:02:02 UTC
Description of problem: The 'crontab' program loops forever doing DNS lookups if the nscd service is not running.


Version-Release number of selected component (if applicable):
cronie-1.4.7-3.fc15.i686

How reproducible: every time on this host.


Steps to Reproduce:
1. As root, enter "crontab -l". ("strace crontab -l" is fun too.)
2. Make coffee, read a book, ^C
3. service nscd start
3. re-enter step 1 command and get results.
  
Actual results:
stract crontab -l shows endless loop of DSN requests.

Expected results:
list of root's crontab. (I've been editing /var/spool/cron/root by hand and restarting crond to work around it.)


Additional info: This machine has been upgraded from Fedora 6, 7, 8, 9, 10, 11, 13, to 15, but I've "yum -y reinstall cronie" with no improvement.

Comment 1 Tomas Mraz 2011-06-13 07:06:35 UTC
This is most likely not a bug in cronie. Can you please attach your /etc/nsswitch.conf, /etc/pam.d/system-auth, /etc/pam.d/password-auth, /etc/pam.d/crond files?

Comment 2 todd_lewis 2011-09-28 11:39:57 UTC
Created attachment 525322 [details]
/etc/nsswitch.conf

Comment 3 todd_lewis 2011-09-28 11:41:11 UTC
Created attachment 525324 [details]
/etc/pam.d/system-auth

Comment 4 todd_lewis 2011-09-28 11:42:05 UTC
Created attachment 525325 [details]
/etc/pam.d/password-auth

Comment 5 todd_lewis 2011-09-28 11:43:32 UTC
Created attachment 525326 [details]
/etc/pam.d/crond

Comment 6 Tomas Mraz 2011-09-29 07:01:06 UTC
I'm suspecting the pam_krb5 to be the culprit. If you temporarily disable kerberos with authconfig: 'authconfig --disablekrb5 --update', does it help?

Comment 7 todd_lewis 2011-09-29 14:27:31 UTC
Here's an annotated 'history' of my recent commands to test whether
'authconfig --disablekrb5 --update' affects things:

  913  cp -av pam.d pam.d-orig
Backup my /etc/pam.d

  914  crontab -l
Test. (nscd was running.) This worked fine, as expected.

  915  service nscd stop
  916  crontab -l
As expected, this never returned. Had to ^C out. So, the system is still exhibiting the problem behavior before any changes.

  917  service nscd start
  918  crontab -l
...and it works again after restarting nscd. So far no surprises.
Now for the real test....

  919  authconfig --disablekrb5 --update
  920  crontab -l
nscd is still running; this worked fine.

  921  service nscd stop
  922  crontab -l
This failed. Again, had to ^C. So disabling krb5 via authconfig didn't fix anything.

  923  diff pam.d pam.d-orig
  924  diff -u pam.d pam.d-orig
pam.d files were different. Not unexpected.

  925  authconfig --enablekrb5 --update
  926  diff -u pam.d pam.d-orig
pam.d files are all back to their original state.

  927  crontab -l
This now works! Because...

  929  service nscd status
  930  service nscd stop
It turns out (after a little experimenting) that
'authconfig --enablekrb5 --update' has the side effect of activating nscd, which I didn't expect. So I stopped it again.

  931  crontab -l
And this fails again, either with or without krb5 enabled as long a nscd is not running.

(In reply to comment #6)
> I'm suspecting the pam_krb5 to be the culprit. If you temporarily disable
> kerberos with authconfig: 'authconfig --disablekrb5 --update', does it help?

Comment 8 Tomas Mraz 2011-09-29 15:10:13 UTC
There is also pam_access in the crond. It normally should not be doing "looping DNS requests" but what is in your /etc/security/access.conf? Also when you look into the strace, what is it looking up?

Comment 9 todd_lewis 2011-09-29 15:41:24 UTC
(In reply to comment #8)
> There is also pam_access in the crond. It normally should not be doing "looping
> DNS requests" but what is in your /etc/security/access.conf? Also when you look
> into the strace, what is it looking up?

That's it! My /etc/security/access.conf has about 140 lines of stuff that starts like this:

# Block all logins from China
-:ALL: 58.14.0.0/15 58.16.0.0/16 58.17.0.0/17 58.17.128.0/17 58.18.0.0/16 58.19.0.0/16 58.20.0.0/16 58.21.0.0/16 58.22.0.0/15
-:ALL: 58.24.0.0/15 58.30.0.0/15 58.32.0.0/13 58.40.0.0/15 58.42.0.0/16 58.43.0.0/16 58.44.0.0/14 58.48.0.0/13 58.56.0.0/15
-:ALL: 58.58.0.0/16 58.59.0.0/17 58.59.128.0/17 58.60.0.0/14 58.66.0.0/15 58.68.128.0/17 58.82.0.0/15 58.87.64.0/18 58.99.128.0/17
-:ALL: 58.100.0.0/15 58.116.0.0/14 58.128.0.0/13 58.144.0.0/16 58.154.0.0/15 58.192.0.0/15 58.194.0.0/15 58.196.0.0/15 58.198.0.0/15
-:ALL: 58.200.0.0/13 58.208.0.0/12 58.240.0.0/15 58.242.0.0/15 58.244.0.0/15 58.246.0.0/15 58.248.0.0/13 59.32.0.0/13 59.40.0.0/15

... it goes on. That's 1249 subnets. Removing them makes 'crontab -l' run instantly with nscd stopped. With nscd running and all those "-:ALL:" lines in place, there is a slight delay of maybe a second that I'd not really noticed until it was gone.

The strace does show it hitting my nameserver, presumably once for each denied subnet spec. But why would a local 'crontab -l' need to hit the nameserver for each subnet mentioned in those "-:ALL:" lines?

Moving the "+:ALL:LOCAL" line above the "#Block all logins from China" section (it was below before) makes 'crontab -l' start quickly regardless of nscd. D'oh!

Still, seems strange that DNS would get hit for all those subnets.

Comment 10 Tomas Mraz 2011-09-29 16:44:38 UTC
You can add debug option to pam_access and see what it tries to do. But I actually think it does not try to resolve the china netmasks but the local tty name again and again. It could probably be made more clever to avoid the DNS lookup on the tty name and do it only on hostnames perhaps also caching the result. Unfortunately nobody expected 1249 subnets in access.conf.

Comment 11 Fedora Update System 2011-11-24 14:21:17 UTC
pam-1.1.5-1.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/pam-1.1.5-1.fc16

Comment 12 Fedora Update System 2011-11-24 14:21:37 UTC
pam-1.1.5-1.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/pam-1.1.5-1.fc15

Comment 13 Fedora Update System 2011-11-25 02:11:39 UTC
Package pam-1.1.5-1.fc15:
* should fix your issue,
* was pushed to the Fedora 15 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing pam-1.1.5-1.fc15'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2011-16365/pam-1.1.5-1.fc15
then log in and leave karma (feedback).

Comment 14 Fedora Update System 2011-12-04 02:23:18 UTC
pam-1.1.5-1.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 15 Fedora Update System 2011-12-10 20:04:53 UTC
pam-1.1.5-1.fc15 has been pushed to the Fedora 15 stable repository.  If problems still persist, please make note of it in this bug report.