Bug 770869

Summary: systemctl restart nscd.service kills but does not (re)start nscd
Product: [Fedora] Fedora Reporter: Ville Skyttä <ville.skytta>
Component: glibcAssignee: Jeff Law <law>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 16CC: Francis.Montagnac, fweimer, jakub, law, schwab
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-02-03 20:43:11 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
No forking, nscd -d, fix typo: -i services
none
Patch for glibc that introduces daemon-like foreground mode in nscd none

Description Ville Skyttä 2011-12-29 20:54:38 UTC
With nscd enabled and started at boot, it works.  However, systemctl restart nscd.service manages to kill nscd but never brings it up.  After trying to restart:

nscd.service - Name Service Cache Daemon
	  Loaded: loaded (/lib/systemd/system/nscd.service; enabled)
	  Active: failed since Thu, 29 Dec 2011 22:48:16 +0200; 5s ago
	 Process: 2598 ExecStop=/usr/sbin/nscd --shutdown (code=exited, status=1/FAILURE)
	 Process: 2595 ExecStart=/usr/sbin/nscd $NSCD_OPTIONS (code=exited, status=0/SUCCESS)
	Main PID: 2596 (code=exited, status=0/SUCCESS)
	  CGroup: name=systemd:/system/nscd.service

In /var/log/messages I see a bunch of these:

Dec 29 22:50:16 viper systemd[1]: PID 2597 read from file /run/nscd/nscd.pid does not exist. Your service or init script might be broken.
Dec 29 22:50:16 viper systemd[1]: PID 2597 read from file /run/nscd/nscd.pid does not exist. Your service or init script might be broken.
Dec 29 22:50:16 viper systemd[1]: nscd.service holdoff time over, scheduling restart.
Dec 29 22:50:16 viper systemd[1]: PID 2673 read from file /run/nscd/nscd.pid does not exist. Your service or init script might be broken.
Dec 29 22:50:16 viper systemd[1]: PID 2673 read from file /run/nscd/nscd.pid does not exist. Your service or init script might be broken.
Dec 29 22:50:16 viper systemd[1]: nscd.service: control process exited, code=exited status=1
Dec 29 22:50:16 viper systemd[1]: nscd.service holdoff time over, scheduling restart.
Dec 29 22:50:16 viper systemd[1]: Unit nscd.service entered failed state.
[etc, repeated a few times]

...and in /var/log/nscd.log with nscd debug level set to 10 in /etc/nscd.conf:

Thu 29 Dec 2011 10:50:16 PM EET - 2673: register trace file /etc/passwd for database passwd
Thu 29 Dec 2011 10:50:16 PM EET - 2673: register trace file /etc/group for database group
Thu 29 Dec 2011 10:50:16 PM EET - 2673: register trace file /etc/hosts for database hosts
Thu 29 Dec 2011 10:50:16 PM EET - 2673: register trace file /etc/resolv.conf for database hosts
Thu 29 Dec 2011 10:50:16 PM EET - 2673: register trace file /etc/services for database services
Thu 29 Dec 2011 10:50:16 PM EET - 2673: register trace file /etc/netgroup for database netgroup
Thu 29 Dec 2011 10:50:16 PM EET - 2673: cannot stat() file `/etc/netgroup': No such file or directory
Thu 29 Dec 2011 10:50:16 PM EET - 2673: handle_request: request received (Version = 2) from PID 2674
Thu 29 Dec 2011 10:50:16 PM EET - 2673:         SHUTDOWN
Thu 29 Dec 2011 10:50:16 PM EET - 2688: register trace file /etc/passwd for database passwd
Thu 29 Dec 2011 10:50:16 PM EET - 2688: register trace file /etc/group for database group
Thu 29 Dec 2011 10:50:16 PM EET - 2688: register trace file /etc/hosts for database hosts
Thu 29 Dec 2011 10:50:16 PM EET - 2688: register trace file /etc/resolv.conf for database hosts
Thu 29 Dec 2011 10:50:16 PM EET - 2688: register trace file /etc/services for database services
Thu 29 Dec 2011 10:50:16 PM EET - 2688: register trace file /etc/netgroup for database netgroup
Thu 29 Dec 2011 10:50:16 PM EET - 2688: cannot stat() file `/etc/netgroup': No such file or directory
[etc, repeated a few times]

Comment 1 Francis.Montagnac 2012-01-01 14:50:21 UTC
Created attachment 550156 [details]
No forking, nscd -d, fix typo: -i services

Comment 2 Francis.Montagnac 2012-01-01 15:08:57 UTC
I have the same problem.

Replacing /lib/systemd/system/nscd.service by the proposed attachement
(https://bugzilla.redhat.com/attachment.cgi?id=550156) seems (be always
carefull with nscd :-)) to work for me.

You may prefer to install this file in

 /etc/systemd/system/nscd.service 

and to link 

 /etc/systemd/system/multi-user.target.wants/nscd.service 

to:

  ../nscd.service 

This as the disadvantage of logging too much of nscd in syslog. This
could be changed in nscd.service by requesting to log to /dev/null
(not tested yet).

Francis

PS: This also fixes a typo in this service file:

! ExecReload=/usr/sbin/nscd -i service

...

! ExecReload=/usr/sbin/nscd -i services

Comment 3 Ville Skyttä 2012-01-01 22:36:02 UTC
Confirmed that the modified unit file works, and too much logging confirmed as well :(

By the way, just curious, I wonder why there's no "ExecReload=/usr/sbin/nscd -i netgroup" in the unit file (nor the original one)...

Comment 4 Francis.Montagnac 2012-01-02 07:49:30 UTC
> Confirmed that the modified unit file works, 

Fine.

> and too much logging confirmed as well :(

Just add in the [Service] section of nscd.service:

StandardOutput=null

> By the way, just curious, I wonder why there's no
> "ExecReload=/usr/sbin/nscd -i netgroup" in the unit file (nor the
> original one)...

Because nscd is not currently caching netgroups.

Comment 5 Jeff Law 2012-01-02 17:49:59 UTC
What is interesting here is the message about being unable to find /run/nscd/nscd.pid

Can you send the output of the following commands:

cat /proc/mounts
ls /var/run
ls /run

Comment 6 Ville Skyttä 2012-01-03 07:03:02 UTC
(In reply to comment #4)

> StandardOutput=null

Sounds like a big hammer that would possibly also lose useful information.

> Because nscd is not currently caching netgroups.

Even though /etc/nscd.conf contains a netgroups section which is enabled by default?  If it has no effect, I suggest removing it then.

(In reply to comment #5)
> What is interesting here is the message about being unable to find
> /run/nscd/nscd.pid

But that's not the message in my logs: "PID 2597 read from file /run/nscd/nscd.pid does not exist. Your service or init script might be broken."

I think that means that a pid (2597) was read from /run/nscd/nscd.pid, but there's no process with that pid.

Comment 7 Francis.Montagnac 2012-01-03 07:52:17 UTC
>> StandardOutput=null

> Sounds like a big hammer 

I would say: a turnaround needed to the lack of an option of nscd to
*not* daemonize, unlike for example "crond -n"

> that would possibly also lose useful information.

Without the -d option, nscd do not log to syslog. If you want logging,
it's I think better to declare it in nscd.conf, to log to a file.

>> Because nscd is not currently caching netgroups.

> Even though /etc/nscd.conf contains a netgroups section which is
> enabled by default?

Sorry, I was wrong, nscd caches now also the netgroups. I did an
upgrade that kept the old (modified) version of nscd.conf.

Thanks to point out that.

>> What is interesting here is the message about being unable to find
>> /run/nscd/nscd.pid
...
> I think that means that a pid (2597) was read from /run/nscd/nscd.pid,
> but there's no process with that pid.

I agree, and you can reproduce it like this:

/usr/sbin/nscd -K # stop nscd

/usr/sbin/nscd; cat /run/nscd/nscd.pid; ps auxc|grep nscd; sleep 1; cat /run/nscd/nscd.pid
1879
nscd      1901  0.0  0.0 509044  1020 ?        Ssl  08:30   0:00 nscd
1901

This last command shows that nscd.pid is *not* valid after nscd
returns (and daemonize), but becomes valid after 1 second.

IMHO:

 - the proper way to correct that should be to add a -n option to nscd
   for running in the foreground.

 - in the meantime, launch it in debug mode (-d), but redirect its
   output to /dev/null

Comment 8 Ville Skyttä 2012-01-03 19:14:45 UTC
(In reply to comment #7)

> Without the -d option, nscd do not log to syslog.

You mean _with_ the -d option?

> If you want logging,
> it's I think better to declare it in nscd.conf, to log to a file.

My nscd does not log anything to the logfile configured in nscd.conf if invoked with -d.

Comment 9 Alexandre Oliva 2012-01-21 03:34:58 UTC
Created attachment 556663 [details]
Patch for glibc that introduces daemon-like foreground mode in nscd

Yeah, we really don't want to use nscd -d, it reduces security too (disables paranoid mode).  I'm proposing this patch for glibc to introduce --foreground (or -F) to run in foreground, but like a daemon.