Bug 1011968

Summary: chronyd service does not starts on ppc64 when ntpd is running
Product: Red Hat Enterprise Linux 7 Reporter: Patrik Kis <pkis>
Component: chronyAssignee: Miroslav Lichvar <mlichvar>
Status: CLOSED CURRENTRELEASE QA Contact: Jakub Prokes <jprokes>
Severity: high Docs Contact:
Priority: high    
Version: 7.0CC: jmoskovc, jprokes, mlichvar, mschmidt, nphilipp, pkis, systemd-maint-list
Target Milestone: betaKeywords: TestBlocker
Target Release: ---   
Hardware: All   
OS: Unspecified   
Whiteboard:
Fixed In Version: chrony-1.29-2.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-06-13 09:41:27 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 917637    

Description Patrik Kis 2013-09-25 13:15:57 UTC
Description of problem:
I'm not sure if it is chronyd or ntpd issue but probably either of them.
The problem is that chronyd does not starts at the first attempt when ntpd is runnung. On the second attempt choryd starts.
NOTE: this works out of the box on x86_64, but I had no chance to check it on s390x yet

Version-Release number of selected component (if applicable):
# rpm -qa chrony ntp systemd
systemd-207-1.el7.ppc64
ntp-4.2.6p5-13.el7.ppc64
chrony-1.29-1.el7.ppc64

How reproducible:
always

Steps to Reproduce:
[ppc64]# systemctl start chronyd.service; echo $?
0
[ppc64]# systemctl | grep -e ntp -e chrony
chronyd.service             loaded active running   NTP client/server
[ppc64]# systemctl start ntpd.service; echo $?
0
[ppc64]# systemctl | grep -e ntp -e chrony
ntpd.service                loaded active running   Network Time Service
[ppc64]# systemctl start chronyd.service; echo $?
Job for chronyd.service failed. See 'systemctl status chronyd.service' and 'journalctl -xn' for details.
1
[ppc64]# systemctl | grep -e ntp -e chrony
chronyd.service             loaded failed failed    NTP client/server
[ppc64]# systemctl start chronyd.service; echo $?
0
[ppc64]# systemctl | grep -e ntp -e chrony
chronyd.service             loaded active running   NTP client/server
[ppc64]# 


Actual results:
chronyd does not start

Expected results:
Like on x86_64 systems:

Comment 1 Patrik Kis 2013-09-25 13:20:46 UTC
It looks like the same issue appears also on s390x architecture.

Comment 3 Miroslav Lichvar 2013-09-26 07:38:23 UTC
Can you please attach the logs from the ntpd and chronyd services? It looks like chronyd is started before ntpd is completely stopped.

Comment 6 Miroslav Lichvar 2013-09-26 08:42:29 UTC
From the log it seems ntpd was stopped before chronyd was started, but chronyd still couldn't bind to port 123. I'm wondering why it happens only on ppc or s390x.

Comment 8 Nils Philippsen 2013-09-27 10:33:47 UTC
Phil asked me to test this. On my x86_64 VM, this works as I would expect it: If both units are enabled, only chronyd.service gets started ("Conflicts=ntpd.service" in chronyd.service). If I manually start ntpd.service ("systemctl start ntpd.service"), it gets started but chronyd.service is shut down and vice versa.

Package versions:
systemd-207-2.el7.x86_64
chrony-1.29-1.el7.x86_64
ntp-4.2.6p5-13.el7.x86_64

Let me know if I should test anything else.

Comment 9 Patrik Kis 2013-09-27 12:47:03 UTC
Well I tried also with systemd-207-2.el7.ppc64, but that doesn't changed anything. Please note that the issue newer appeared to me on x86_64.

Comment 10 Miroslav Lichvar 2013-09-30 10:17:41 UTC
I was able to reproduce the problem on an x86_64 system too. Looks like a race condition.

I patched the ntpd signal handler to wait 5 seconds before calling exit(). This is what I see in the system log when running "systemctl start ntpd; sleep 3; systemctl start chronyd":

Sep 30 06:09:22 dhcp-25-211 ntpd[892]: 0.0.0.0 c011 01 freq_not_set
Sep 30 06:09:25 dhcp-25-211 systemd: Stopping Network Time Service...
Sep 30 06:09:25 dhcp-25-211 ntpd[892]: ntpd exiting on signal 15, waiting 5 seconds
Sep 30 06:09:25 dhcp-25-211 systemd: Starting NTP client/server...
Sep 30 06:09:25 dhcp-25-211 chronyd[898]: chronyd version 1.29 starting
Sep 30 06:09:25 dhcp-25-211 chronyd[898]: Linux kernel major=3 minor=10 patch=0  
Sep 30 06:09:25 dhcp-25-211 chronyd[898]: hz=100 shift_hz=7 freq_scale=1.00000000 nominal_tick=10000 slew_delta_tick=833 max_tick_bias=1000 shift_pll=2
Sep 30 06:09:25 dhcp-25-211 chronyd[898]: Could not bind IPv4 NTP socket : Address already in use
Sep 30 06:09:25 dhcp-25-211 chronyd[898]: Could not bind IPv6 NTP socket : Address already in use
Sep 30 06:09:25 dhcp-25-211 chronyd[898]: Fatal error : Could not open any NTP socket
Sep 30 06:09:25 dhcp-25-211 chronyd: Could not open any NTP socket
Sep 30 06:09:25 dhcp-25-211 systemd: chronyd.service: control process exited, code=exited status=1
Sep 30 06:09:25 dhcp-25-211 systemd: Failed to start NTP client/server.
Sep 30 06:09:25 dhcp-25-211 systemd: Unit chronyd.service entered failed state.
Sep 30 06:09:30 dhcp-25-211 ntpd[892]: exiting now
Sep 30 06:09:30 dhcp-25-211 systemd: Stopped Network Time Service.


The ntpd process seems to be still alive when chronyd is started. That is with systemd-207-2.el7.x86_64. Reassigning to systemd.

Comment 11 Michal Schmidt 2013-09-30 13:02:32 UTC
chronyd.service says:
Conflicts=ntpd.service

This is a (negative) requirement dependency, but it does not imply any ordering dependency between the two. So it is permitted for systemd to start chronyd while stopping ntpd simultaneously.

This is from "man systemd.unit", the paragraph about "Before=, After=":
  If two units have no ordering dependencies between them they are shut down
  or started up simultaneously, and no ordering takes place.

If you add an ordering dependency (in this case it does not matter if you choose Before or After), the following rule would apply:

  If one unit with an ordering dependency on another unit is shut down while
  the latter is started up, the shut down is ordered before the start-up
  regardless whether the ordering dependency is actually of type After= or
  Before=.

Another resolution would be to drop the Conflicts entirely and simply rely on the administrators to stop or disable any conflicting services by themselves. (I personally dislike the use of Conflicts for alternative implementations of the same kind of a service. Others may disagree.)

Comment 12 Miroslav Lichvar 2013-10-03 07:56:52 UTC
Ok, adding ntpd.service to After= in the chronyd unit file seems to fix the problem. Thanks for pointing that out.

Comment 14 Jakub Prokes 2013-10-03 13:20:08 UTC
FailedQA is set by mistake

Comment 17 Ludek Smid 2014-06-13 09:41:27 UTC
This request was resolved in Red Hat Enterprise Linux 7.0.

Contact your manager or support representative in case you have further questions about the request.