Bug 229217

Summary: enhancing ntp support
Product: [Fedora] Fedora Reporter: Miroslav Lichvar <mlichvar>
Component: system-config-dateAssignee: Nils Philippsen <nphilipp>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: medium Docs Contact:
Priority: medium    
Version: 9CC: archimerged
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: 1.9.32-1.fc8 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2008-07-09 02:42:36 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
enable ntpdate service
none
ntp.conf using tos orphan and tos minsane statements.
none
Script to find nearby timeservers from us.pool.ntp.org by pinging
none
Script to figure out which timeservers are nearest.
none
Assigns the nearby timeservers to the local peers so each list is different. none

Description Miroslav Lichvar 2007-02-19 16:46:26 UTC
Description of problem:

ntpd (version 4.2.4) supports a new keyword dynamic as a server option. It's
useful on for systems where network isn't completely up when ntpd is started,
s-c-d should use the option by default.

A server line in ntp.conf would look like this:
server 1.2.3.4 dynamic

Would be nice if s-c-d allowed to use iburst option, to speed up ntp 

Version-Release number of selected component (if applicable):
system-config-date-1.8.11-1.fc7

Comment 1 Miroslav Lichvar 2007-02-19 16:51:50 UTC
Another thing I forgot to mention, s-c-d shouldn't add the restrict lines for
each server. It was required for ntp-4.1.2, but default restrict is now sufficient.

Comment 2 Nils Philippsen 2007-10-09 15:40:54 UTC
I guess I'll have to defer that until after F8 is out, but in the meantime: what
is the iburst option?

Comment 3 Miroslav Lichvar 2007-10-09 16:04:32 UTC
iburst reduces the initial delay to set the clock. Usually it'll take just 10
seconds instead of 5 minutes, this could be convenient for some users.

Comment 4 Miroslav Lichvar 2008-03-03 14:13:58 UTC
Created attachment 296601 [details]
enable ntpdate service

ntp >= 4.2.4p4-4 has a separate init script for ntpdate, this patch
enables/disables the service as needed.

Comment 5 Nils Philippsen 2008-03-04 10:17:22 UTC
(In reply to comment #4)
> ntp >= 4.2.4p4-4 has a separate init script for ntpdate, this patch
> enables/disables the service as needed.

I'm just building system-config-date-1.9.23 which has this patch applied. Thanks!

Comment 6 archimerged Ark submedes 2008-04-03 07:28:42 UTC
In F9 beta, 'dynamic' is still not present on server statements written to
/etc/ntp.conf by system-config-date or by /sbin/dhclient-script.  Presumably
this would mask the problem of ntpd dropping all servers because service network
is completely off by default (at least if you ask for automatic configuration of
all interfaces) and NetworkManager doesn't start until after ntpd.


Comment 7 Nils Philippsen 2008-04-03 16:26:48 UTC
The "dynamic" keyword thing should be fixed in s-c-date 1.9.28 which is building
right now. Is there a separate bug for dhclient-script?

The "iburst" thing will have to wait until after F9, as that would need new
translatable strings.

Comment 8 archimerged Ark submedes 2008-04-04 20:18:05 UTC
No, I didn't add one.  Actually, I suppose those servers added by
dhclient-script don't really need dynamic since dhclient-script will restart
ntpd if any of them change.  (It uses diff to compare ntp.conf.predhclient and
ntp.conf)  I haven't looked at exactly what "dynamic" does inside ntpd.  Maybe
it should always be used...  BTW probably dhclient-script and s-c-date should
change ntp.conf but instead of restarting ntpd, should apply the changes via
ntpdc so ntpd doesn't have to start from scratch with polling at 64 seconds. 
Avoid unnecessary load on the servers.  And ntpdate is deprecated; Dave Mills
wants you to use ntpd -q.  See man ntpd.  (IIRC that note has been in man ntpd
for over 14 years...)


Other thoughts:

In nearly any configuration where you don't have a high-reliability redundant
provider internet connection, you need 'dynamic' so that you can boot the system
while the internet connection is down and have ntpd start working when the
connection comes up.  (Assuming this is how it works -- I haven't tested it).

Other changes in system-config-date to wish for:

* Provide a one-click per box local timeserver with a "listen for local
timerserver broadcast" checkbox and a one click "provide local broadcast time
service."

* Change the label "Advanced options" to "Special situation options."

* The option to use ntpdate should be marked deprecated as unnecessary because
ntpd will step the clock (once) an unrestricted amount after it is sure of the
time (since /etc/sysconfig/nptd contains the -g option).  Otherwise, users will
assume the old situation applies and tend to turn ntpdate on unnecessarily.

For example, add a text box containing "Note that by default, ntpd will correct
the date within about half an hour after startup."

* The advanced option "use a local time source" should include the description
"Permit this machine to act as timeserver for local clients even if it cannot
reach any outside timeservers."

However, to fully replace the local time source, ntp.conf should include the
"tos orphan 6" statement.  See /usr/share/doc/ntp-4.2.4p4/manyopt.html:
"For the most flexible and reliable operation, all servers and clients in the
subnet should include the 'orphan' command in the configuration file and with
the same orphan stratum. This provides mutual redundancy and diversity for all
NTP modes of operation, including broadcast."

* Add a "peers" section to s-c-date to specify /etc/hosts names or local DNS
server names or IP numbers of local peer timeservers, creating peer statements
in ntp.conf.

* For "server" and "peer" lines, add a minimum polling interval (offering the
choice of 64, 128, 256, 512, 1024, and 2048 seconds) so that considerate users
can easily reduce the load they put on the timeservers without having to read
all the ntp documentation.  Include the request that a single person should
never use more than three or four servers with minimum polling interval below
1024, and no two peers should use the same server.  (But this requires resolving
the *.fedora.pool.ntp.org domain name...)

* Add an option to include the recommended 'tos minsane 4' (separate line in
ntp.conf) which means ntpd won't adjust the clock at all unless it has four
servers or peers which show - or + on ntpq -p.  "tos minsane specifies the
minimum number of candidates available to the clock selection algorithm in order
to produce one or more truechimers for the clustering algorithm. If fewer than
this number are available, the clock is undisciplined and allowed to run free.
The default is 1 for legacy purposes. However, according to principles of
Byzantine agreement, minsane should be at least 4 in order to detect and discard
a single falseticker."  Only enable this if there are at least 4 peers and
servers specified.

My experience with minsane 4:

I will attach my ntp.conf and the script I used to find some nearby (low ping
time) timeservers.  I have 6 boxes and I give each box one 64 second poll
server, and three 1024 second poll servers so that I can use tos minsane 4.
The are also all peers to each other.

Then ntpq -p shows 9 lines (I took the trouble to avoid making a box its own
peer).  When 6 or more are marked with 'space' or 'x' for the flag character,
the others have '#' flag.  When 5 or fewer are marked with 'space' or 'x', one
is '*' and the others are '+' or '-'.  Sometimes there are only two with '+' in
addition to the one with '*', so minsane 4 does not mean a minimum of three '+'.
(Not sure if I saw a '*' with only one or zero '-').

Don't know how this would go over with the folks running timeservers -- I
imagine fedora can't make this sort of change without talking to them, but for
people with lots of local machines, the default arrangement puts a lot of load
on servers (three for each box) and if the boxes all come up at once, they all
use the same 15 servers and sometimes have all of them on 64 second polling. 
(0.fedora.pool.ntp.org is CNAME 0.pool.ntp.org, which returns the same 5 IP
numbers until the DNS TTL expires, 1200 seconds = 20 minutes).

Ideally, a site ought to redetermine the nearby servers every month or so and
revise the configuration files...  There is no way to use anything but IP
numbers for local servers because the dig -x results might give a DNS name which
isn't always going to point to the timeserver.

Among all local boxes, there should be only about four or five minpoll 6 server
statements among all boxes, but if only a few boxes are up, and you don't know
which ones will be up, then the peer ntpd daemons ought to agree among
themselves to limit outside polling.  Lacking that, a script running at random
intervals could use ntpdc to look at the other peers and increase its own
minpoll if it sees enough other minpoll 6 servers configured.

With a really large number of boxes, most of them shouldn't consult outside
servers at all.  But if most of them are down (and you can't predict which),
then all should be _able_ to consult outside servers when necessary.


Comment 9 archimerged Ark submedes 2008-04-04 20:50:56 UTC
Created attachment 301339 [details]
ntp.conf using tos orphan and tos minsane statements.

This ntp.conf uses the options described above.  Each peer gets a different
/etc/ntp/ntpservers file, created by some hacked scripts to be attached. 
Depending on interest I can do more work in this direction.

Comment 10 archimerged Ark submedes 2008-04-04 20:56:20 UTC
Created attachment 301340 [details]
Script to find nearby timeservers from us.pool.ntp.org by pinging

This script makes a file saving results of pinging the IP numbers returned from
three different nameservers for {0,1,2}.us.pool.ntp.org.  Run it a few times at
widely spaced intervals to get a good chance to find the nearest servers.  The
next script will average the results.

Comment 11 archimerged Ark submedes 2008-04-04 21:01:35 UTC
Created attachment 301342 [details]
Script to figure out which timeservers are nearest.

Takes data files produced by previous script, sorts by IP number, averages ping
times if the same IP was encountered at different runs of previous script, and
sorts in order by average ping time.

(These are all quick and ugly scripts that I used just to avoid doing it by
hand).

Comment 12 archimerged Ark submedes 2008-04-04 21:07:22 UTC
Created attachment 301344 [details]
Assigns the nearby timeservers to the local peers so each list is different.

This script creates the /etc/ntp/ntpservers files, one for each local peer.
Each is different.  They include the server statements, so that each peer gets
one nearby server with minpoll 6, and three others with minpoll 10.  Also, each
peer gets the other peers listed on peer statements.

The script uses a list of IP numbers, which was created with this one-liner:

./summarize-nearby-timeservers | tail --lines=+2 | tr -s " " \\t | cut -f 1-1 >
nearby-timeserver-list

Comment 13 Bug Zapper 2008-05-14 02:38:03 UTC
Changing version to '9' as part of upcoming Fedora 9 GA.
More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 14 Fedora Update System 2008-07-01 09:53:18 UTC
system-config-date-1.9.32-1.fc8 has been submitted as an update for Fedora 8

Comment 15 Fedora Update System 2008-07-02 06:31:41 UTC
system-config-date-1.9.32-1.fc8 has been pushed to the Fedora 8 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update system-config-date'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/F8/FEDORA-2008-5927

Comment 16 Fedora Update System 2008-07-09 02:42:10 UTC
system-config-date-1.9.32-1.fc8 has been pushed to the Fedora 8 stable repository.  If problems still persist, please make note of it in this bug report.