Bug 1478074

Summary: Fedora 26 upgrade breaks ntp pps
Product: [Fedora] Fedora Reporter: udo <udovdh>
Component: ntpAssignee: Miroslav Lichvar <mlichvar>
Status: CLOSED NOTABUG QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 26CC: linville, mlichvar, udovdh
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-02-27 10:28:23 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description udo 2017-08-03 14:16:34 UTC
Description of problem:
After upgrading to Fedora 26 fro 24 I noticed that ntpd no longer 'sees' my Garmin GPS18LVC GPS freceiver nor any external ntp servers as shown in `ntpq -pn`.

Version-Release number of selected component (if applicable):
# rpm -q ntp
error: bad option 'ptflags' at (null):19
ntp-4.2.8p10-1.fc26.i686
Yes, notice the additional error that was not there while running Fedora 24.

How reproducible:
Have working Fedora 24 ntp setup with gps receiver using PPSapi like Garmin GPS18LVC
Upgrade to Fedora 26 using dnf.
After upgrade check the gps using ntpq -pn (etc)


Actual results:
All clocks gone.

Expected results:
No clocks lost.

Additional info:
In /var/log/messages we find:
Aug  3 16:10:04 epia ntpd[9944]: GPS_NMEA(0) set PPSAPI params fails
Aug  3 16:10:04 epia ntpd[9944]: giving up resolving host surfplank2: Servname not supported for ai_socktype (-8)
Aug  3 16:10:04 epia ntpd[9944]: giving up resolving host ntp.xs4all.nl: Servname not supported for ai_socktype (-8)
Aug  3 16:10:04 epia ntpd[9944]: giving up resolving host ntp2.xs4all.nl: Servname not supported for ai_socktype (-8)
Aug  3 16:10:04 epia ntpd[9944]: giving up resolving host ntp0.nl.net: Servname not supported for ai_socktype (-8)
Aug  3 16:10:04 epia ntpd[9944]: giving up resolving host ntp2.nl.net: Servname not supported for ai_socktype (-8)
Aug  3 16:10:04 epia ntpd[9944]: giving up resolving host keetweej.vanheusden.com: Servname not supported for ai_socktype (-8)
Aug  3 16:10:04 epia ntpd[9944]: giving up resolving host ntp.nmi.nl: Servname not supported for ai_socktype (-8)

Comment 1 Miroslav Lichvar 2017-08-03 14:20:51 UTC
Do you set flag3 to 1 for the NMEA refclock? The Fedora kernel doesn't support PPS kernel discipline and older ntp versions silently ignored the error.

Comment 2 udo 2017-08-03 14:22:16 UTC
# grep ^ntp /etc/services 
ntp             123/tcp
ntp             123/udp                         # Network Time Protocol
# grep ^services nsswitch.conf
services:   files
#

Kernel 4.9.40 has:

# grep PPS .config
# PPS support
CONFIG_PPS=y
# CONFIG_PPS_DEBUG is not set
# PPS clients support
CONFIG_PPS_CLIENT_KTIMER=m
CONFIG_PPS_CLIENT_LDISC=y
# CONFIG_PPS_CLIENT_PARPORT is not set
# CONFIG_PPS_CLIENT_GPIO is not set
# PPS generators support
#

Just like previous kernels that worked wonders.

Comment 3 udo 2017-08-03 14:23:02 UTC
I use my own kernel.
Because.

And now two things break.
PPS.
And network clocks.

Comment 4 udo 2017-08-03 14:27:16 UTC
I did not change anything in ntp.conf for ages:

# grep -v ^# ntp.conf 


server 127.127.20.0 minpoll 4 
fudge 127.127.20.0 flag3 1 flag2 0 flag1 1 time1 0.00000006 time2 0.160

disable monitor

restrict default kod nomodify notrap nopeer noquery
restrict -6 default kod nomodify notrap nopeer noquery

server surfplank2
server ntp.xs4all.nl 
server ntp2.xs4all.nl 
server ntp0.nl.net
server ntp2.nl.net
server keetweej.vanheusden.com 
server ntp.nmi.nl

driftfile /var/lib/ntp/drift
broadcastdelay  0.008
logconfig -syncstatus -sysevents
logfile /dev/log


discard average 5 minimum 2 

restrict 127.127.0.0      mask 255.255.0.0  nopeer # internal clocks
restrict 127.0.0.1        mask 255.255.255.255 # accept local network
restrict ::1        		# accept local network
restrict 192.168.10.0 mask 255.255.255.0 nomodify notrap nopeer # notrust

statistics loopstats
statsdir /var/log/ntp/
filegen peerstats file peers type day link disable
filegen loopstats file loops type day enable
#

Comment 5 udo 2017-08-03 14:33:32 UTC
Even worse:

When ntpd does not run in chroot things appear OK.
When in chroot the symptoms as described happen.
So the issue has to do with chroot.
strace did not yet yield any clues.

Comment 6 udo 2017-08-03 14:39:49 UTC
In chroot /dev/log is no longer found although /dev/log is present:

# ls -l /dev/log
srw-rw-rw- 1 root root 0 Aug  3 15:39 /dev/log
# ls -l /chroot/ntpd/dev/log 
srw-rw-rw- 1 root root 0 Aug  3 15:39 /chroot/ntpd/dev/log

Yet we see:

Aug  3 16:34:36 epia ntpd[10618]: Cannot open logfile /dev/log: No such device or address

And perhaps we can explain these:

Aug  3 16:34:36 epia ntpd[10618]: refclock_params: time_pps_kcbind: Operation not supported
Aug  3 16:34:36 epia ntpd[10618]: GPS_NMEA(0) set PPSAPI params fails

by the GPS serial ports not being found in the chroot /dev but in the system /dev.

This has worked for years but is now broken.

Comment 7 Miroslav Lichvar 2017-08-03 14:49:40 UTC
Your kernel config is missing CONFIG_NTP_PPS. Please recompile the kernel with that option enabled (you will need to disable NO_HZ) or remove "flag3 1" from ntp.conf.

As for the chroot, does it include the /etc/service file?

Comment 8 udo 2017-08-03 14:53:40 UTC
Kernel 4.9.40 .config does not have CONFIG_NTP_PPS.

Having services and nsswitch.conf in /chroot/ntp/etc does not have any impact, behaviour does not change:

# ls -l /chroot/ntpd/etc/
total 684
-rw-r--r--  2 root root   1834 Aug  3 16:18 nsswitch.conf
drwxrwx---. 2 root ntp    4096 Oct 24  2004 ntp
-rw-r-----. 2 root ntp    3315 Jun  8  2015 ntp.conf
-rw-r--r--  2 root root 687295 Dec  7  2016 services

Comment 9 udo 2017-08-03 14:58:36 UTC
When not running in chroot, ntpq -pn reports all local and remote clocks.
But:

# /usr/sbin/ntpdc -c loopinfo
localhost.localdomain: timed out, nothing received
***Request timed out

Why?
This worked before the upgrade.

Comment 10 Miroslav Lichvar 2017-08-03 15:03:48 UTC
Have you tried "strace -ff ntpd -d -u ntp |& grep open" to see what files it is missing?

ntpdc doesn't work in default configuration. The protocol is disabled by default as it allows very large traffic amplification. It can be enabled by adding "enable mode7" to ntp.conf.

Comment 11 udo 2017-08-03 15:15:13 UTC
enable mode7 added, thanks.

the strace reveals that, besides some libs that are not on my system anyways, only /dev/log is not found.
Yet /dev/log and /chroot/ntpd/dev/log both exist.

Comment 12 Miroslav Lichvar 2018-02-27 10:28:23 UTC
If I understand it correctly, all issues mentioned here were either resolved by updating the ntp configuration or need to be addressed outside ntp (kernel configuration, chroot setup). I'm closing the bug.