Bug 918127 - Chronyd is flooding the audit system with time adjustment events
Summary: Chronyd is flooding the audit system with time adjustment events
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: chrony
Version: 18
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
Assignee: Miroslav Lichvar
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-03-05 14:42 UTC by Steve Grubb
Modified: 2013-08-06 00:22 UTC (History)
1 user (show)

Fixed In Version: chrony-1.28-1.fc18
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2013-07-02 00:33:41 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Steve Grubb 2013-03-05 14:42:44 UTC
Description of problem:
Users that require audit events typically have this set of rules:
-a always,exit -F arch=b32 -S adjtimex -S settimeofday -S stime -k time-change
-a always,exit -F arch=b64 -S adjtimex -S settimeofday -k time-change
-a always,exit -F arch=b32 -S clock_settime -F a0=0 -k time-change
-a always,exit -F arch=b64 -S clock_settime -F a0=0 -k time-change
-w /etc/localtime -p wa -k time-change

The problem is that by default, the chrony program adjusts time several times a millisecond continuously.
type=SYSCALL msg=audit(03/05/2013 05:47:55.745:26) : arch=x86_64 syscall=adjtimex success=yes exit=0 a0=0x7fff47656650 a1=0x0 a2=0x0 a3=0x4000 items=0 ppid=1462 pid=1463 auid=unset uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root ses=unset tty=(none) comm=chronyd exe=/usr/sbin/chronyd subj=system_u:system_r:chronyd_t:s0 key=time-change 
----
type=SYSCALL msg=audit(03/05/2013 05:47:55.745:27) : arch=x86_64 syscall=adjtimex success=yes exit=5 a0=0x7fff47656650 a1=0x0 a2=0x0 a3=0x4000 items=0 ppid=1462 pid=1463 auid=unset uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root ses=unset tty=(none) comm=chronyd exe=/usr/sbin/chronyd subj=system_u:system_r:chronyd_t:s0 key=time-change 
----
type=SYSCALL msg=audit(03/05/2013 05:47:55.745:28) : arch=x86_64 syscall=adjtimex success=yes exit=0 a0=0x7fff47656650 a1=0x0 a2=0x0 a3=0x4000 items=0 ppid=1462 pid=1463 auid=unset uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root ses=unset tty=(none) comm=chronyd exe=/usr/sbin/chronyd subj=system_u:system_r:chronyd_t:s0 key=time-change 
----
type=SYSCALL msg=audit(03/05/2013 05:47:55.745:29) : arch=x86_64 syscall=adjtimex success=yes exit=0 a0=0x7fff47656640 a1=0x7f4de6627550 a2=0x0 a3=0x4000 items=0 ppid=1462 pid=1463 auid=unset uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root ses=unset tty=(none) comm=chronyd exe=/usr/sbin/chronyd subj=system_u:system_r:chronyd_t:s0 key=time-change 
----
type=SYSCALL msg=audit(03/05/2013 05:47:55.745:30) : arch=x86_64 syscall=adjtimex success=yes exit=0 a0=0x7fff47656610 a1=0x7fff47656708 a2=0x7f4de640dc10 a3=0x4000 items=0 ppid=1462 pid=1463 auid=unset uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root ses=unset tty=(none) comm=chronyd exe=/usr/sbin/chronyd subj=system_u:system_r:chronyd_t:s0 key=time-change 

This is far too aggressive for casual use.


Version-Release number of selected component (if applicable):
chrony-1.27-0.5.pre1.git1ca844.fc18.x86_64

How reproducible:
Always

Steps to Reproduce:
1. Add the above rules to /etc/audit/audit.rules
2. Let system run
3. ausearch --start today -i --key time-change

Additional Information:
I think the solution is to provide a default configuration that syncs only occasionally rather than many times a millisecond. Also, I did not install this package by intention. It was pulled in by some dependencies. So, this is not being pulled in because I want my system super accurate.

Comment 1 Miroslav Lichvar 2013-03-05 15:00:34 UTC
I suspect most of these are read-only adjtimex calls (modes=0) to obtain the remaining offset correction when reading the clock. Normally, the clock should be modified only about once per 1024 seconds after the synchronization has stabilized.

Is it possible to extend the rule to check the modes field in the timex structure passed to the syscall?

You can use ntpd from the ntp package instead of chrony, it shouldn't call adjtimex as often as chronyd does.

Comment 2 Steve Grubb 2013-03-05 15:55:40 UTC
Its not possible to do anything with the adjtimex syscall because it has one argument and its a pointer to a user space timex struct. The audit system can only check arguments passed by value rather than dereference pointers to the value.

The synchronization never seems to settle. My system has been on 2 hours and I have a little over 1900 events. By default, this is too aggressive.

As for ntpd, well...this package is required by revisor and anaconda. So, its pulled in and the daemon is enabled by default. This leads to flooding the audit logs. If ntpd does it, I'll open a bz on that package as well. But out of the box, this daemon is too aggressive. For example, looking at how often it does something:

10:39:12.791:1303
10:39:12.791:1304
10:39:12.911:1305
10:39:12.911:1306
10:39:12.911:1307
10:39:12.791:1302
10:40:59.077:1309
10:40:59.077:1310
10:40:59.184:1311
10:40:59.184:1312
10:40:59.184:1313
10:40:59.077:1308
10:43:25.722:1315
10:43:25.722:1316
10:43:25.811:1317
10:43:25.811:1318
10:43:25.811:1319
10:43:25.722:1314
10:47:34.406:1321
10:47:34.406:1322
10:47:34.448:1323
10:47:34.448:1324
10:47:34.448:1325
10:47:34.448:1326
10:47:34.406:1320
10:47:34.448:1327
10:47:34.448:1328
10:47:34.448:1329
10:47:34.448:1330
10:47:34.448:1331
10:47:48.475:1333
10:47:48.475:1334
10:47:48.597:1335
10:47:48.597:1336
10:47:48.597:1337
10:47:48.475:1332

This is hour:minute:second.millisecond:serial number within the millisecond. Its seems to let the system go for a couple minutes or a couple seconds, but then starts all the syscalls again.

Comment 3 Miroslav Lichvar 2013-03-05 17:22:06 UTC
Perhaps the audit daemon could be improved to merge identical successive messages?

By default there are 4 servers in chrony.conf. The initial polling interval is 64 seconds, which is gradually increased to 1024 seconds. For each server query there are 6 time stamps made (adjtimex calls). One after select() timeouts, two before the packet is send, one for select() when the packet is received, one to correct SO_TIMESTAMP and another in source selection.

I think it's working as expected. From that six calls two or three could probably be eliminated/cached, but I'm not sure if that would help you.

There shouldn't be any hard dependencies on chronyd in default install as it is supposed to be replaceable with ntp, it might be worth to file some bugs.

Comment 4 Miroslav Lichvar 2013-06-06 17:44:39 UTC
The upstream code has now reduced the number of adjtimex calls approximately by half. I think this is best what can we do now.

http://git.tuxfamily.org/chrony/chrony.git/?p=chrony/chrony.git;a=commitdiff;h=8aa9eb19c826f3ea4a9c2ce5e3a71b74aeeb99d0;hp=62027f1b47ada22a910d02d7eb953d2e6a45bf84

Comment 5 Steve Grubb 2013-06-06 18:14:18 UTC
Thanks. It is a mandatory requirement on government system to audit any change to the clock because that could mess up correlation of logs should an intruder try to make it look like something happened at a different time.

Comment 6 Fedora Update System 2013-06-21 16:31:25 UTC
chrony-1.28-0.1.pre1.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/chrony-1.28-0.1.pre1.fc19

Comment 7 Fedora Update System 2013-06-21 19:15:37 UTC
Package chrony-1.28-0.1.pre1.fc19:
* should fix your issue,
* was pushed to the Fedora 19 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing chrony-1.28-0.1.pre1.fc19'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-11434/chrony-1.28-0.1.pre1.fc19
then log in and leave karma (feedback).

Comment 8 Fedora Update System 2013-07-02 00:33:41 UTC
chrony-1.28-0.1.pre1.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 9 Fedora Update System 2013-07-18 15:12:06 UTC
chrony-1.28-1.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/chrony-1.28-1.fc18

Comment 10 Fedora Update System 2013-08-06 00:22:14 UTC
chrony-1.28-1.fc18 has been pushed to the Fedora 18 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.