Bug 1900898

Summary: Nov 23 22:55:56 ip-10-0-141-210 /usr/lib/systemd/system-generators/coreos-platform-chrony: line 59: /run/coreos-platform-chrony.conf: Permission denied
Product: OpenShift Container Platform Reporter: Colin Walters <walters>
Component: RHCOSAssignee: Timothée Ravier <travier>
Status: CLOSED DUPLICATE QA Contact: Michael Nguyen <mnguyen>
Severity: low Docs Contact:
Priority: low    
Version: 4.6CC: bbreard, imcleod, jligon, miabbott, nstielau
Target Milestone: ---Keywords: UpcomingSprint
Target Release: 4.8.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-02-08 16:30:23 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Colin Walters 2020-11-23 23:02:42 UTC
I saw this in the journal of a 4.6 cluster in AWS:

Nov 23 22:55:56 ip-10-0-141-210 /usr/lib/systemd/system-generators/coreos-platform-chrony: line 59: /run/coreos-platform-chrony.conf: Permission denied

Problem is SELinux of course
type=AVC msg=audit(1606172156.043:49): avc:  denied  { write } for  pid=1875 comm="coreos-platform" name="coreos-platform-chrony.conf" dev="tmpfs" ino=24703 scontext=system_u:system_r:init_t:s0 tcontext=system_u:object_r:etc_t:s0 tclass=file permissive=0

We really need an e2e that checks this.

Comment 1 Micah Abbott 2020-12-01 21:15:32 UTC
This seems like something we can address for 4.7; setting med/low for priority/severity.

Comment 2 Timothée Ravier 2020-12-04 14:42:51 UTC
Will investigate in the next sprint.

Comment 3 Timothée Ravier 2020-12-24 09:59:39 UTC
Will be for next sprint.

Comment 4 Timothée Ravier 2021-01-05 11:42:28 UTC
Just launched a cluster with cluster-bot and I do not have those errors.

# ls -alZ /run/coreos-platform-chrony.conf 
-rw-r--r--. 1 root root system_u:object_r:etc_t:s0 1514 Jan  5 11:09 /run/coreos-platform-chrony.conf

# journalctl -b 0 | grep chrony
Jan 05 11:09:00 localhost coreos-platform-chrony: Updated chrony to use aws configuration /run/coreos-platform-chrony.conf
Jan 05 11:09:05 localhost chronyd[1138]: chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP +SCFILTER +SIGND +ASYNCDNS +SECHASH +IPV6 +DEBUG)
Jan 05 11:09:05 localhost chronyd[1138]: Frequency 33.639 +/- 2.463 ppm read from /var/lib/chrony/drift
Jan 05 11:09:05 localhost chronyd[1138]: Using right/UTC timezone to obtain leap second data
Jan 05 11:09:12 ip-10-0-141-86 chronyd[1138]: Selected source 169.254.169.123
Jan 05 11:09:12 ip-10-0-141-86 chronyd[1138]: System clock TAI offset set to 37 seconds

# journalctl -b -1 | grep chrony
Jan 05 11:06:39 ip-10-0-141-86 coreos-platform-chrony: Updated chrony to use aws configuration /run/coreos-platform-chrony.conf
Jan 05 11:06:46 ip-10-0-141-86 chronyd[1485]: chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP +SCFILTER +SIGND +ASYNCDNS +SECHASH +IPV6 +DEBUG)
Jan 05 11:06:46 ip-10-0-141-86 chronyd[1485]: Using right/UTC timezone to obtain leap second data
Jan 05 11:06:54 ip-10-0-141-86 chronyd[1485]: Selected source 169.254.169.123
Jan 05 11:06:54 ip-10-0-141-86 chronyd[1485]: System clock TAI offset set to 37 seconds

Wondering if there could potentially be a race somewhere.

Comment 5 Timothée Ravier 2021-01-05 15:20:58 UTC
Output of:
# grep -i denied /var/log/audit/audit.log

is also empty on a fresh 4.6.9 cluster on AWS.

# rpm-ostree status
State: idle
Deployments:
* pivot://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee4994c87fd8aef1da4676820522de94c82679b9a248578f828592d1c17cafa
              CustomOrigin: Managed by machine-config-operator
                   Version: 46.82.202012151054-0 (2020-12-15T10:57:15Z)

Comment 6 Micah Abbott 2021-01-08 16:21:16 UTC
Comments #3 and #4, combined with a search of the last 2 weeks of CI jobs seems to point that this issue is hard to find/reproduce.

I think we can continue to pursue tooling up a test to check for SELinux denials, however I don't think there is anything to do here in terms of fixing RHCOS for 4.7

Going to set this to low priority and reset the Target Release so that we can revisit it in 4.8.

Comment 7 Timothée Ravier 2021-02-08 16:30:23 UTC
Nothing happened here for a month so closing as we can not reproduce the issue for now. Will re-open/investigate if we encounter it again.

Comment 8 Micah Abbott 2021-02-08 16:32:46 UTC
Ah, I forgot about this issue.  Something similar cropped up here - https://bugzilla.redhat.com/show_bug.cgi?id=1924869

Comment 9 Timothée Ravier 2021-02-08 16:45:26 UTC
Closing this one as duplicate of the other which as more info. Will investigate the new one.

*** This bug has been marked as a duplicate of bug 1924869 ***