Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1977730

Summary:	Different file permission for secrets/user-serving-cert-000/tls.crt and secrets/user-serving-cert-000/tls.key
Product:	OpenShift Container Platform	Reporter:	Divyam Pateriya <dpateriy>
Component:	kube-apiserver	Assignee:	Emily Moss <emoss>
Status:	CLOSED ERRATA	QA Contact:	Rahul Gangwar <rgangwar>
Severity:	high	Docs Contact:
Priority:	medium
Version:	4.7	CC:	aos-bugs, dtambat, emoss, kvatteka, mfojtik, mirollin, mrobson, nkaushik, pawankum, rdey, rgangwar, sbiradar, shchan, simore, sttts, surbania, suyama, wking, xxia
Target Milestone:	---	Flags:	emoss: needinfo+
Target Release:	4.9.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:	Cause: Inconsistent permissions Consequence: Need consistent Fix: All 600s Result: Now consistent	Story Points:	---
Clone Of:		Environment:
Last Closed:	2022-01-10 08:50:46 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	2013838, 2026089

Description Divyam Pateriya 2021-06-30 11:30:17 UTC

Description of problem: 

Incorrect file permission for the below files.

-rw-r--r--. 1 root root 6030 Jun 24 10:44 /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/user-serving-cert-000/tls.crt
-rw-r--r--. 1 root root 1951 Jun 24 10:44 /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/user-serving-cert-000/tls.key



Version-Release number of selected component (if applicable):

OCP 4.6.19, 4.7

How reproducible:

NotSure.

Steps to Reproduce:

1. Only certificates belonging to the Custom API Name certificate for the cluster has different file permission of 0644. 


2. Rest all the files have the desired 0600 file permission.


Actual results:

All the files should have the 0600 file permission.

Expected results:

Only these two files have different 0644 "/etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/user-serving-cert-000/tls.crt" and "/etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/user-serving-cert-000/tls.key "

Additional info:

Comment 1 Stefan Schimanski 2021-06-30 12:26:07 UTC

4.6.19 is not the latest release, but 4.6.36 is. Please verify on the latter before opening bugs.

Comment 23 Suman Yama 2021-08-18 04:51:11 UTC

I am seeing this permission issue on 4.6.39 at customer cluster.

# ls -l /etc/kubernetes/static-pod-resources/kube-*/secrets/*/tls.crt | grep -v "rw-------"
-rw-r--r--. 1 root root 7405 Aug 17 10:17 /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/user-serving-cert-000/tls.crt
# ls -l /etc/kubernetes/static-pod-resources/kube-*/secrets/*/tls.key | grep -v "rw-------"
-rw-r--r--. 1 root root 1704 Aug 17 10:17 /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/user-serving-cert-000/tls.key

[root@slmcpapor1c ~]# oc version
Client Version: 4.5.9
Server Version: 4.6.39
Kubernetes Version: v1.19.0+4c3480d
[root@slmcpapor1c ~]#

Comment 24 pawankum 2021-08-18 12:35:11 UTC

Hello Team,

Any update on this?


Regards,
Pawan Kumar

Comment 25 pawankum 2021-08-20 07:51:41 UTC

Hello Team,

Is there any way around for 4.6.39 version?



Regards,
Pawan Kumar

Comment 26 pawankum 2021-08-31 12:28:28 UTC

Hello Team,

Is there any update on this?



Regards,
Pawan Kumar

Comment 42 Rahul Gangwar 2021-09-30 06:54:47 UTC

oc get clusterversion
NAME      VERSION                              AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.10.0-0.nightly-2021-09-29-060153   True        False         121m    Cluster version is 4.10.0-0.nightly-2021-09-29-060153

Created self-signed certs for API:
openssl req -newkey rsa:2048 -nodes -keyout key.pem -x509 -days 365 -out certificate.pem  -subj "/CN=<API_FQDN>"
Generating a 2048 bit RSA private key
.................................+++
...........+++
writing new private key to 'key.pem'
-----

Created secret in openshift-config namespace:
oc create secret tls api-secret --cert=certificate.pem  --key=key.pem  -n openshift-config

secret/api-secret created

Updated the API server to reference the created secret:
oc get apiserver cluster -o yaml

~~~
servingCertificate:
  name: api-secret

All masters have 600 permissions for certs.

for i in `oc get node|grep master|awk '{print $1}'`;do oc debug node/$i -T  -- chroot /host bash -c "ls -ltrh /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/*/tls.crt"; done

W0930 12:13:11.760736   81390 warnings.go:70] would violate "latest" version of "baseline" PodSecurity profile: host namespaces (hostNetwork=true, hostPID=true), hostPath volumes (volume "host"), privileged (container "container-00" must not set securityContext.privileged=true)
Starting pod/ip-10-0-146-168us-east-2computeinternal-debug ...
To use host binaries, run `chroot /host`
-rw-------. 1 root root 1.2K Sep 30 05:08 /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/aggregator-client/tls.crt
-rw-------. 1 root root 2.5K Sep 30 05:08 /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/internal-loadbalancer-serving-certkey/tls.crt
-rw-------. 1 root root 2.5K Sep 30 05:08 /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/external-loadbalancer-serving-certkey/tls.crt
-rw-------. 1 root root 1.2K Sep 30 05:08 /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/control-plane-node-admin-client-cert-key/tls.crt
-rw-------. 1 root root 1.2K Sep 30 05:08 /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/check-endpoints-client-cert-key/tls.crt
-rw-------. 1 root root 2.7K Sep 30 05:08 /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/service-network-serving-certkey/tls.crt
-rw-------. 1 root root 2.4K Sep 30 05:08 /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/localhost-serving-cert-certkey/tls.crt
-rw-------. 1 root root 1.2K Sep 30 05:08 /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/kubelet-client/tls.crt
-rw-------. 1 root root  981 Sep 30 05:08 /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/user-serving-cert-000/tls.crt

Removing debug pod ...
W0930 12:13:15.882078   81391 warnings.go:70] would violate "latest" version of "baseline" PodSecurity profile: host namespaces (hostNetwork=true, hostPID=true), hostPath volumes (volume "host"), privileged (container "container-00" must not set securityContext.privileged=true)
Starting pod/ip-10-0-182-214us-east-2computeinternal-debug ...
To use host binaries, run `chroot /host`
-rw-------. 1 root root 1.2K Sep 30 05:03 /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/check-endpoints-client-cert-key/tls.crt
-rw-------. 1 root root 1.2K Sep 30 05:03 /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/aggregator-client/tls.crt
-rw-------. 1 root root 2.4K Sep 30 05:03 /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/localhost-serving-cert-certkey/tls.crt
-rw-------. 1 root root 1.2K Sep 30 05:03 /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/kubelet-client/tls.crt
-rw-------. 1 root root 2.5K Sep 30 05:03 /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/internal-loadbalancer-serving-certkey/tls.crt
-rw-------. 1 root root 2.5K Sep 30 05:03 /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/external-loadbalancer-serving-certkey/tls.crt
-rw-------. 1 root root 1.2K Sep 30 05:03 /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/control-plane-node-admin-client-cert-key/tls.crt
-rw-------. 1 root root  981 Sep 30 05:03 /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/user-serving-cert-000/tls.crt
-rw-------. 1 root root 2.7K Sep 30 05:03 /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/service-network-serving-certkey/tls.crt

Removing debug pod ...
W0930 12:13:21.353581   81392 warnings.go:70] would violate "latest" version of "baseline" PodSecurity profile: host namespaces (hostNetwork=true, hostPID=true), hostPath volumes (volume "host"), privileged (container "container-00" must not set securityContext.privileged=true)
Starting pod/ip-10-0-193-73us-east-2computeinternal-debug ...
To use host binaries, run `chroot /host`
-rw-------. 1 root root 1.2K Sep 30 05:13 /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/control-plane-node-admin-client-cert-key/tls.crt
-rw-------. 1 root root 1.2K Sep 30 05:13 /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/check-endpoints-client-cert-key/tls.crt
-rw-------. 1 root root 1.2K Sep 30 05:13 /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/aggregator-client/tls.crt
-rw-------. 1 root root 2.7K Sep 30 05:13 /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/service-network-serving-certkey/tls.crt
-rw-------. 1 root root 2.4K Sep 30 05:13 /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/localhost-serving-cert-certkey/tls.crt
-rw-------. 1 root root 1.2K Sep 30 05:13 /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/kubelet-client/tls.crt
-rw-------. 1 root root 2.5K Sep 30 05:13 /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/internal-loadbalancer-serving-certkey/tls.crt
-rw-------. 1 root root 2.5K Sep 30 05:13 /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/external-loadbalancer-serving-certkey/tls.crt
-rw-------. 1 root root  981 Sep 30 05:13 /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/user-serving-cert-000/tls.crt

Removing debug pod ...

Comment 44 Emily Moss 2021-10-11 19:09:08 UTC

*** Bug 2009763 has been marked as a duplicate of this bug. ***

Comment 55 Maciej Szulik 2021-11-30 16:02:19 UTC

*** Bug 2027717 has been marked as a duplicate of this bug. ***

Comment 60 errata-xmlrpc 2022-01-10 08:50:46 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.9.13 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:0029

Comment 62 Red Hat Bugzilla 2023-09-15 01:10:47 UTC

The needinfo request[s] on this closed bug have been removed as they have been unresolved for 500 days