1717970 – machine-config-daemon mark node as degraded due to "open /etc/machine-config-daemon/node-annotations.json: no such file or directory"

Bug 1717970 - machine-config-daemon mark node as degraded due to "open /etc/machine-config-daemon/node-annotations.json: no such file or directory"

Summary: machine-config-daemon mark node as degraded due to "open /etc/machine-config-...

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	OpenShift Container Platform
Classification:	Red Hat
Component:	Machine Config Operator
Sub Component:
Version:	4.2.0
Hardware:	Unspecified
OS:	Unspecified
Priority:	high
Severity:	high
Target Milestone:	---
Target Release:	4.2.0
Assignee:	Antonio Murdaca
QA Contact:	Micah Abbott
Docs Contact:
URL:
Whiteboard:
Depends On:	1702626
Blocks:
TreeView+	depends on / blocked

Reported:	2019-06-06 14:44 UTC by Micah Abbott
Modified:	2019-10-16 06:31 UTC (History)
CC List:	8 users (show)
Fixed In Version:
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:	1702626
Environment:
Last Closed:	2019-10-16 06:31:35 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Product Errata	RHBA-2019:2922	0	None	None	None	2019-10-16 06:31:52 UTC

Comment 1 Antonio Murdaca 2019-06-06 14:46:06 UTC

https://github.com/openshift/machine-config-operator/pull/807

Comment 3 weiwei jiang 2019-06-28 11:43:05 UTC

Checked with 4.2.0-0.nightly-2019-06-28-003744, and found node have no machineconfig annotation anymore.

===> node:> ip-10-0-165-52.ap-northeast-1.compute.internal
machine.openshift.io/machine : openshift-machine-api/wjiang-ocp-5dk47-worker-ap-northeast-1d-c5djd
machineconfiguration.openshift.io/reason : machineconfig.machineconfiguration.openshift.io "" not found
machineconfiguration.openshift.io/state : Degraded
volumes.kubernetes.io/controller-managed-attach-detach : true

Comment 4 Antonio Murdaca 2019-07-02 11:40:04 UTC

Rested this myself on 4.2.0-0.okd-2019-07-02-091950 and it works, the correct steps to reproduce are:


- deploy ssh bastion
- grab node ip
- oc delete node/<node>
- ssh to the node
- systemctl restart kubelet
- node comes up again
- wait some time and node will have annotations again (also watch oc get mcp)


Please retest this using the above steps

Comment 6 weiwei jiang 2019-07-25 10:18:30 UTC

Checked with 4.2.0-0.nightly-2019-07-24-220922, this has been fixed from the final result.


After the worker come back, there is a short time worker is marked as degraded.

===> node:> ip-10-0-152-181.us-east-2.compute.internal                                                                                  
machine.openshift.io/machine : openshift-machine-api/wjiang0725-bsbg6-worker-us-east-2b-lkx7s
machineconfiguration.openshift.io/currentConfig : rendered-worker-745ec85447a402109496addd8ed608ce
machineconfiguration.openshift.io/desiredConfig : rendered-worker-745ec85447a402109496addd8ed608ce
machineconfiguration.openshift.io/reason : machineconfiguration.openshift.io/desiredConfig annotation not found in &Node{ObjectMeta:k8s_io_apimachinery_pkg_apis_meta_v1.ObjectMeta{Name:ip-10-0-152-181.us-east-2.compute.internal,GenerateName:,Namespace:,SelfLink:/api/v1/n$
des/ip-10-0-152-181.us-east-2.compute.internal,UID:e0e04016-aec4-11e9-82e5-0a2cfb5135f6,ResourceVersion:36216,Generation:0,CreationTimestamp:2019-07-25 10:13:42 +0000 UTC,DeletionTimestamp:<nil>,DeletionGracePeriodSeconds:nil,Labels:map[string]string{beta.kubernetes.io/a$
ch: amd64,beta.kubernetes.io/instance-type: m4.large,beta.kubernetes.io/os: linux,failure-domain.beta.kubernetes.io/region: us-east-2,failure-domain.beta.kubernetes.io/zone: us-east-2b,kubernetes.io/arch: amd64,kubernetes.io/hostname: ip-10-0-152-181,kubernetes.io/os: li$
ux,node-role.kubernetes.io/worker: ,node.openshift.io/os_id: rhcos,},Annotations:map[string]string{machine.openshift.io/machine: openshift-machine-api/wjiang0725-bsbg6-worker-us-east-2b-lkx7s,machineconfiguration.openshift.io/currentConfig: rendered-worker-745ec85447a402$
09496addd8ed608ce,machineconfiguration.openshift.io/reason: machineconfiguration.openshift.io/desiredConfig annotation not found in &Node{ObjectMeta:k8s_io_apimachinery_pkg_apis_meta_v1.ObjectMeta{Name:ip-10-0-152-181.us-east-2.compute.internal,GenerateName:,Namespace:,Se
lfLink:/api/v1/nodes/ip-10-0-152-181.us-east-2.compute.internal,UID:e0e04016-aec4-11e9-82e5-0a2cfb5135f6,ResourceVersion:36214,Generation:0,CreationTimestamp:2019-07-25 10:13:42 +0000 UTC,DeletionTimestamp:<nil>,DeletionGracePeriodSeconds:nil,Labels:map[string]string{beta
.kubernetes.io/arch: amd64,beta.kubernetes.io/instance-type: m4.large,beta.kubernetes.io/os: linux,failure-domain.beta.kubernetes.io/region: us-east-2,failure-domain.beta.kubernetes.io/zone: us-east-2b,kubernetes.io/arch: amd64,kubernetes.io/hostname: ip-10-0-152-181,kube
rnetes.io/os: linux,node-role.kubernetes.io/worker: ,node.openshift.io/os_id: rhcos,},Annotations:map[string]string{machine.openshift.io/machine: openshift-machine-api/wjiang0725-bsbg6-worker-us-east-2b-lkx7s,machineconfiguration.openshift.io/currentConfig: rendered-worke
r-745ec85447a402109496addd8ed608ce,volumes.kubernetes.io/controller-managed-attach-detach: true,},OwnerReferences:[],Finalizers:[],ClusterName:,Initializers:nil,ManagedFields:[],},Spec:NodeSpec{PodCIDR:,DoNotUse_ExternalID:,ProviderID:aws:///us-east-2b/i-0b7eaf2018327bda0
,Unschedulable:false,Taints:[{node.kubernetes.io/not-ready  NoExecute 2019-07-25 10:13:53 +0000 UTC}],ConfigSource:nil,},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-aws-ebs: {{39 0} {<nil>} 39 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},hugepages-1Gi: {{0 0
} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{8358801408 0} {<nil>} 8162892Ki BinarySI},pods: {{250 0} {<nil>} 250 DecimalSI},},Allocatable:ResourceList{attachable-volumes-aws-ebs: {{39 0} {<nil>} 39 DecimalSI},cpu: {{1500 -3} {<nil>} 1500m D
ecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7729655808 0} {<nil>} 7548492Ki BinarySI},pods: {{250 0} {<nil>} 250 DecimalSI},},Phase:,Conditions:[{MemoryPressure False 2019-07-25 10:14:02 +0000 UTC 2019-07-25 10
:13:42 +0000 UTC KubeletHasSufficientMemory kubelet has sufficient memory available} {DiskPressure False 2019-07-25 10:14:02 +0000 UTC 2019-07-25 10:13:42 +0000 UTC KubeletHasNoDiskPressure kubelet has no disk pressure} {PIDPressure False 2019-07-25 10:14:02 +0000 UTC 201
9-07-25 10:13:42 +0000 UTC KubeletHasSufficientPID kubelet has sufficient PID available} {Ready True 2019-07-25 10:14:02 +0000 UTC 2019-07-25 10:14:02 +0000 UTC KubeletReady kubelet is posting ready status}],Addresses:[{InternalIP 10.0.152.181} {InternalDNS ip-10-0-152-18
1.us-east-2.compute.internal} {Hostname ip-10-0-152-181.us-east-2.compute.internal}],DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:75ec6cc3ccf74da5899237f049dbd8eb,SystemUUID:ec2ad01d-af29-91d5-ba90-c6d
74b234c69,BootID:1e888353-2b00-437a-8d19-46d7cd501605,KernelVersion:4.18.0-80.4.2.el8_0.x86_64,OSImage:Red Hat Enterprise Linux CoreOS 42.80.20190724.2 (Ootpa),ContainerRuntimeVersion:cri-o://1.14.10-0.2.dev.rhaos4.2.git381fefe.el8-dev,KubeletVersion:v1.14.0+b3e2c8a2b,Kub
eProxyVersion:v1.14.0+b3e2c8a2b,OperatingSystem:linux,Architecture:amd64,},Images:[{[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:62a1cd7943b7df70c5c611cb9e93e31aaa42cddb12eb6c691b2196545d8d0c93 <none>:<none>] 753774966} {[quay.io/openshift-release-dev/ocp-v4.0-a
rt-dev@sha256:cc8bd647863d4be2f322d0192bd84cf7165e57b798fc8d77d0eda040e4978b0e <none>:<none>] 397664402} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b1fce04f26df0bb8738fd327a258046cbe70f81bc14441b94efd3429505b0d85 <none>:<none>] 394200757} {[quay.io/openshift-
release-dev/ocp-v4.0-art-dev@sha256:da4ad4a71f87cb0960390a24f55163902fe4cc0cf86c1056bc22e83094b765f2 <none>:<none>] 393752553} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39e6dfd5e30e001bf1891d6d9a22f3df46b000a77769c1f986fd2de607d186d7 <none>:<none>] 324200904
} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c35f0c7c00bca4b4491147688d4e52c100d8cb1fb9e7ed66a00a4502fe70e7e0 <none>:<none>] 322021040} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9dd7e3e89af03eb7b6e388c1d0dbef37a75ab465ec23353f6bd9150cf2124e84 <n
one>:<none>] 307171281} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2cee6f2f1cf76e9b39c180a2771e91238a954038d8761b83ff66c797bfedfc5e <none>:<none>] 305342520} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:04c9c317a5547df040987ee4f0fdefe4b117f29989957
5bc90f4b3491a82cf02 <none>:<none>] 304033253} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:66d4a2a6ec11525a8b1cbd8d3c0445cf06a14d84789df2eceacb9e67b9962e2e <none>:<none>] 299596966} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:96392554334baef6be2a63f
374373b3650bafba8bbe2a5c61e4e22654c21d702 <none>:<none>] 296997571} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:adb2c6b747142c2384a3f08903decd1626faa1416f84ef3bb2cd4eb197fc3116 <none>:<none>] 293949901} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2
01c3c0da91165392995c11ed8392c08f04166dad7b04d6579b341c966bc3476 <none>:<none>] 285886451} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7b23b04d9265407b7d563b3ac13b2c20a2834b2c971b42e7627f939af31ae9cd <none>:<none>] 283675187} {[quay.io/openshift-release-dev/ocp
-v4.0-art-dev@sha256:ee217e5f4f985f1f1bc8c1dbf428deb25b212b49556f1574fa62620687863c39 <none>:<none>] 280959414} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:88406eca7634b9b84af0c744c097f3ba2878c2dd181a32ba0285edbbdb3c1a83 <none>:<none>] 279777314} {[quay.io/ope
nshift-release-dev/ocp-v4.0-art-dev@sha256:d02641dabf58927d8761562dbd5b181c35b23eb8950dd9e2959748db0cc5df17 <none>:<none>] 278500475} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:30e670f00cb90a1320abc2af1ffa32428a8ccabf8916e82e27fb61e5b8a754bb <none>:<none>] 27
7994559} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0d7dd33a3883ba3248e4e106fcead1ecd63a29eba682f93ccb605224e3e5ff7 <none>:<none>] 268686661} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6d548e915b46181087e5c8d4e84ac67b9d9acadea7b0bc5b55e282634815
0b1f <none>:<none>] 264236964} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1d4967eedf1759ececd718af5e69ca07b89a278d7fc1afc6ff1988e640437712 <none>:<none>] 258059345} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:97e255db208b9da546b1a3519246502acda1ca
54cecb294010f36a8619a9453f <none>:<none>] 257007720} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9b89e94a0215eaeede16fb8fb8a99f8b0ecd43615839eb788d5bbebadb971623 <none>:<none>] 251426306} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0e0ace13114e94b
fababb4517d0de6aadd268fc4b3679c8a811d0d396322656 <none>:<none>] 246009528} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:aff6a7616278d03b23ee67dc0ae767d723b57b85aeb41d0697f9fc14e805dcd7 <none>:<none>] 239666982}],VolumesInUse:[],VolumesAttached:[],Config:nil,},}
,machineconfiguration.openshift.io/state: Degraded,volumes.kubernetes.io/controller-managed-attach-detach: true,},OwnerReferences:[],Finalizers:[],ClusterName:,Initializers:nil,ManagedFields:[],},Spec:NodeSpec{PodCIDR:,DoNotUse_ExternalID:,ProviderID:aws:///us-east-2b/i-0
b7eaf2018327bda0,Unschedulable:false,Taints:[],ConfigSource:nil,},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-aws-ebs: {{39 0} {<nil>} 39 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 
DecimalSI},memory: {{8358801408 0} {<nil>} 8162892Ki BinarySI},pods: {{250 0} {<nil>} 250 DecimalSI},},Allocatable:ResourceList{attachable-volumes-aws-ebs: {{39 0} {<nil>} 39 DecimalSI},cpu: {{1500 -3} {<nil>} 1500m DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hu
gepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7729655808 0} {<nil>} 7548492Ki BinarySI},pods: {{250 0} {<nil>} 250 DecimalSI},},Phase:,Conditions:[{MemoryPressure False 2019-07-25 10:14:02 +0000 UTC 2019-07-25 10:13:42 +0000 UTC KubeletHasSufficientMemory kubelet has
 sufficient memory available} {DiskPressure False 2019-07-25 10:14:02 +0000 UTC 2019-07-25 10:13:42 +0000 UTC KubeletHasNoDiskPressure kubelet has no disk pressure} {PIDPressure False 2019-07-25 10:14:02 +0000 UTC 2019-07-25 10:13:42 +0000 UTC KubeletHasSufficientPID kube
let has sufficient PID available} {Ready True 2019-07-25 10:14:02 +0000 UTC 2019-07-25 10:14:02 +0000 UTC KubeletReady kubelet is posting ready status}],Addresses:[{InternalIP 10.0.152.181} {InternalDNS ip-10-0-152-181.us-east-2.compute.internal} {Hostname ip-10-0-152-181
.us-east-2.compute.internal}],DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:75ec6cc3ccf74da5899237f049dbd8eb,SystemUUID:ec2ad01d-af29-91d5-ba90-c6d74b234c69,BootID:1e888353-2b00-437a-8d19-46d7cd501605,K
ernelVersion:4.18.0-80.4.2.el8_0.x86_64,OSImage:Red Hat Enterprise Linux CoreOS 42.80.20190724.2 (Ootpa),ContainerRuntimeVersion:cri-o://1.14.10-0.2.dev.rhaos4.2.git381fefe.el8-dev,KubeletVersion:v1.14.0+b3e2c8a2b,KubeProxyVersion:v1.14.0+b3e2c8a2b,OperatingSystem:linux,A
rchitecture:amd64,},Images:[{[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:62a1cd7943b7df70c5c611cb9e93e31aaa42cddb12eb6c691b2196545d8d0c93 <none>:<none>] 753774966} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cc8bd647863d4be2f322d0192bd84cf7165e57b79
8fc8d77d0eda040e4978b0e <none>:<none>] 397664402} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b1fce04f26df0bb8738fd327a258046cbe70f81bc14441b94efd3429505b0d85 <none>:<none>] 394200757} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:da4ad4a71f87cb09603
90a24f55163902fe4cc0cf86c1056bc22e83094b765f2 <none>:<none>] 393752553} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39e6dfd5e30e001bf1891d6d9a22f3df46b000a77769c1f986fd2de607d186d7 <none>:<none>] 324200904} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha2
56:c35f0c7c00bca4b4491147688d4e52c100d8cb1fb9e7ed66a00a4502fe70e7e0 <none>:<none>] 322021040} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9dd7e3e89af03eb7b6e388c1d0dbef37a75ab465ec23353f6bd9150cf2124e84 <none>:<none>] 307171281} {[quay.io/openshift-release-dev
/ocp-v4.0-art-dev@sha256:2cee6f2f1cf76e9b39c180a2771e91238a954038d8761b83ff66c797bfedfc5e <none>:<none>] 305342520} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:04c9c317a5547df040987ee4f0fdefe4b117f299899575bc90f4b3491a82cf02 <none>:<none>] 304033253} {[quay.io
/openshift-release-dev/ocp-v4.0-art-dev@sha256:66d4a2a6ec11525a8b1cbd8d3c0445cf06a14d84789df2eceacb9e67b9962e2e <none>:<none>] 299596966} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:96392554334baef6be2a63f374373b3650bafba8bbe2a5c61e4e22654c21d702 <none>:<none>
] 296997571} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:adb2c6b747142c2384a3f08903decd1626faa1416f84ef3bb2cd4eb197fc3116 <none>:<none>] 293949901} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:201c3c0da91165392995c11ed8392c08f04166dad7b04d6579b341c9
66bc3476 <none>:<none>] 285886451} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7b23b04d9265407b7d563b3ac13b2c20a2834b2c971b42e7627f939af31ae9cd <none>:<none>] 283675187} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ee217e5f4f985f1f1bc8c1dbf428deb25b
212b49556f1574fa62620687863c39 <none>:<none>] 280959414} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:88406eca7634b9b84af0c744c097f3ba2878c2dd181a32ba0285edbbdb3c1a83 <none>:<none>] 279777314} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d02641dabf58
927d8761562dbd5b181c35b23eb8950dd9e2959748db0cc5df17 <none>:<none>] 278500475} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:30e670f00cb90a1320abc2af1ffa32428a8ccabf8916e82e27fb61e5b8a754bb <none>:<none>] 277994559} {[quay.io/openshift-release-dev/ocp-v4.0-art-d
ev@sha256:e0d7dd33a3883ba3248e4e106fcead1ecd63a29eba682f93ccb605224e3e5ff7 <none>:<none>] 268686661} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6d548e915b46181087e5c8d4e84ac67b9d9acadea7b0bc5b55e2826348150b1f <none>:<none>] 264236964} {[quay.io/openshift-rele
ase-dev/ocp-v4.0-art-dev@sha256:1d4967eedf1759ececd718af5e69ca07b89a278d7fc1afc6ff1988e640437712 <none>:<none>] 258059345} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:97e255db208b9da546b1a3519246502acda1ca54cecb294010f36a8619a9453f <none>:<none>] 257007720} {[
quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9b89e94a0215eaeede16fb8fb8a99f8b0ecd43615839eb788d5bbebadb971623 <none>:<none>] 251426306} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0e0ace13114e94bfababb4517d0de6aadd268fc4b3679c8a811d0d396322656 <none>
:<none>] 246009528} {[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:aff6a7616278d03b23ee67dc0ae767d723b57b85aeb41d0697f9fc14e805dcd7 <none>:<none>] 239666982}],VolumesInUse:[],VolumesAttached:[],Config:nil,},}
machineconfiguration.openshift.io/state : Degraded
volumes.kubernetes.io/controller-managed-attach-detach : true



But after several seconds, it come back to work well.

===> node:> ip-10-0-152-181.us-east-2.compute.internal
machine.openshift.io/machine : openshift-machine-api/wjiang0725-bsbg6-worker-us-east-2b-lkx7s
machineconfiguration.openshift.io/currentConfig : rendered-worker-745ec85447a402109496addd8ed608ce
machineconfiguration.openshift.io/desiredConfig : rendered-worker-745ec85447a402109496addd8ed608ce
machineconfiguration.openshift.io/reason : 
machineconfiguration.openshift.io/ssh : accessed
machineconfiguration.openshift.io/state : Done
volumes.kubernetes.io/controller-managed-attach-detach : true

Comment 8 errata-xmlrpc 2019-10-16 06:31:35 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2922

Note You need to log in before you can comment on or make changes to this bug.