Bug 1797790

Summary: MachineConfigDaemonReasonAnnotationKey gives node description beyond 262144 characters
Product: OpenShift Container Platform Reporter: Erica von Buelow <evb>
Component: Machine Config OperatorAssignee: Erica von Buelow <evb>
Status: CLOSED ERRATA QA Contact: Michael Nguyen <mnguyen>
Severity: high Docs Contact:
Priority: high    
Version: 4.2.zCC: amurdaca, evb, igreen, mnguyen, scuppett, travi
Target Milestone: ---   
Target Release: 4.2.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1792914 Environment:
Last Closed: 2020-03-26 07:21:27 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1809693    
Bug Blocks:    

Comment 4 Michael Nguyen 2020-03-16 15:52:16 UTC
verified on 4.2.0-0.nightly-2020-03-16-055952


apiVersion: v1
kind: Node
metadata:
  annotations:
    machine.openshift.io/machine: openshift-machine-api/ci-ln-q7xk1g2-d5d6b-dtm64-worker-us-west-1a-6drrq
    machineconfiguration.openshift.io/currentConfig: rendered-worker-4605cb9bf152301e9b58405831c365f0
    machineconfiguration.openshift.io/reason: machineconfiguration.openshift.io/desiredConfig
      annotation not found on node 'ip-10-0-133-186.us-west-1.compute.internal'
    machineconfiguration.openshift.io/state: Degraded
    volumes.kubernetes.io/controller-managed-attach-detach: "true"
  creationTimestamp: "2020-03-16T15:09:40Z"
  labels:
    beta.kubernetes.io/arch: amd64
    beta.kubernetes.io/instance-type: m4.xlarge
    beta.kubernetes.io/os: linux

Comment 5 Antonio Murdaca 2020-03-16 16:06:24 UTC
(In reply to Michael Nguyen from comment #4)
> verified on 4.2.0-0.nightly-2020-03-16-055952
> 
> 
> apiVersion: v1
> kind: Node
> metadata:
>   annotations:
>     machine.openshift.io/machine:
> openshift-machine-api/ci-ln-q7xk1g2-d5d6b-dtm64-worker-us-west-1a-6drrq
>     machineconfiguration.openshift.io/currentConfig:
> rendered-worker-4605cb9bf152301e9b58405831c365f0
>     machineconfiguration.openshift.io/reason:
> machineconfiguration.openshift.io/desiredConfig
>       annotation not found on node
> 'ip-10-0-133-186.us-west-1.compute.internal'


wait, wait, wait, why there's no desiredConfig annotation here? where did it go?

>     machineconfiguration.openshift.io/state: Degraded
>     volumes.kubernetes.io/controller-managed-attach-detach: "true"
>   creationTimestamp: "2020-03-16T15:09:40Z"
>   labels:
>     beta.kubernetes.io/arch: amd64
>     beta.kubernetes.io/instance-type: m4.xlarge
>     beta.kubernetes.io/os: linux

Comment 6 Erica von Buelow 2020-03-16 18:49:31 UTC
(In reply to Antonio Murdaca from comment #5)
It was explicitly deleted to trigger the bug. I don't think it's possible to get into this situation without user error (right now), but the problem is that it prevents the controller from recovering and can block other important functions by blocking all annotation updates.

Comment 8 errata-xmlrpc 2020-03-26 07:21:27 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0826