Bug 1807471 - nodeca daemon isn't installed if registry is Removed
Summary: nodeca daemon isn't installed if registry is Removed
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Image
Version: 4.1.z
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 4.5.0
Assignee: Oleg Bulatov
QA Contact: XiuJuan Wang
URL:
Whiteboard:
: 1817975 (view as bug list)
Depends On:
Blocks: 1809176
TreeView+ depends on / blocked
 
Reported: 2020-02-26 12:51 UTC by Oleg Bulatov
Modified: 2020-07-29 17:29 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: nodeca daemon was created only when the registry is managed Consequence: when the registry is removed, nodeca daemon is not created Fix: always create nodeca daemon Result: nodeca daemon is created even if the registry is removed
Clone Of:
: 1809176 (view as bug list)
Environment:
Last Closed: 2020-07-13 17:21:31 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Github openshift cluster-image-registry-operator pull 507 None closed Bug 1807471: add always-on controller for image-registry-certificates 2020-09-24 13:18:41 UTC
Red Hat Knowledge Base (Solution) 4882291 None None None 2020-04-10 10:55:27 UTC
Red Hat Product Errata RHBA-2020:2409 None None None 2020-07-13 17:22:05 UTC

Description Oleg Bulatov 2020-02-26 12:51:22 UTC
If managementState for the image-registry is Removed, the nodeca daemon doesn't get installed.

As a result, additionalTrustedCA is not applied to the cluster.

Comment 2 Adam Kaplan 2020-02-26 15:56:24 UTC
Work-around - set the registry to `Managed` with `emptyDir` storage to enable the nodeca daemon.

Comment 3 Eduardo Minguez 2020-02-26 15:59:44 UTC
(In reply to Adam Kaplan from comment #2)
> Work-around - set the registry to `Managed` with `emptyDir` storage to
> enable the nodeca daemon.

oc patch configs.imageregistry.operator.openshift.io cluster --type merge --patch '{"spec":{"storage":{"emptyDir":{}}}}'
oc patch configs.imageregistry.operator.openshift.io cluster --type merge --patch '{"spec":{"managementState":"Managed"}}'

That worked for me, thanks.

Comment 10 XiuJuan Wang 2020-03-13 10:42:13 UTC
When set image registry to Removed, the node-ca pod can't be running due to "image-registry-certificates" not found
$ oc get co  image-registry 
NAME             VERSION                             AVAILABLE   PROGRESSING   DEGRADED   SINCE
image-registry   4.5.0-0.nightly-2020-03-12-233449   True        False         False      53m

$ oc get config.image -o yaml 
apiVersion: v1
items:
- apiVersion: imageregistry.operator.openshift.io/v1
  kind: Config
  metadata:
    creationTimestamp: "2020-03-13T09:46:49Z"
    finalizers:
    - imageregistry.operator.openshift.io/finalizer
    generation: 2
    name: cluster
    resourceVersion: "43408"
    selfLink: /apis/imageregistry.operator.openshift.io/v1/configs/cluster
    uid: 2f128c22-5e50-4b92-b438-67368250f2e9
  spec:
    defaultRoute: true
    httpSecret: fc4f238f53cb562dd89a19290f7cbc61f66e4f9c77a6d45e796a297ff8de33cd34f6c8b0f5364209a24e829d3e7c362a0b997a71e4bce6d09ea0f3355550ee17
    logging: 2
    managementState: Removed
    proxy: {}
    replicas: 1
    requests:
      read:
        maxWaitInQueue: 0s
      write:
        maxWaitInQueue: 0s
    rolloutStrategy: RollingUpdate
    storage: {}
  status:
    conditions:
    - lastTransitionTime: "2020-03-13T10:38:59Z"
      message: All registry resources are removed
      reason: Removed
      status: "False"
      type: Progressing
    - lastTransitionTime: "2020-03-13T09:46:50Z"
      message: The registry is removed
      reason: Removed
      status: "True"
      type: Available
    - lastTransitionTime: "2020-03-13T09:46:50Z"
      status: "False"
      type: Degraded
    - lastTransitionTime: "2020-03-13T09:46:50Z"
      message: The registry is removed
      reason: Removed
      status: "True"
      type: Removed
    observedGeneration: 2
    readyReplicas: 0
    storage: {}
    storageManaged: false
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""



$oc get pods
NAME                                               READY   STATUS              RESTARTS   AGE
cluster-image-registry-operator-6cc6df7544-5t7hj   2/2     Running             0          50m
node-ca-8jgmg                                      0/1     ContainerCreating   0          50m
node-ca-ckk6b                                      0/1     ContainerCreating   0          46m
node-ca-cv8xg                                      0/1     ContainerCreating   0          50m
node-ca-fdzw6                                      0/1     ContainerCreating   0          50m
node-ca-nqvtb                                      0/1     ContainerCreating   0          45m

$ oc describe pods node-ca-8jgmg 
=============================
Node-Selectors:  kubernetes.io/os=linux
Tolerations:     
Events:
  Type     Reason       Age                   From                                    Message
  ----     ------       ----                  ----                                    -------
  Normal   Scheduled    52m                   default-scheduler                       Successfully assigned openshift-image-registry/node-ca-8jgmg to wxj-bm3-b49lw-control-plane-2
  Warning  FailedMount  33m (x2 over 39m)     kubelet, wxj-bm3-b49lw-control-plane-2  Unable to attach or mount volumes: unmounted volumes=[serviceca], unattached volumes=[host node-ca-token-s5bqd serviceca]: timed out waiting for the condition
  Warning  FailedMount  31m (x2 over 45m)     kubelet, wxj-bm3-b49lw-control-plane-2  Unable to attach or mount volumes: unmounted volumes=[serviceca], unattached volumes=[node-ca-token-s5bqd serviceca host]: timed out waiting for the condition
  Warning  FailedMount  6m47s (x14 over 50m)  kubelet, wxj-bm3-b49lw-control-plane-2  Unable to attach or mount volumes: unmounted volumes=[serviceca], unattached volumes=[serviceca host node-ca-token-s5bqd]: timed out waiting for the condition
  Warning  FailedMount  102s (x33 over 52m)   kubelet, wxj-bm3-b49lw-control-plane-2  MountVolume.SetUp failed for volume "serviceca" : configmap "image-registry-certificates" not found

$oc get cm  -n openshift-image-registry 
NAME         DATA   AGE
trusted-ca   1      82m

Comment 11 Oleg Bulatov 2020-03-27 14:21:33 UTC
*** Bug 1817975 has been marked as a duplicate of this bug. ***

Comment 13 XiuJuan Wang 2020-04-10 08:38:39 UTC
Test in 4.5.0-0.nightly-2020-04-09-231931 cluster, and pass.
When set image registry to Removed, the node-ca daemon is installed, and pods could be running.

$oc get pods
NAME                                               READY   STATUS    RESTARTS   AGE
cluster-image-registry-operator-845886ff5f-8vbtr   2/2     Running   0          139m
node-ca-5n8xf                                      1/1     Running   0          139m
node-ca-9dgrh                                      1/1     Running   0          139m
node-ca-jx7x8                                      1/1     Running   0          139m
node-ca-qfxrz                                      1/1     Running   1          138m

$ oc get config.image -o yaml  
apiVersion: v1
items:
- apiVersion: imageregistry.operator.openshift.io/v1
  kind: Config
  metadata:
    creationTimestamp: "2020-04-10T05:53:59Z"
    finalizers:
    - imageregistry.operator.openshift.io/finalizer
    generation: 1
    name: cluster
    resourceVersion: "74364"
    selfLink: /apis/imageregistry.operator.openshift.io/v1/configs/cluster
    uid: d0066a10-ceba-4d9f-9435-0c9543331049
  spec:
    httpSecret: 1cf7d118baceec502f091760b99398b2c081b050f7f8158ed9d290dea80bee09da8a910246774a7fcc2105d67492c3d774fcc025e1ae12286479325f313feb86
    logging: 2
    managementState: Removed
    proxy: {}
    replicas: 1
    requests:
      read:
        maxWaitInQueue: 0s
      write:
        maxWaitInQueue: 0s
    rolloutStrategy: RollingUpdate
    storage: {}
  status:
    conditions:
    - lastTransitionTime: "2020-04-10T08:30:02Z"
      message: All registry resources are removed
      reason: Removed
      status: "False"
      type: Progressing
    - lastTransitionTime: "2020-04-10T05:53:59Z"
      message: The registry is removed
      reason: Removed
      status: "True"
      type: Available
    - lastTransitionTime: "2020-04-10T05:53:59Z"
      message: The registry is removed
      reason: Removed
      status: "False"
      type: Degraded
    - lastTransitionTime: "2020-04-10T05:53:59Z"
      message: The registry is removed
      reason: Removed
      status: "True"
      type: Removed
    observedGeneration: 1
    readyReplicas: 0
    storage: {}
    storageManaged: false
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

Comment 17 errata-xmlrpc 2020-07-13 17:21:31 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:2409


Note You need to log in before you can comment on or make changes to this bug.