Bug 1964270 - Failed to install 'cluster-kube-descheduler-operator' with error: "clusterkubedescheduleroperator.4.8.0-202105211057.p0.assembly.stream\": must be no more than 63 characters"
Summary: Failed to install 'cluster-kube-descheduler-operator' with error: "clusterkub...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: kube-scheduler
Version: 4.8
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 4.8.0
Assignee: Mike Dame
QA Contact: RamaKasturi
URL:
Whiteboard:
Depends On:
Blocks: 1966603
TreeView+ depends on / blocked
 
Reported: 2021-05-25 07:12 UTC by zhou ying
Modified: 2021-08-24 13:40 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-07-27 23:10:05 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2021:2438 0 None None None 2021-07-27 23:10:27 UTC

Description zhou ying 2021-05-25 07:12:43 UTC
Description of problem:
Failed to install 'cluster-kube-descheduler-operator' with error:
"message": "error creating clusterrole clusterkubedescheduleroperator.4.8.0-202105211057.p0-6d99c5598c: ClusterRole.rbac.authorization.k8s.io \"clusterkubedescheduleroperator.4.8.0-202105211057.p0-6d99c5598c\" is invalid: metadata.labels: Invalid value: \"clusterkubedescheduleroperator.4.8.0-202105211057.p0.assembly.stream\": must be no more than 63 characters"


Version-Release number of selected component (if applicable):
[root@localhost ~]# oc get csv clusterkubedescheduleroperator.4.8.0-202105211057.p0.assembly.stream
NAME                                                                   DISPLAY                     VERSION                                 REPLACES   PHASE
clusterkubedescheduleroperator.4.8.0-202105211057.p0.assembly.stream   Kube Descheduler Operator   4.8.0-202105211057.p0.assembly.stream              Pending

How reproducible:
Always

Steps to Reproduce:
1) Create namespace :openshift-kube-descheduler-operator 
2) Create  "cluster-kube-descheduler-operator" from console;

Actual results:
2) Failed to create the  "cluster-kube-descheduler-operator" find error: 
[root@localhost ~]# oc get events 
LAST SEEN   TYPE     REASON                OBJECT                                                                                       MESSAGE
48m         Normal   RequirementsUnknown   clusterserviceversion/clusterkubedescheduleroperator.4.8.0-202105211057.p0.assembly.stream   requirements not yet checked
48m         Normal   RequirementsNotMet    clusterserviceversion/clusterkubedescheduleroperator.4.8.0-202105211057.p0.assembly.stream   one or more requirements couldn't be found
49m         Normal   CreatedSCCRanges      namespace/openshift-kube-descheduler-operator

[root@localhost ~]# oc get ip install-wdpl7 -o json |jq '.status'
{
  "bundleLookups": [
    {
      "catalogSourceRef": {
        "name": "qe-app-registry",
        "namespace": "openshift-marketplace"
      },
      "identifier": "clusterkubedescheduleroperator.4.8.0-202105211057.p0.assembly.stream",
      "path": "registry-proxy.engineering.redhat.com/rh-osbs/openshift-ose-ose-cluster-kube-descheduler-operator-bundle@sha256:1ba792a91f426344612680ac38b9db4786e4377330f08f34b70a6e11b6685616",
      "properties": "{\"properties\":[{\"type\":\"olm.gvk\",\"value\":{\"group\":\"operator.openshift.io\",\"kind\":\"KubeDescheduler\",\"version\":\"v1\"}},{\"type\":\"olm.gvk\",\"value\":{\"group\":\"operator.openshift.io\",\"kind\":\"KubeDescheduler\",\"version\":\"v1beta1\"}},{\"type\":\"olm.package\",\"value\":{\"packageName\":\"cluster-kube-descheduler-operator\",\"version\":\"4.8.0-202105211057.p0.assembly.stream\"}}]}",
      "replaces": ""
    }
  ],
  "catalogSources": [],
  "conditions": [
    {
      "lastTransitionTime": "2021-05-25T05:28:59Z",
      "lastUpdateTime": "2021-05-25T05:28:59Z",
      "message": "error creating clusterrole clusterkubedescheduleroperator.4.8.0-202105211057.p0-6d99c5598c: ClusterRole.rbac.authorization.k8s.io \"clusterkubedescheduleroperator.4.8.0-202105211057.p0-6d99c5598c\" is invalid: metadata.labels: Invalid value: \"clusterkubedescheduleroperator.4.8.0-202105211057.p0.assembly.stream\": must be no more than 63 characters",
      "reason": "InstallComponentFailed",
      "status": "False",
      "type": "Installed"
    }
  ],
....

Expected results:
2) Succceed create the "cluster-kube-descheduler-operator" from console.


Additional info:

Comment 1 zhou ying 2021-05-25 07:27:53 UTC
[root@localhost ~]# oc  describe sub cluster-kube-descheduler-operator
Name:         cluster-kube-descheduler-operator
Namespace:    openshift-kube-descheduler-operator
Labels:       operators.coreos.com/cluster-kube-descheduler-operator.openshift-kube-descheduler-op=
Annotations:  <none>
API Version:  operators.coreos.com/v1alpha1
Kind:         Subscription
Metadata:
  Creation Timestamp:  2021-05-25T05:27:47Z
  Generation:          1
  Managed Fields:
    API Version:  operators.coreos.com/v1alpha1
    Fields Type:  FieldsV1
    fieldsV1:
      f:spec:
        .:
        f:channel:
        f:installPlanApproval:
        f:name:
        f:source:
        f:sourceNamespace:
        f:startingCSV:
    Manager:      Mozilla
    Operation:    Update
    Time:         2021-05-25T05:27:47Z
    API Version:  operators.coreos.com/v1alpha1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:labels:
          .:
          f:operators.coreos.com/cluster-kube-descheduler-operator.openshift-kube-descheduler-op:
    Manager:      olm
    Operation:    Update
    Time:         2021-05-25T05:27:47Z
    API Version:  operators.coreos.com/v1alpha1
    Fields Type:  FieldsV1
    fieldsV1:
      f:status:
        .:
        f:catalogHealth:
        f:conditions:
        f:currentCSV:
        f:installPlanGeneration:
        f:installPlanRef:
          .:
          f:apiVersion:
          f:kind:
          f:name:
          f:namespace:
          f:resourceVersion:
          f:uid:
        f:installedCSV:
        f:installplan:
          .:
          f:apiVersion:
          f:kind:
          f:name:
          f:uuid:
        f:lastUpdated:
        f:state:
    Manager:         catalog
    Operation:       Update
    Time:            2021-05-25T05:27:59Z
  Resource Version:  88492
  UID:               dca0e80f-9501-4595-bd4e-72f834a57c73
Spec:
  Channel:                4.8
  Install Plan Approval:  Automatic
  Name:                   cluster-kube-descheduler-operator
  Source:                 qe-app-registry
  Source Namespace:       openshift-marketplace
  Starting CSV:           clusterkubedescheduleroperator.4.8.0-202105211057.p0.assembly.stream
Status:
  Catalog Health:
    Catalog Source Ref:
      API Version:       operators.coreos.com/v1alpha1
      Kind:              CatalogSource
      Name:              community-operators
      Namespace:         openshift-marketplace
      Resource Version:  84144
      UID:               2c05155e-670f-48b0-880e-e54a9606a40c
    Healthy:             true
    Last Updated:        2021-05-25T05:27:47Z
    Catalog Source Ref:
      API Version:       operators.coreos.com/v1alpha1
      Kind:              CatalogSource
      Name:              qe-app-registry
      Namespace:         openshift-marketplace
      Resource Version:  61412
      UID:               bf10f54a-90ad-4efb-b1e8-c1936647cc77
    Healthy:             true
    Last Updated:        2021-05-25T05:27:47Z
  Conditions:
    Last Transition Time:   2021-05-25T05:27:47Z
    Message:                all available catalogsources are healthy
    Reason:                 AllCatalogSourcesHealthy
    Status:                 False
    Type:                   CatalogSourcesUnhealthy
    Last Transition Time:   2021-05-25T05:29:00Z
    Reason:                 InstallComponentFailed
    Status:                 True
    Type:                   InstallPlanFailed
  Current CSV:              clusterkubedescheduleroperator.4.8.0-202105211057.p0.assembly.stream
  Install Plan Generation:  1
  Install Plan Ref:
    API Version:       operators.coreos.com/v1alpha1
    Kind:              InstallPlan
    Name:              install-wdpl7
    Namespace:         openshift-kube-descheduler-operator
    Resource Version:  87867
    UID:               1a55e2c3-ec19-4dbc-a73b-8655895f9c06
  Installed CSV:       clusterkubedescheduleroperator.4.8.0-202105211057.p0.assembly.stream
  Installplan:
    API Version:  operators.coreos.com/v1alpha1
    Kind:         InstallPlan
    Name:         install-wdpl7
    Uuid:         1a55e2c3-ec19-4dbc-a73b-8655895f9c06
  Last Updated:   2021-05-25T05:29:00Z
  State:          AtLatestKnown
Events:           <none>

Comment 2 RamaKasturi 2021-05-25 15:56:24 UTC
Thanks Yinzhou for reporting the bug, i hit similar issue during upgrade as well.

[knarra@knarra ~]$ oc get csv
NAME                                                                   DISPLAY                            VERSION                                 REPLACES                                               PHASE
clusterkubedescheduleroperator.4.7.0-202105141603.p0                   Kube Descheduler Operator          4.7.0-202105141603.p0                                                                          Replacing
clusterkubedescheduleroperator.4.8.0-202105211057.p0.assembly.stream   Kube Descheduler Operator          4.8.0-202105211057.p0.assembly.stream   clusterkubedescheduleroperator.4.7.0-202105141603.p0   Pending

Events:
  Type    Reason               Age                    From                        Message
  ----    ------               ----                   ----                        -------
  Normal  RequirementsUnknown  2m58s (x2 over 2m58s)  operator-lifecycle-manager  requirements not yet checked
  Normal  RequirementsNotMet   2m57s (x2 over 2m58s)  operator-lifecycle-manager  one or more requirements couldn't be found

Comment 3 RamaKasturi 2021-05-26 08:55:23 UTC
setting the blocker ? flag as due to this bug QE is unable to install / upgrade or test descheduler operator. Install / upgrade gives the error above.

Comment 4 Kevin Rizza 2021-05-26 13:19:47 UTC
Hi folks,

So this particular limit is because this operator is hitting a kube limit: OLM uses labels to track related resources from an operator install, and the name of this operator's CSV is too long to actually fit in that label. In the short term, the fix for this has to be that the operator reduces the length of the CSV name, otherwise this will never install on a cluster with OLM. Even if we add support for longer CSV names, it's still going to run into this issue for any cluster that doesn't have a change like that backported. Who owns the descheduler operator? It seems like the immediate fix should be for them to reduce the length of the CSV name and republish this operator bundle.

Longer term, we can potentially try to spec out some changes that could enable longer CSV names, but that is bordering on a feature request and will most likely not be something that we would backport to previous versions of OCP. If anything, the only immediate fix we could add here is validation to test that operators are hitting this limit before they are released.

For now, I'm setting this to blocker- and updating the priority and severity to medium/medium. I'm also reaching out on slack to continue this conversation there.

Comment 6 Maciej Szulik 2021-06-08 14:27:37 UTC
https://issues.redhat.com/browse/ART-3002 looks completed so I'm moving this to modified.

Comment 8 RamaKasturi 2021-06-09 11:32:37 UTC
Verified bug with the payload below and could install descheduler successfully.

[knarra@knarra ~]$ oc get csv
NAME                                                DISPLAY                     VERSION              REPLACES   PHASE
clusterkubedescheduleroperator.4.8.0-202106081938   Kube Descheduler Operator   4.8.0-202106081938              Succeeded

[knarra@knarra ~]$ oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.8.0-0.nightly-2021-06-09-023740   True        False         3h57m   Cluster version is 4.8.0-0.nightly-2021-06-09-023740

[knarra@knarra ~]$ oc get pods 
NAME                                    READY   STATUS    RESTARTS   AGE
cluster-8478444647-lfhll                1/1     Running   0          3m26s
descheduler-operator-5db59c8558-s9k8r   1/1     Running   0          56m

Based on the above moving bug to verified state.

Comment 11 Mike Dame 2021-06-16 12:52:02 UTC
The issue was determined to not be specific to the Descheduler, but rather ART's automated builds of operators, see https://issues.redhat.com/browse/ART-3002

From those comments it appears they re-triggered builds for 4.5-4.8, but for any more questions on it I would forward to the ART team. I do not know when the release schedule for prior releases is, unfortunately, sorry

Comment 13 errata-xmlrpc 2021-07-27 23:10:05 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.8.2 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:2438


Note You need to log in before you can comment on or make changes to this bug.