Hide Forgot
Description of problem: After the cluster upgrade to 4.2 from 4.1.6, the `operator-lifecycle-manager-packageserver` state in False status and no VERSION info, like below: mac:~ jianzhang$ oc get clusteroperator NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE ... operator-lifecycle-manager 4.2.0-0.nightly-2019-07-18-032637 True False False 161m operator-lifecycle-manager-catalog 4.2.0-0.nightly-2019-07-18-032637 True False False 161m operator-lifecycle-manager-packageserver False False False 38m Version-Release number of selected component (if applicable): sh-4.2$ olm --version OLM version: 0.10.1 git commit: 7367a8f How reproducible: always Steps to Reproduce: 1. Install OCP 4.1.6 2. Upgrade it to 4.2 Actual results: 1) Clusteroperator packageserver failed, and no version info report, lacks `versions` field. 2) Packageserver csv failed. mac:~ jianzhang$ oc get csv -n openshift-operator-lifecycle-manager NAME DISPLAY VERSION REPLACES PHASE packageserver Package Server 0.10.1 Failed packageserver.v0.9.0 Package Server 0.9.0 Succeeded mac:~ jianzhang$ oc get clusteroperator operator-lifecycle-manager-packageserver -o yaml apiVersion: config.openshift.io/v1 kind: ClusterOperator metadata: creationTimestamp: "2019-07-18T10:06:35Z" generation: 1 name: operator-lifecycle-manager-packageserver resourceVersion: "66892" selfLink: /apis/config.openshift.io/v1/clusteroperators/operator-lifecycle-manager-packageserver uid: b9eb8651-a943-11e9-9a61-066433738abc spec: {} status: conditions: - lastTransitionTime: "2019-07-18T10:06:35Z" status: "False" type: Degraded - lastTransitionTime: "2019-07-18T10:06:35Z" status: "False" type: Available - lastTransitionTime: "2019-07-18T10:08:47Z" message: Failed to deploy 0.10.1 status: "False" type: Progressing extension: null relatedObjects: - group: "" name: openshift-operator-lifecycle-manager resource: namespaces - group: operators.coreos.com name: packageserver namespace: openshift-operator-lifecycle-manager resource: ClusterServiceVersion Expected results: OLM packageserver upgrade success. Additional info: Packageserver still is 0.9.0 version, failed to upgrade to 0.10.1. Like below: mac:~ jianzhang$ oc get pods NAME READY STATUS RESTARTS AGE catalog-operator-7cdff8f4d-w8nwc 1/1 Running 0 49m olm-operator-69d54b799c-nln8f 1/1 Running 0 49m olm-operators-26wsf 1/1 Running 0 80m packageserver-8c65d6877-j9hh9 1/1 Running 0 75m packageserver-8c65d6877-r77gn 1/1 Running 0 75m mac:~ jianzhang$ oc rsh packageserver-8c65d6877-j9hh9 sh-4.2$ olm --version OLM version: 0.9.0 git commit: 4878aa4 Failed packageserver csv info: `reason: OwnerConflict` mac:~ jianzhang$ oc get csv -n openshift-operator-lifecycle-manager packageserver -o yaml apiVersion: operators.coreos.com/v1alpha1 kind: ClusterServiceVersion metadata: annotations: olm.operatorGroup: olm-operators olm.operatorNamespace: openshift-operator-lifecycle-manager olm.targetNamespaces: openshift-operator-lifecycle-manager creationTimestamp: "2019-07-18T10:08:47Z" generation: 12 labels: olm.api.4bca9f23e412d79d: provided olm.clusteroperator.name: operator-lifecycle-manager-packageserver olm.version: 0.10.1 name: packageserver namespace: openshift-operator-lifecycle-manager resourceVersion: "81263" selfLink: /apis/operators.coreos.com/v1alpha1/namespaces/openshift-operator-lifecycle-manager/clusterserviceversions/packageserver uid: 083a5387-a944-11e9-9a61-066433738abc spec: apiservicedefinitions: owned: - containerPort: 5443 deploymentName: packageserver description: A PackageManifest is a resource generated from existing CatalogSources and their ConfigMaps displayName: PackageManifest group: packages.operators.coreos.com kind: PackageManifest name: packagemanifests version: v1 customresourcedefinitions: {} description: Represents an Operator package that is available from a given CatalogSource which will resolve to a ClusterServiceVersion. displayName: Package Server install: spec: clusterPermissions: - rules: - apiGroups: - authorization.k8s.io resources: - subjectaccessreviews verbs: - create - get - apiGroups: - "" resources: - configmaps verbs: - get - list - watch - apiGroups: - operators.coreos.com resources: - catalogsources verbs: - get - list - watch - apiGroups: - packages.operators.coreos.com resources: - packagemanifests verbs: - get - list serviceAccountName: olm-operator-serviceaccount deployments: - name: packageserver spec: replicas: 2 selector: matchLabels: app: packageserver strategy: type: RollingUpdate template: metadata: labels: app: packageserver spec: containers: - command: - /bin/package-server - -v=4 - --secure-port - "5443" - --global-namespace - openshift-marketplace image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:da6b38c9fcc4db7401ced116ff763e6c3adef4f1aa4d44f3daefba60328a0d3e imagePullPolicy: IfNotPresent livenessProbe: httpGet: path: /healthz port: 5443 scheme: HTTPS name: packageserver ports: - containerPort: 5443 readinessProbe: httpGet: path: /healthz port: 5443 scheme: HTTPS terminationMessagePolicy: FallbackToLogsOnError nodeSelector: beta.kubernetes.io/os: linux node-role.kubernetes.io/master: "" priorityClassName: system-cluster-critical serviceAccountName: olm-operator-serviceaccount tolerations: - effect: NoSchedule key: node-role.kubernetes.io/master operator: Exists - effect: NoExecute key: node.kubernetes.io/unreachable operator: Exists tolerationSeconds: 120 - effect: NoExecute key: node.kubernetes.io/not-ready operator: Exists tolerationSeconds: 120 strategy: deployment installModes: - supported: true type: OwnNamespace - supported: true type: SingleNamespace - supported: true type: MultiNamespace - supported: true type: AllNamespaces keywords: - packagemanifests - olm - packages links: - name: Package Server url: https://github.com/operator-framework/operator-lifecycle-manager/tree/master/pkg/package-server maintainers: - email: openshift-operators@redhat.com name: Red Hat maturity: alpha minKubeVersion: 1.11.0 provider: name: Red Hat version: 0.10.1 status: certsLastUpdated: null certsRotateAt: null conditions: - lastTransitionTime: "2019-07-18T10:08:47Z" lastUpdateTime: "2019-07-18T10:08:47Z" message: requirements not yet checked phase: Pending reason: RequirementsUnknown - lastTransitionTime: "2019-07-18T10:08:47Z" lastUpdateTime: "2019-07-18T10:08:47Z" message: unable to adopt APIService phase: Failed reason: OwnerConflict lastTransitionTime: "2019-07-18T10:08:47Z" lastUpdateTime: "2019-07-18T10:08:47Z" message: unable to adopt APIService phase: Failed reason: OwnerConflict requirementStatus: - group: operators.coreos.com kind: ClusterServiceVersion message: CSV minKubeVersion (1.11.0) less than server version (v1.13.4+6569b4f) name: packageserver status: Present version: v1alpha1 - group: apiregistration.k8s.io kind: APIService message: "" name: v1.packages.operators.coreos.com status: DeploymentFound version: v1 - dependents: - group: rbac.authorization.k8s.io kind: PolicyRule message: cluster rule:{"verbs":["create","get"],"apiGroups":["authorization.k8s.io"],"resources":["subjectaccessreviews"]} status: Satisfied version: v1beta1 - group: rbac.authorization.k8s.io kind: PolicyRule message: cluster rule:{"verbs":["get","list","watch"],"apiGroups":[""],"resources":["configmaps"]} status: Satisfied version: v1beta1 - group: rbac.authorization.k8s.io kind: PolicyRule message: cluster rule:{"verbs":["get","list","watch"],"apiGroups":["operators.coreos.com"],"resources":["catalogsources"]} status: Satisfied version: v1beta1 - group: rbac.authorization.k8s.io kind: PolicyRule message: cluster rule:{"verbs":["get","list"],"apiGroups":["packages.operators.coreos.com"],"resources":["packagemanifests"]} status: Satisfied version: v1beta1 group: "" kind: ServiceAccount message: "" name: olm-operator-serviceaccount status: Present version: v1 mac:~ jianzhang$ oc get og -n openshift-operator-lifecycle-manager olm-operators -o yaml apiVersion: operators.coreos.com/v1 kind: OperatorGroup metadata: annotations: olm.providedAPIs: PackageManifest.v1.packages.operators.coreos.com creationTimestamp: "2019-07-18T08:02:31Z" generation: 55 name: olm-operators namespace: openshift-operator-lifecycle-manager resourceVersion: "83910" selfLink: /apis/operators.coreos.com/v1/namespaces/openshift-operator-lifecycle-manager/operatorgroups/olm-operators uid: 64e71542-a932-11e9-9b2c-02ff9ebba722 spec: targetNamespaces: - openshift-operator-lifecycle-manager status: lastUpdated: "2019-07-18T08:04:06Z" namespaces: - openshift-operator-lifecycle-manager mac:~ jianzhang$ oc get clusterrole packageserver.v0.9.0-tcln4 -o yaml apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: creationTimestamp: "2019-07-18T08:04:29Z" labels: olm.owner: packageserver.v0.9.0 olm.owner.kind: ClusterServiceVersion olm.owner.namespace: openshift-operator-lifecycle-manager name: packageserver.v0.9.0-tcln4 resourceVersion: "4445" selfLink: /apis/rbac.authorization.k8s.io/v1/clusterroles/packageserver.v0.9.0-tcln4 uid: ab2d5065-a932-11e9-92d4-02ef6ebf96d0 rules: - apiGroups: - authorization.k8s.io resources: - subjectaccessreviews verbs: - create - get - apiGroups: - "" resources: - configmaps verbs: - get - list - watch - apiGroups: - operators.coreos.com resources: - catalogsources verbs: - get - list - watch - apiGroups: - packages.operators.coreos.com resources: - packagemanifests verbs: - get - list
*** This bug has been marked as a duplicate of bug 1723818 ***