Bug 2012228 - ibmcloud: credentialsrequests invalid for machine-api-operator: resource-group
Summary: ibmcloud: credentialsrequests invalid for machine-api-operator: resource-group
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Cloud Compute
Version: 4.10
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: 4.10.0
Assignee: Joel Speed
QA Contact: Pedro Amoedo
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-10-08 15:29 UTC by Christopher J Schaefer
Modified: 2022-03-10 16:18 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-03-10 16:18:05 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift machine-api-operator pull 928 0 None open Bug 2012228: fix(ibmcloud): Set resource group policy in CredentialsRequest 2021-10-08 21:02:58 UTC
Red Hat Product Errata RHSA-2022:0056 0 None None None 2022-03-10 16:18:31 UTC

Description Christopher J Schaefer 2021-10-08 15:29:40 UTC
Description of problem:
The current CredentialsRequests configuration is invalid for IBM Cloud required permissions. Since IBM Cloud uses Manual Credentials mode, the machine-api-operator credentials/permissions generation is broken and causes IPI deployments to fail when the operator cannot progress.

Version-Release number of selected component (if applicable):
4.10


How reproducible:
Occurs during IPI deployments, requiring the generation of Service Id's with IAM permissions for IBM Cloud, from CredentialsRequests.

Using an IPI deployment and ccoctl, generate the Service Id with IAM permissions for IBM Cloud machine-api-operator with the current CredentialsRequests. The Service Id that is created does not have valid permissions required for the MAPI oeprator.

Steps to Reproduce:
1. During IPI deployment, generate the Service Id's for operators via 'ccoctl ibmcloud create-service-id', using the available CredentialsRequests, specifically the machine-api-operator
2. Continue IPI deployment, with the generated Secrets for Service Id's (specifically for machine-api-operator), and MAPI operator will fail to progress due to invalid IAM permissions

Actual results:
IPI deployment fails when machine-api-operator cannot successfully progress, due to invalid IAM permissions assigned during the 'ccoctl ibmcloud create-service-id' generation.

Expected results:
Fully successful IPI deployment using 'ccoctl ibmcloud create-service-id' for setting up correct Service Id's and IAM permissions for operators

Additional info:
IBM Cloud is working on a fix to assign the correct IAM permissions to address this isssue.

Comment 4 Pedro Amoedo 2021-10-18 12:59:48 UTC
[QA Summary]

[Version]

~~~
$ oc version
Client Version: 4.10.0-0.nightly-2021-10-16-173656

$ ./openshift-install version
./openshift-install 4.10.0-0.nightly-2021-10-16-173656
built from commit 95361b7f82a6539d78c170c6677de3fac776bb8d
release image registry.ci.openshift.org/ocp/release@sha256:ad3e0e971d2df07c7013925f59a9113603f7fea1eef2fc18dec2d7e740bbeb1f
release architecture amd64
~~~

[Environment]

~~~
$ CCO_IMAGE=$(oc adm release info -a pull-secret --image-for='cloud-credential-operator' registry.ci.openshift.org/ocp/release:4.10.0-0.nightly-2021-10-16-173656)
$ oc adm release extract -a pull-secret --credentials-requests --cloud=ibmcloud registry.ci.openshift.org/ocp/release:4.10.0-0.nightly-2021-10-16-173656 --to test42/cco-creds/

$ cat test42/cco-creds/0000_30_machine-api-operator_00_credentials-request.yaml 
---
apiVersion: cloudcredential.openshift.io/v1
kind: CredentialsRequest
metadata:
  annotations:
    include.release.openshift.io/self-managed-high-availability: "true"
  labels:
    controller-tools.k8s.io: "1.0"
  name: openshift-machine-api-ibmcloud
  namespace: openshift-cloud-credential-operator
spec:
  providerSpec:
    apiVersion: cloudcredential.openshift.io/v1
    kind: IBMCloudProviderSpec
    policies:
    - attributes:
      - name: serviceName
        value: is
      roles:
      - crn:v1:bluemix:public:iam::::role:Operator
      - crn:v1:bluemix:public:iam::::role:Editor
      - crn:v1:bluemix:public:iam::::role:Viewer
    - attributes:
      - name: resourceType
        value: resource-group
      roles:
      - crn:v1:bluemix:public:iam::::role:Viewer
  secretRef:
    name: ibmcloud-credentials
    namespace: openshift-machine-api

$ oc image extract $CCO_IMAGE --file="/usr/bin/ccoctl" -a pull-secret
$ chmod +x ccoctl
$ export IC_API_KEY='xxx'
~~~

[Results]

~~~
$ ./ccoctl ibmcloud create-service-id --name="pamoedo-test" --credentials-requests-dir="test42/cco-creds" --output-dir="test42/cco-mnfst"
2021/10/18 14:50:52 Created IAM Access Policy:
...
2021/10/18 14:50:57 Saved credentials configuration to: test42/cco-mnfst/manifests/openshift-cloud-controller-manager-ibm-cloud-credentials-credentials.yaml
2021/10/18 14:50:57 Saved credentials configuration to: test42/cco-mnfst/manifests/openshift-machine-api-ibmcloud-credentials-credentials.yaml
2021/10/18 14:50:57 Saved credentials configuration to: test42/cco-mnfst/manifests/openshift-image-registry-installer-cloud-credentials-credentials.yaml
2021/10/18 14:50:57 Saved credentials configuration to: test42/cco-mnfst/manifests/openshift-ingress-operator-cloud-credentials-credentials.yaml

$ cat test42/cco-mnfst/manifests/openshift-machine-api-ibmcloud-credentials-credentials.yaml 
apiVersion: v1
kind: Secret
metadata:
  creationTimestamp: null
  name: ibmcloud-credentials
  namespace: openshift-machine-api
stringData:
  ibm-credentials.env: |-
    IBMCLOUD_AUTHTYPE=iam
    IBMCLOUD_APIKEY=xxx
  ibmcloud_api_key: xxx
type: Opaque

$ ibmcloud iam service-ids
Getting all services IDs bound to current account as pamoedom...
OK
ID                                               Name                                                                    Created At              Last Updated            Description   Locked   
ServiceId-615d1473-611a-455c-b453-bd5052871fdc   pamoedo-test-openshift-cloud-controller-manager-ibm-cloud-credentials   2021-10-18T12:50+0000   2021-10-18T12:50+0000                 false   
ServiceId-2bc09210-c9cb-489c-8ab6-edd1651ea2f2   pamoedo-test-openshift-image-registry-installer-cloud-credentials       2021-10-18T12:50+0000   2021-10-18T12:50+0000                 false   
ServiceId-59332d85-21d9-4be4-bc9f-a0079e20d146   pamoedo-test-openshift-ingress-operator-cloud-credentials               2021-10-18T12:50+0000   2021-10-18T12:50+0000                 false   
ServiceId-af774c81-dc06-48f5-b474-c9e745c06f4a   pamoedo-test-openshift-machine-api-ibmcloud-credentials                 2021-10-18T12:50+0000   2021-10-18T12:50+0000                 false
~~~

*** PASSED ***

Comment 7 errata-xmlrpc 2022-03-10 16:18:05 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.10.3 security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:0056


Note You need to log in before you can comment on or make changes to this bug.