Bug 2084280 - GCP API Checks Fail if non-required APIs are not enabled
Summary: GCP API Checks Fail if non-required APIs are not enabled
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 4.10
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 4.11.0
Assignee: Aditya Narayanaswamy
QA Contact: Jianli Wei
URL:
Whiteboard:
Depends On:
Blocks: 2099604
TreeView+ depends on / blocked
 
Reported: 2022-05-11 19:04 UTC by Patrick Dillon
Modified: 2022-08-10 11:11 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: The installer had a list of required APIs that it checks for quota and permissions but the list contained some APIs that are not strictly necessary Consequence: Installation failed if the user did not provide the permissions for the optional APIs. Fix: Split the list into required and optional APIs where if the optional APIs are not accessible, only a warning message is displayed. Result: Just a warning message instead of stopping the installation Required APIs: compute.googleapis.com cloudresourcemanager.googleapis.com dns.googleapis.com iam.googleapis.com iamcredentials.googleapis.com Optional APIs: cloudapis.googleapis.com servicemanagement.googleapis.com deploymentmanager.googleapis.com storage-api.googleapis.com storage-component.googleapis.com serviceusage.googleapis.com
Clone Of:
: 2099604 (view as bug list)
Environment:
Last Closed: 2022-08-10 11:11:18 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift installer pull 5915 0 None open Bug 2084280: Remove optional services from permissions check 2022-05-16 14:19:25 UTC
Red Hat Product Errata RHSA-2022:5069 0 None None None 2022-08-10 11:11:45 UTC

Internal Links: 2100281

Description Patrick Dillon 2022-05-11 19:04:42 UTC
Customer has security rules about the API's in Google and they don't want to open all the APIs. Openshift installer makes a check during the installation process and if any of those APIs is not started in the GCP project installation fails

- cloudapis.googleapis.com
- servicemanagement.googleapis.com
- serviceusage.googleapis.com

And the first one is the most problematic one,  because "cloudapis.googleapis.com"option represents the API collection of all the cloud services/APIs offered by Google Cloud Platform, so we are opening all the API.

Also filed as an RFE open https://issues.redhat.com/browse/RFE-2798

Comment 5 Jianli Wei 2022-05-23 08:42:45 UTC
Tested with the build having the PR https://github.com/openshift/installer/pull/5915 (see https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/release-openshift-origin-installer-launch-gcp-modern/1528591473132441600), installation could succeed when some optional APIs are not enabled, but one comment on the PR, thanks! 


>0. Set up the service-account in project "openshift-qe-shared-vpc".

$ gcloud config get account
jiwei-0523-sa.gserviceaccount.com
$ gcloud config get project
openshift-qe-shared-vpc
$ gcloud projects get-iam-policy openshift-qe-shared-vpc --flatten="bindings[].members" --format="table(bindings.role)" --filter="bindings.members:jiwei-0523-sa.gserviceaccount.com"
ROLE
roles/compute.admin
roles/compute.instanceAdmin
roles/compute.loadBalancerAdmin
roles/compute.networkAdmin
roles/compute.securityAdmin
roles/deploymentmanager.editor
roles/dns.admin
roles/iam.securityAdmin
roles/iam.serviceAccountAdmin
roles/iam.serviceAccountKeyAdmin
roles/iam.serviceAccountUser
roles/iam.workloadIdentityPoolAdmin
roles/networkmanagement.admin
roles/storage.admin
$ 


>1. When "cloudresourcemanager.googleapis.com" API is disabled, "create install-config" (or "create manifests" / "create cluster") would fail, as expected.

$ gcloud services list --enabled
NAME                              TITLE
bigquery.googleapis.com           BigQuery API
bigquerystorage.googleapis.com    BigQuery Storage API
clouddebugger.googleapis.com      Cloud Debugger API
cloudtrace.googleapis.com         Cloud Trace API
compute.googleapis.com            Compute Engine API
datastore.googleapis.com          Cloud Datastore API
deploymentmanager.googleapis.com  Cloud Deployment Manager V2 API
dns.googleapis.com                Cloud DNS API
iam.googleapis.com                Identity and Access Management (IAM) API
iamcredentials.googleapis.com     IAM Service Account Credentials API
logging.googleapis.com            Cloud Logging API
monitoring.googleapis.com         Cloud Monitoring API
networksecurity.googleapis.com    Network Security API
oslogin.googleapis.com            Cloud OS Login API
servicemanagement.googleapis.com  Service Management API
serviceusage.googleapis.com       Service Usage API
sql-component.googleapis.com      Cloud SQL
storage-api.googleapis.com        Google Cloud Storage JSON API
storage-component.googleapis.com  Cloud Storage
$ 
$ openshift-install create install-config --dir work
? SSH Public Key /home/fedora/.ssh/openshift-qe.pub
? Platform gcp
INFO Credentials loaded from file "/home/fedora/.gcp/osServiceAccount.json" 
FATAL failed to fetch Install Config: failed to fetch dependency of "Install Config": failed to fetch dependency of "Base Domain": failed to generate asset "Platform": failed to get projects: googleapi: Error 403: Cloud Resource Manager API has not been used in project 673766552981 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/cloudresourcemanager.googleapis.com/overview?project=673766552981 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry. 
FATAL Details:                                     
FATAL [                                            
FATAL   {                                          
FATAL     "@type": "type.googleapis.com/google.rpc.Help", 
FATAL     "links": [                               
FATAL       {                                      
FATAL         "description": "Google developers console API activation", 
FATAL         "url": "https://console.developers.google.com/apis/api/cloudresourcemanager.googleapis.com/overview?project=673766552981" 
FATAL       }                                      
FATAL     ]                                        
FATAL   },                                         
FATAL   {                                          
FATAL     "@type": "type.googleapis.com/google.rpc.ErrorInfo", 
FATAL     "domain": "googleapis.com",              
FATAL     "metadata": {
FATAL       "consumer": "projects/673766552981",
FATAL       "service": "cloudresourcemanager.googleapis.com"
FATAL     },
FATAL     "reason": "SERVICE_DISABLED"
FATAL   }
FATAL ]
FATAL , accessNotConfigured
$ 


>2. When all required APIs are enabled and some optional APIs are disabled, intallation could succeed. 

$ gcloud services list --enabled
NAME                                 TITLE
bigquery.googleapis.com              BigQuery API
bigquerystorage.googleapis.com       BigQuery Storage API
clouddebugger.googleapis.com         Cloud Debugger API
cloudresourcemanager.googleapis.com  Cloud Resource Manager API
cloudtrace.googleapis.com            Cloud Trace API
compute.googleapis.com               Compute Engine API
datastore.googleapis.com             Cloud Datastore API
deploymentmanager.googleapis.com     Cloud Deployment Manager V2 API
dns.googleapis.com                   Cloud DNS API
iam.googleapis.com                   Identity and Access Management (IAM) API
iamcredentials.googleapis.com        IAM Service Account Credentials API
logging.googleapis.com               Cloud Logging API
monitoring.googleapis.com            Cloud Monitoring API
networksecurity.googleapis.com       Network Security API
oslogin.googleapis.com               Cloud OS Login API
serviceusage.googleapis.com          Service Usage API
sql-component.googleapis.com         Cloud SQL
$
$ openshift-install create cluster --dir work
? SSH Public Key /home/fedora/.ssh/openshift-qe.pub
? Platform gcp
INFO Credentials loaded from file "/home/fedora/.gcp/osServiceAccount.json"
? Project ID OpenShift QE Shared VPC (openshift-qe-shared-vpc)
? Region us-central1
? Base Domain qe-shared-vpc.qe.gcp.devcluster.openshift.com
? Cluster Name jiwei-0523-11
? Pull Secret [? for help] ******
WARNING the following optional services are not enabled in this project: cloudapis.googleapis.com,servicemanagement.googleapis.com,storage-api.googleapis.com,storage-component.googleapis.com
INFO Creating infrastructure resources...
INFO Waiting up to 20m0s (until 8:19AM) for the Kubernetes API at https://api.jiwei-0523-11.qe-shared-vpc.qe.gcp.devcluster.openshift.com:6443...
INFO API v1.23.3-2051+ad897c4a7774dd-dirty up
INFO Waiting up to 30m0s (until 8:30AM) for bootstrapping to complete...
INFO Destroying the bootstrap resources...
INFO Waiting up to 40m0s (until 8:51AM) for the cluster at https://api.jiwei-0523-11.qe-shared-vpc.qe.gcp.devcluster.openshift.com:6443 to initialize...
INFO Waiting up to 10m0s (until 8:32AM) for the openshift-console route to be created... 
INFO Install complete!                            
INFO To access the cluster as the system:admin user when using 'oc', run 'export KUBECONFIG=/home/fedora/work/auth/kubeconfig' 
INFO Access the OpenShift web-console here: https://console-openshift-console.apps.jiwei-0523-11.qe-shared-vpc.qe.gcp.devcluster.openshift.com 
INFO Login to the console with user: "kubeadmin", and password: "D3s73-c5K44-Kovw3-wknoZ" 
INFO Time elapsed: 26m13s                         
$ 
$ export KUBECONFIG=/home/fedora/work/auth/kubeconfig
$ oc get clusterversion
NAME      VERSION                                                   AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.11.0-0.ci.test-2022-05-23-051805-ci-ln-6zw89i2-latest   True        False         42s     Cluster version is 4.11.0-0.ci.test-2022-05-23-051805-ci-ln-6zw89i2-latest
$ oc get nodes
NAME                                                              STATUS   ROLES    AGE   VERSION
jiwei-0523-11-pttqj-master-0.c.openshift-qe-shared-vpc.internal   Ready    master   19m   v1.23.3+ad897c4
jiwei-0523-11-pttqj-master-1.c.openshift-qe-shared-vpc.internal   Ready    master   20m   v1.23.3+ad897c4
jiwei-0523-11-pttqj-master-2.c.openshift-qe-shared-vpc.internal   Ready    master   20m   v1.23.3+ad897c4
jiwei-0523-11-pttqj-worker-a-648xt                                Ready    worker   11m   v1.23.3+ad897c4
jiwei-0523-11-pttqj-worker-b-69l2c                                Ready    worker   11m   v1.23.3+ad897c4
jiwei-0523-11-pttqj-worker-c-q6xtc                                Ready    worker   11m   v1.23.3+ad897c4
$ oc get co | grep -Ev "True        False         False"
NAME                                       VERSION                                                   AVAILABLE   PROGRESSING   DEGRADED   SINCE   MESSAGE
$ oc get mc
NAME                                               GENERATEDBYCONTROLLER                      IGNITIONVERSION   AGE
00-master                                          b6cfd42372427eb5d7aee44ce30c81c74ae12b08   3.2.0             19m
00-worker                                          b6cfd42372427eb5d7aee44ce30c81c74ae12b08   3.2.0             19m
01-master-container-runtime                        b6cfd42372427eb5d7aee44ce30c81c74ae12b08   3.2.0             19m
01-master-kubelet                                  b6cfd42372427eb5d7aee44ce30c81c74ae12b08   3.2.0             19m
01-worker-container-runtime                        b6cfd42372427eb5d7aee44ce30c81c74ae12b08   3.2.0             19m
01-worker-kubelet                                  b6cfd42372427eb5d7aee44ce30c81c74ae12b08   3.2.0             19m
99-master-generated-registries                     b6cfd42372427eb5d7aee44ce30c81c74ae12b08   3.2.0             19m
99-master-ssh                                                                                 3.2.0             21m
99-worker-generated-registries                     b6cfd42372427eb5d7aee44ce30c81c74ae12b08   3.2.0             19m
99-worker-ssh                                                                                 3.2.0             21m
rendered-master-3bffd1ecfb2e73259110a5e074401ad1   b6cfd42372427eb5d7aee44ce30c81c74ae12b08   3.2.0             18m
rendered-worker-be338dee83aca056100a913b30f2f32c   b6cfd42372427eb5d7aee44ce30c81c74ae12b08   3.2.0             18m
$ oc get mcp
NAME     CONFIG                                             UPDATED   UPDATING   DEGRADED   MACHINECOUNT   READYMACHINECOUNT   UPDATEDMACHINECOUNT   DEGRADEDMACHINECOUNT   AGE
master   rendered-master-3bffd1ecfb2e73259110a5e074401ad1   True      False      False      3              3                   3                     0                      19m
worker   rendered-worker-be338dee83aca056100a913b30f2f32c   True      False      False      3              3                   3                     0                      19m
$ 
$ openshift-install destroy cluster --dir work
INFO Credentials loaded from file "/home/fedora/.gcp/osServiceAccount.json"
INFO Stopped instance jiwei-0523-11-pttqj-worker-a-648xt
INFO Stopped instance jiwei-0523-11-pttqj-worker-b-69l2c
INFO Stopped instance jiwei-0523-11-pttqj-worker-c-q6xtc
INFO Stopped instance jiwei-0523-11-pttqj-master-1
INFO Stopped instance jiwei-0523-11-pttqj-master-2
INFO Stopped instance jiwei-0523-11-pttqj-master-0
INFO Deleted disk jiwei-0523-11-pttqj-master-0
INFO Deleted disk jiwei-0523-11-pttqj-worker-a-648xt
INFO Deleted IAM project role bindings
INFO Deleted service account projects/openshift-qe-shared-vpc/serviceAccounts/jiwei-0523-1-openshift-m-bmqxb.gserviceaccount.com
INFO Deleted service account projects/openshift-qe-shared-vpc/serviceAccounts/jiwei-0523-11-pttqj-m.gserviceaccount.com
INFO Deleted service account projects/openshift-qe-shared-vpc/serviceAccounts/jiwei-0523-1-openshift-i-sll4m.gserviceaccount.com
INFO Deleted service account projects/openshift-qe-shared-vpc/serviceAccounts/jiwei-0523-1-cloud-crede-j8dnn.gserviceaccount.com
INFO Deleted service account projects/openshift-qe-shared-vpc/serviceAccounts/jiwei-0523-1-openshift-c-wvq5n.gserviceaccount.com
INFO Deleted service account projects/openshift-qe-shared-vpc/serviceAccounts/jiwei-0523-11-pttqj-w.gserviceaccount.com
INFO Deleted service account projects/openshift-qe-shared-vpc/serviceAccounts/jiwei-0523-1-openshift-i-mndjp.gserviceaccount.com
INFO Deleted service account projects/openshift-qe-shared-vpc/serviceAccounts/jiwei-0523-1-openshift-g-kvtfq.gserviceaccount.com
INFO Deleted service account projects/openshift-qe-shared-vpc/serviceAccounts/jiwei-0523-1-openshift-g-j7m8c.gserviceaccount.com
INFO Deleted 2 recordset(s) in zone qe-shared-vpc
INFO Deleted 3 recordset(s) in zone jiwei-0523-11-pttqj-private-zone
INFO Deleted DNS zone jiwei-0523-11-pttqj-private-zone
INFO Deleted bucket jiwei-0523-11-pttqj-image-registry-us-central1-uviftttiobfsgec
INFO Deleted instance jiwei-0523-11-pttqj-worker-b-69l2c
INFO Deleted instance jiwei-0523-11-pttqj-master-2
INFO Deleted instance jiwei-0523-11-pttqj-worker-c-q6xtc
INFO Deleted instance jiwei-0523-11-pttqj-master-0
INFO Deleted instance jiwei-0523-11-pttqj-worker-a-648xt
INFO Deleted instance jiwei-0523-11-pttqj-master-1
INFO Deleted disk jiwei-0523-11-pttqj-master-1
INFO Deleted disk jiwei-0523-11-pttqj-worker-b-69l2c
INFO Deleted disk jiwei-0523-11-pttqj-master-2
INFO Deleted disk jiwei-0523-11-pttqj-worker-c-q6xtc
INFO Deleted firewall rule jiwei-0523-11-pttqj-control-plane
INFO Deleted firewall rule jiwei-0523-11-pttqj-etcd
INFO Deleted firewall rule jiwei-0523-11-pttqj-health-checks
INFO Deleted firewall rule jiwei-0523-11-pttqj-internal-cluster
INFO Deleted firewall rule jiwei-0523-11-pttqj-internal-network
INFO Deleted firewall rule k8s-fw-aca270a5a008b444fafb535cbfaa4197
INFO Deleted firewall rule k8s-aca270a5a008b444fafb535cbfaa4197-http-hc
INFO Deleted firewall rule jiwei-0523-11-pttqj-api
INFO Deleted address jiwei-0523-11-pttqj-cluster-ip
INFO Deleted address jiwei-0523-11-pttqj-cluster-public-ip
INFO Deleted forwarding rule aca270a5a008b444fafb535cbfaa4197
INFO Deleted forwarding rule jiwei-0523-11-pttqj-api
INFO Deleted forwarding rule jiwei-0523-11-pttqj-api-internal
INFO Deleted router jiwei-0523-11-pttqj-router
INFO Deleted subnetwork jiwei-0523-11-pttqj-worker-subnet
INFO Deleted target pool jiwei-0523-11-pttqj-api
INFO Deleted target pool aca270a5a008b444fafb535cbfaa4197
INFO Deleted backend service jiwei-0523-11-pttqj-api-internal
INFO Deleted subnetwork jiwei-0523-11-pttqj-master-subnet
INFO Deleted instance group jiwei-0523-11-pttqj-master-us-central1-c
INFO Deleted instance group jiwei-0523-11-pttqj-master-us-central1-a
INFO Deleted instance group jiwei-0523-11-pttqj-master-us-central1-b
INFO Deleted health check jiwei-0523-11-pttqj-api-internal
INFO Deleted HTTP health check aca270a5a008b444fafb535cbfaa4197
INFO Deleted HTTP health check jiwei-0523-11-pttqj-api
INFO Deleted network jiwei-0523-11-pttqj-network
INFO Time elapsed: 3m47s   
$

Comment 6 Jianli Wei 2022-05-24 13:44:33 UTC
Tested again with today's build having the PR https://github.com/openshift/installer/pull/5915 (see https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/release-openshift-origin-installer-launch-gcp-modern/1528912531274338304), installation could succeed when some optional APIs are not enabled. 

$ gcloud config get account
jiwei-0523-sa.gserviceaccount.com
$ gcloud config get project
openshift-qe-shared-vpc
$ 
$ gcloud services list --enabled
NAME                                 TITLE
bigquery.googleapis.com              BigQuery API
bigquerystorage.googleapis.com       BigQuery Storage API
clouddebugger.googleapis.com         Cloud Debugger API
cloudresourcemanager.googleapis.com  Cloud Resource Manager API
cloudtrace.googleapis.com            Cloud Trace API
compute.googleapis.com               Compute Engine API
datastore.googleapis.com             Cloud Datastore API
deploymentmanager.googleapis.com     Cloud Deployment Manager V2 API
dns.googleapis.com                   Cloud DNS API
iamcredentials.googleapis.com        IAM Service Account Credentials API
logging.googleapis.com               Cloud Logging API
monitoring.googleapis.com            Cloud Monitoring API
networksecurity.googleapis.com       Network Security API
oslogin.googleapis.com               Cloud OS Login API
serviceusage.googleapis.com          Service Usage API
sql-component.googleapis.com         Cloud SQL
$
$ openshift-install create cluster --dir work
? SSH Public Key /home/fedora/.ssh/openshift-qe.pub
? Platform gcp
INFO Credentials loaded from file "/home/fedora/.gcp/osServiceAccount.json"
? Project ID OpenShift QE Shared VPC (openshift-qe-shared-vpc)
? Region us-central1
? Base Domain qe-shared-vpc.qe.gcp.devcluster.openshift.com
? Cluster Name jiwei-0524-31
? Pull Secret [? for help] ******
FATAL failed to fetch Cluster: failed to fetch dependency of "Cluster": failed to generate asset "Platform Permissions Check": failed to validate services in this project: the following required services are not enabled in this project: iam.googleapis.com
$
$ gcloud services list --enabled
NAME                                 TITLE
bigquery.googleapis.com              BigQuery API
bigquerystorage.googleapis.com       BigQuery Storage API
clouddebugger.googleapis.com         Cloud Debugger API
cloudresourcemanager.googleapis.com  Cloud Resource Manager API
cloudtrace.googleapis.com            Cloud Trace API
compute.googleapis.com               Compute Engine API
datastore.googleapis.com             Cloud Datastore API
deploymentmanager.googleapis.com     Cloud Deployment Manager V2 API
dns.googleapis.com                   Cloud DNS API
iam.googleapis.com                   Identity and Access Management (IAM) API
iamcredentials.googleapis.com        IAM Service Account Credentials API
logging.googleapis.com               Cloud Logging API
monitoring.googleapis.com            Cloud Monitoring API
networksecurity.googleapis.com       Network Security API
oslogin.googleapis.com               Cloud OS Login API
serviceusage.googleapis.com          Service Usage API
sql-component.googleapis.com         Cloud SQL
$
$ openshift-install create cluster --dir work2
? SSH Public Key /home/fedora/.ssh/openshift-qe.pub
? Platform gcp
INFO Credentials loaded from file "/home/fedora/.gcp/osServiceAccount.json"
? Project ID OpenShift QE Shared VPC (openshift-qe-shared-vpc)
? Region us-central1
? Base Domain qe-shared-vpc.qe.gcp.devcluster.openshift.com
? Cluster Name jiwei-0524-32
? Pull Secret [? for help] ******
WARNING the following optional services are not enabled in this project: cloudapis.googleapis.com,servicemanagement.googleapis.com,storage-api.googleapis.com,storage-component.googleapis.com
INFO Creating infrastructure resources...
INFO Waiting up to 20m0s (until 1:23PM) for the Kubernetes API at https://api.jiwei-0524-32.qe-shared-vpc.qe.gcp.devcluster.openshift.com:6443... 
INFO API v1.23.3-2051+ad897c4a7774dd-dirty up     
INFO Waiting up to 30m0s (until 1:34PM) for bootstrapping to complete... 
INFO Destroying the bootstrap resources...        
INFO Waiting up to 40m0s (until 1:56PM) for the cluster at https://api.jiwei-0524-32.qe-shared-vpc.qe.gcp.devcluster.openshift.com:6443 to initialize...
INFO Waiting up to 10m0s (until 1:39PM) for the openshift-console route to be created... 
INFO Install complete!                            
INFO To access the cluster as the system:admin user when using 'oc', run 'export KUBECONFIG=/home/fedora/work2/auth/kubeconfig' 
INFO Access the OpenShift web-console here: https://console-openshift-console.apps.jiwei-0524-32.qe-shared-vpc.qe.gcp.devcluster.openshift.com 
INFO Login to the console with user: "kubeadmin", and password: "Reeej-4m8TB-T9VrG-fS4dc" 
INFO Time elapsed: 29m45s                         
$ 
$ export KUBECONFIG=/home/fedora/work2/auth/kubeconfig
$ oc get clusterversion
NAME      VERSION                                                   AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.11.0-0.ci.test-2022-05-24-024258-ci-ln-ftlb6v2-latest   True        False         4m13s   Cluster version is 4.11.0-0.ci.test-2022-05-24-024258-ci-ln-ftlb6v2-latest
$ oc get nodes
NAME                                                              STATUS   ROLES    AGE   VERSION
jiwei-0524-32-4n8cp-master-0.c.openshift-qe-shared-vpc.internal   Ready    master   26m   v1.23.3+ad897c4
jiwei-0524-32-4n8cp-master-1.c.openshift-qe-shared-vpc.internal   Ready    master   26m   v1.23.3+ad897c4
jiwei-0524-32-4n8cp-master-2.c.openshift-qe-shared-vpc.internal   Ready    master   27m   v1.23.3+ad897c4
jiwei-0524-32-4n8cp-worker-a-x9xsp                                Ready    worker   15m   v1.23.3+ad897c4
jiwei-0524-32-4n8cp-worker-b-svwvf                                Ready    worker   14m   v1.23.3+ad897c4
jiwei-0524-32-4n8cp-worker-c-6pr5c                                Ready    worker   15m   v1.23.3+ad897c4
$ 
$ oc get co
NAME                                       VERSION                                                   AVAILABLE   PROGRESSING   DEGRADED 
  SINCE   MESSAGE
authentication                             4.11.0-0.ci.test-2022-05-24-024258-ci-ln-ftlb6v2-latest   True        False         False    
  6m19s
baremetal                                  4.11.0-0.ci.test-2022-05-24-024258-ci-ln-ftlb6v2-latest   True        False         False    
  23m
cloud-controller-manager                   4.11.0-0.ci.test-2022-05-24-024258-ci-ln-ftlb6v2-latest   True        False         False      27m
cloud-credential                           4.11.0-0.ci.test-2022-05-24-024258-ci-ln-ftlb6v2-latest   True        False         False      28m
cluster-autoscaler                         4.11.0-0.ci.test-2022-05-24-024258-ci-ln-ftlb6v2-latest   True        False         False      23m
config-operator                            4.11.0-0.ci.test-2022-05-24-024258-ci-ln-ftlb6v2-latest   True        False         False      25m
console                                    4.11.0-0.ci.test-2022-05-24-024258-ci-ln-ftlb6v2-latest   True        False         False      4m33s
csi-snapshot-controller                    4.11.0-0.ci.test-2022-05-24-024258-ci-ln-ftlb6v2-latest   True        False         False      25m
dns                                        4.11.0-0.ci.test-2022-05-24-024258-ci-ln-ftlb6v2-latest   True        False         False      24m
etcd                                       4.11.0-0.ci.test-2022-05-24-024258-ci-ln-ftlb6v2-latest   True        False         False      21m
image-registry                             4.11.0-0.ci.test-2022-05-24-024258-ci-ln-ftlb6v2-latest   True        False         False      13m
ingress                                    4.11.0-0.ci.test-2022-05-24-024258-ci-ln-ftlb6v2-latest   True        False         False      13m
insights                                   4.11.0-0.ci.test-2022-05-24-024258-ci-ln-ftlb6v2-latest   True        False         False      18m
kube-apiserver                             4.11.0-0.ci.test-2022-05-24-024258-ci-ln-ftlb6v2-latest   True        False         False      9m14s
kube-controller-manager                    4.11.0-0.ci.test-2022-05-24-024258-ci-ln-ftlb6v2-latest   True        False         False      21m
kube-scheduler                             4.11.0-0.ci.test-2022-05-24-024258-ci-ln-ftlb6v2-latest   True        False         False      21m
kube-storage-version-migrator              4.11.0-0.ci.test-2022-05-24-024258-ci-ln-ftlb6v2-latest   True        False         False      25m
machine-api                                4.11.0-0.ci.test-2022-05-24-024258-ci-ln-ftlb6v2-latest   True        False         False      20m
machine-approver                           4.11.0-0.ci.test-2022-05-24-024258-ci-ln-ftlb6v2-latest   True        False         False      24m
machine-config                             4.11.0-0.ci.test-2022-05-24-024258-ci-ln-ftlb6v2-latest   True        False         False      23m
marketplace                                4.11.0-0.ci.test-2022-05-24-024258-ci-ln-ftlb6v2-latest   True        False         False      23m
monitoring                                 4.11.0-0.ci.test-2022-05-24-024258-ci-ln-ftlb6v2-latest   True        False         False      10m
network                                    4.11.0-0.ci.test-2022-05-24-024258-ci-ln-ftlb6v2-latest   True        False         False      27m
node-tuning                                4.11.0-0.ci.test-2022-05-24-024258-ci-ln-ftlb6v2-latest   True        False         False      25m
openshift-apiserver                        4.11.0-0.ci.test-2022-05-24-024258-ci-ln-ftlb6v2-latest   True        False         False      9m16s
openshift-controller-manager               4.11.0-0.ci.test-2022-05-24-024258-ci-ln-ftlb6v2-latest   True        False         False      21m
openshift-samples                          4.11.0-0.ci.test-2022-05-24-024258-ci-ln-ftlb6v2-latest   True        False         False      15m
operator-lifecycle-manager                 4.11.0-0.ci.test-2022-05-24-024258-ci-ln-ftlb6v2-latest   True        False         False      24m
operator-lifecycle-manager-catalog         4.11.0-0.ci.test-2022-05-24-024258-ci-ln-ftlb6v2-latest   True        False         False      24m
operator-lifecycle-manager-packageserver   4.11.0-0.ci.test-2022-05-24-024258-ci-ln-ftlb6v2-latest   True        False         False      17m
service-ca                                 4.11.0-0.ci.test-2022-05-24-024258-ci-ln-ftlb6v2-latest   True        False         False      25m
storage                                    4.11.0-0.ci.test-2022-05-24-024258-ci-ln-ftlb6v2-latest   True        False         False      24m
$ 
$ openshift-install destroy cluster --dir work2
INFO Credentials loaded from file "/home/fedora/.gcp/osServiceAccount.json"
INFO Stopped instance jiwei-0524-32-4n8cp-worker-c-6pr5c
INFO Stopped instance jiwei-0524-32-4n8cp-worker-b-svwvf
INFO Stopped instance jiwei-0524-32-4n8cp-worker-a-x9xsp
INFO Stopped instance jiwei-0524-32-4n8cp-master-1
INFO Stopped instance jiwei-0524-32-4n8cp-master-2
INFO Stopped instance jiwei-0524-32-4n8cp-master-0
INFO Deleted IAM project role bindings
INFO Deleted service account projects/openshift-qe-shared-vpc/serviceAccounts/jiwei-0524-3-cloud-crede-6gd9x
.gserviceaccount.com
INFO Deleted service account projects/openshift-qe-shared-vpc/serviceAccounts/jiwei-0524-3-openshift-g-q78pb
.gserviceaccount.com
INFO Deleted service account projects/openshift-qe-shared-vpc/serviceAccounts/jiwei-0524-32-4n8cp-w.gservice
account.com
INFO Deleted service account projects/openshift-qe-shared-vpc/serviceAccounts/jiwei-0524-3-openshift-i-4lcfx
.gserviceaccount.com
INFO Deleted service account projects/openshift-qe-shared-vpc/serviceAccounts/jiwei-0524-32-4n8cp-m.gservice
account.com
INFO Deleted service account projects/openshift-qe-shared-vpc/serviceAccounts/jiwei-0524-3-openshift-c-5rwqq.gserviceaccount.com
INFO Deleted service account projects/openshift-qe-shared-vpc/serviceAccounts/jiwei-0524-3-openshift-g-cpdwg.gserviceaccount.com
INFO Deleted service account projects/openshift-qe-shared-vpc/serviceAccounts/jiwei-0524-3-openshift-m-f28s5.gserviceaccount.com
INFO Deleted service account projects/openshift-qe-shared-vpc/serviceAccounts/jiwei-0524-3-openshift-i-s4vwb.gserviceaccount.com
INFO Deleted 2 recordset(s) in zone qe-shared-vpc
INFO Deleted 3 recordset(s) in zone jiwei-0524-32-4n8cp-private-zone
INFO Deleted DNS zone jiwei-0524-32-4n8cp-private-zone
INFO Deleted bucket jiwei-0524-32-4n8cp-image-registry-us-central1-pjkqcpvnggqghkb
INFO Deleted instance jiwei-0524-32-4n8cp-master-1
INFO Deleted instance jiwei-0524-32-4n8cp-worker-b-svwvf
INFO Deleted instance jiwei-0524-32-4n8cp-master-2
INFO Deleted instance jiwei-0524-32-4n8cp-worker-c-6pr5c
INFO Deleted instance jiwei-0524-32-4n8cp-master-0
INFO Deleted instance jiwei-0524-32-4n8cp-worker-a-x9xsp
INFO Deleted disk jiwei-0524-32-4n8cp-master-2
INFO Deleted disk jiwei-0524-32-4n8cp-worker-c-6pr5c
INFO Deleted disk jiwei-0524-32-4n8cp-master-1
INFO Deleted disk jiwei-0524-32-4n8cp-worker-b-svwvf
INFO Deleted disk jiwei-0524-32-4n8cp-master-0
INFO Deleted disk jiwei-0524-32-4n8cp-worker-a-x9xsp
INFO Deleted firewall rule k8s-fw-a4d9c2107f6274b3eb1173d06aa98f24
INFO Deleted firewall rule k8s-a4d9c2107f6274b3eb1173d06aa98f24-http-hc
INFO Deleted firewall rule jiwei-0524-32-4n8cp-api
INFO Deleted firewall rule jiwei-0524-32-4n8cp-control-plane
INFO Deleted firewall rule jiwei-0524-32-4n8cp-etcd
INFO Deleted firewall rule jiwei-0524-32-4n8cp-health-checks
INFO Deleted firewall rule jiwei-0524-32-4n8cp-internal-cluster
INFO Deleted firewall rule jiwei-0524-32-4n8cp-internal-network
INFO Deleted address jiwei-0524-32-4n8cp-cluster-public-ip
INFO Deleted address jiwei-0524-32-4n8cp-cluster-ip
INFO Deleted forwarding rule a4d9c2107f6274b3eb1173d06aa98f24
INFO Deleted forwarding rule jiwei-0524-32-4n8cp-api
INFO Deleted forwarding rule jiwei-0524-32-4n8cp-api-internal
INFO Deleted router jiwei-0524-32-4n8cp-router
INFO Deleted subnetwork jiwei-0524-32-4n8cp-worker-subnet
INFO Deleted target pool a4d9c2107f6274b3eb1173d06aa98f24
INFO Deleted target pool jiwei-0524-32-4n8cp-api
INFO Deleted backend service jiwei-0524-32-4n8cp-api-internal
INFO Deleted subnetwork jiwei-0524-32-4n8cp-master-subnet
INFO Deleted instance group jiwei-0524-32-4n8cp-master-us-central1-b
INFO Deleted instance group jiwei-0524-32-4n8cp-master-us-central1-c
INFO Deleted instance group jiwei-0524-32-4n8cp-master-us-central1-a
INFO Deleted health check jiwei-0524-32-4n8cp-api-internal
INFO Deleted HTTP health check a4d9c2107f6274b3eb1173d06aa98f24
INFO Deleted HTTP health check jiwei-0524-32-4n8cp-api
INFO Deleted network jiwei-0524-32-4n8cp-network
INFO Time elapsed: 3m37s   
$

Comment 12 Ben Scott 2022-07-14 19:00:32 UTC
@anarayan Can you please provide a list of the optional APIs so that I can update the documentation? Thank you

Comment 13 Aditya Narayanaswamy 2022-07-18 13:17:14 UTC
I added them to the docs text.

Comment 14 errata-xmlrpc 2022-08-10 11:11:18 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: OpenShift Container Platform 4.11.0 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:5069


Note You need to log in before you can comment on or make changes to this bug.