Bug 2015624

Summary: Conformance tests set to skip on all IBM Cloud offerings (managed and unmanaged)
Product: OpenShift Container Platform Reporter: Christopher J Schaefer <cschaefe>
Component: Test FrameworkAssignee: Christopher J Schaefer <cschaefe>
Status: CLOSED WONTFIX QA Contact: Jian Zhang <jiazha>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 4.10CC: jiazha, wking
Target Milestone: ---   
Target Release: 4.10.0   
Hardware: Unspecified   
OS: Unspecified   
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2024-04-30 18:04:53 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Christopher J Schaefer 2021-10-19 16:52:38 UTC
Description of problem:
Some conformance tests have been set to skip if the detected cloud provider is IBM Cloud. This was due to the tests not supporting or passing on the IBM Cloud managed Openshift offering.

However, with OCP 4.10, IBM Cloud will be supporting an IPI installation of OCP on IBM Cloud (unmanaged), and we expect these tests to be run and pass. The hardcoded skips on the IBM Cloud provider need to be fixed to be dependent upon the "ControlPlaneTopology" being "External" (managed) rather than just based off the cloud provider.

Version-Release number of selected component (if applicable):

How reproducible:

All invocations of openshift-tests using an openshift/conformance test suite will skip tests if the provider is detected as 'ibmcloud'

Steps to Reproduce:

1. Create an OCP cluster on IBM Cloud (for instance via IPI)
2. Setup KUBECONFIG to the new OCP cluster
3. Run the conformance test suite against the cluster, providing the "provider" as IBM Cloud.  (openshift-tests run openshift/conformance --provider='{"type":"ibmcloud"}'

Actual results:

A number of test cases are skipped since it is expected they are running against a "managed" OCP cluster.

Expected results:

All expected test cases should run against a normal OCP cluster (including on IBM Cloud).

Additional info:

IBM Cloud is working on a potential fix to address this current issue.
The long term solution is likely to include a fix that will also skip these tests for other providers that have an "External" CP topology, since they won't be limited to just IBM Cloud (some tests will no longer depend on a check of IBM Cloud provider, just "External" CP topology).

Example of such a "skip" determined when "provider" is 'ibmcloud', which would also be skipped for "unmanaged" (IPI) deployed OCP clusters.

Comment 1 Devan Goodwin 2021-11-17 14:29:54 UTC
Hey Jeff, could you assign this to whomever is working on the change to address this?

Comment 4 Jian Zhang 2022-01-25 10:10:25 UTC
Create a cluster that running on IBM cloud, and its infrastructureTopology is NOT `External`.

mac:amd64 jianzhang$ oc get clusterversion
NAME      VERSION                              AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.10.0-0.nightly-2022-01-25-023600   True        False         11m     Cluster version is 4.10.0-0.nightly-2022-01-25-023600

mac:amd64 jianzhang$ oc get infrastructure cluster -o yaml
apiVersion: config.openshift.io/v1
kind: Infrastructure
  creationTimestamp: "2022-01-25T08:29:24Z"
  generation: 1
  name: cluster
  resourceVersion: "592"
  uid: 98c6461a-4be6-42c6-810c-100a2e34ddd0
    key: config
    name: cloud-provider-config
    type: IBMCloud
  apiServerInternalURI: https://api-int.jiazha25.ibmcloud.qe.devcluster.openshift.com:6443
  apiServerURL: https://api.jiazha25.ibmcloud.qe.devcluster.openshift.com:6443
  controlPlaneTopology: HighlyAvailable
  etcdDiscoveryDomain: ""
  infrastructureName: jiazha25-s9scx
  infrastructureTopology: HighlyAvailable
  platform: IBMCloud
      cisInstanceCRN: 'crn:v1:bluemix:public:internet-svcs:global:a/fdc2e14cf8bc4d53a67f972dc2e2c861:e8ee6ca1-4b31-4307-8190-e67f6925f83b::'
      location: eu-gb
      resourceGroupName: jiazha25-s9scx
    type: IBMCloud

Run a e2e test that was skipped before, for example,
mac:amd64 jianzhang$ ./openshift-tests run all --dry-run|grep "verify build metrics"|./openshift-tests run -f -
I0125 18:05:53.957692   31733 test_context.go:419] Tolerating taints "node-role.kubernetes.io/master" when considering if nodes are ready
I0125 18:05:59.281911   31735 test_context.go:419] Tolerating taints "node-role.kubernetes.io/master" when considering if nodes are ready
started: (0/1/1) "[Feature:Prometheus][Feature:Builds] Prometheus when installed on the cluster [Top Level] [Feature:Prometheus][Feature:Builds] Prometheus when installed on the cluster should start and expose a secured proxy and verify build metrics [Suite:openshift/conformance/parallel]"

fail [github.com/openshift/origin/test/extended/util/client.go:681]: Jan 25 18:06:40.317: OAuthAccessToken.oauth.openshift.io "Xbg3_c3aSiGrgVzmeUOVqQAAAAAAAAAA" is invalid: metadata.name: Invalid value: "Xbg3_c3aSiGrgVzmeUOVqQAAAAAAAAAA": only the new format of tokens (prefixed with sha256~) is allowed

Failing tests:

[Feature:Prometheus][Feature:Builds] Prometheus when installed on the cluster [Top Level] [Feature:Prometheus][Feature:Builds] Prometheus when installed on the cluster should start and expose a secured proxy and verify build metrics [Suite:openshift/conformance/parallel]

error: 1 fail, 0 pass, 0 skip (2m28s)

As you can see, it is not skipped now, LGTM, verify it.

Comment 7 Rory Thrasher 2024-04-30 18:04:53 UTC
OCP is no longer using Bugzilla and this bug appears to have been left in an orphaned state. If the bug is still relevant, please open a new issue in the OCPBUGS Jira project: https://issues.redhat.com/projects/OCPBUGS/summary