Bug 2015624 - Conformance tests set to skip on all IBM Cloud offerings (managed and unmanaged)
Summary: Conformance tests set to skip on all IBM Cloud offerings (managed and unmanaged)
Keywords:
Status: VERIFIED
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Test Framework
Version: 4.10
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: 4.10.0
Assignee: Christopher J Schaefer
QA Contact: Jian Zhang
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-10-19 16:52 UTC by Christopher J Schaefer
Modified: 2022-12-16 07:30 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift origin pull 26537 0 None Merged Bug 2015624: Refactor IBM Cloud skips to External 2021-12-21 21:13:53 UTC

Description Christopher J Schaefer 2021-10-19 16:52:38 UTC
Description of problem:
Some conformance tests have been set to skip if the detected cloud provider is IBM Cloud. This was due to the tests not supporting or passing on the IBM Cloud managed Openshift offering.

However, with OCP 4.10, IBM Cloud will be supporting an IPI installation of OCP on IBM Cloud (unmanaged), and we expect these tests to be run and pass. The hardcoded skips on the IBM Cloud provider need to be fixed to be dependent upon the "ControlPlaneTopology" being "External" (managed) rather than just based off the cloud provider.

Version-Release number of selected component (if applicable):
4.10

How reproducible:

All invocations of openshift-tests using an openshift/conformance test suite will skip tests if the provider is detected as 'ibmcloud'

Steps to Reproduce:

1. Create an OCP cluster on IBM Cloud (for instance via IPI)
2. Setup KUBECONFIG to the new OCP cluster
3. Run the conformance test suite against the cluster, providing the "provider" as IBM Cloud.  (openshift-tests run openshift/conformance --provider='{"type":"ibmcloud"}'

Actual results:

A number of test cases are skipped since it is expected they are running against a "managed" OCP cluster.

Expected results:

All expected test cases should run against a normal OCP cluster (including on IBM Cloud).

Additional info:

IBM Cloud is working on a potential fix to address this current issue.
The long term solution is likely to include a fix that will also skip these tests for other providers that have an "External" CP topology, since they won't be limited to just IBM Cloud (some tests will no longer depend on a check of IBM Cloud provider, just "External" CP topology).

Example of such a "skip" determined when "provider" is 'ibmcloud', which would also be skipped for "unmanaged" (IPI) deployed OCP clusters.
https://github.com/openshift/origin/blob/b418b4058c91c873b5a2cc180c8718686793c4db/test/extended/csrapprover/csrapprover.go#L41-L44

Comment 1 Devan Goodwin 2021-11-17 14:29:54 UTC
Hey Jeff, could you assign this to whomever is working on the change to address this?

Comment 4 Jian Zhang 2022-01-25 10:10:25 UTC
Create a cluster that running on IBM cloud, and its infrastructureTopology is NOT `External`.

mac:amd64 jianzhang$ oc get clusterversion
NAME      VERSION                              AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.10.0-0.nightly-2022-01-25-023600   True        False         11m     Cluster version is 4.10.0-0.nightly-2022-01-25-023600

mac:amd64 jianzhang$ oc get infrastructure cluster -o yaml
apiVersion: config.openshift.io/v1
kind: Infrastructure
metadata:
  creationTimestamp: "2022-01-25T08:29:24Z"
  generation: 1
  name: cluster
  resourceVersion: "592"
  uid: 98c6461a-4be6-42c6-810c-100a2e34ddd0
spec:
  cloudConfig:
    key: config
    name: cloud-provider-config
  platformSpec:
    type: IBMCloud
status:
  apiServerInternalURI: https://api-int.jiazha25.ibmcloud.qe.devcluster.openshift.com:6443
  apiServerURL: https://api.jiazha25.ibmcloud.qe.devcluster.openshift.com:6443
  controlPlaneTopology: HighlyAvailable
  etcdDiscoveryDomain: ""
  infrastructureName: jiazha25-s9scx
  infrastructureTopology: HighlyAvailable
  platform: IBMCloud
  platformStatus:
    ibmcloud:
      cisInstanceCRN: 'crn:v1:bluemix:public:internet-svcs:global:a/fdc2e14cf8bc4d53a67f972dc2e2c861:e8ee6ca1-4b31-4307-8190-e67f6925f83b::'
      location: eu-gb
      resourceGroupName: jiazha25-s9scx
    type: IBMCloud

Run a e2e test that was skipped before, for example,
mac:amd64 jianzhang$ ./openshift-tests run all --dry-run|grep "verify build metrics"|./openshift-tests run -f -
I0125 18:05:53.957692   31733 test_context.go:419] Tolerating taints "node-role.kubernetes.io/master" when considering if nodes are ready
I0125 18:05:59.281911   31735 test_context.go:419] Tolerating taints "node-role.kubernetes.io/master" when considering if nodes are ready
started: (0/1/1) "[Feature:Prometheus][Feature:Builds] Prometheus when installed on the cluster [Top Level] [Feature:Prometheus][Feature:Builds] Prometheus when installed on the cluster should start and expose a secured proxy and verify build metrics [Suite:openshift/conformance/parallel]"
...

fail [github.com/openshift/origin/test/extended/util/client.go:681]: Jan 25 18:06:40.317: OAuthAccessToken.oauth.openshift.io "Xbg3_c3aSiGrgVzmeUOVqQAAAAAAAAAA" is invalid: metadata.name: Invalid value: "Xbg3_c3aSiGrgVzmeUOVqQAAAAAAAAAA": only the new format of tokens (prefixed with sha256~) is allowed
...
...

Failing tests:

[Feature:Prometheus][Feature:Builds] Prometheus when installed on the cluster [Top Level] [Feature:Prometheus][Feature:Builds] Prometheus when installed on the cluster should start and expose a secured proxy and verify build metrics [Suite:openshift/conformance/parallel]

error: 1 fail, 0 pass, 0 skip (2m28s)

As you can see, it is not skipped now, LGTM, verify it.


Note You need to log in before you can comment on or make changes to this bug.