Bug 1887509 - Openshift-tests conformance TopologyManager tests run when Machine Config Operator is not installed on cluster
Summary: Openshift-tests conformance TopologyManager tests run when Machine Config Ope...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Node
Version: 4.5
Hardware: x86_64
OS: Unspecified
unspecified
low
Target Milestone: ---
: 4.7.0
Assignee: Francesco Romani
QA Contact: Walid A.
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-10-12 16:25 UTC by Joseph Goergen
Modified: 2021-02-24 15:25 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: topology manager end to end test assume machine config operator running on each worker node, which is true if deployed on RHCOS nodes but not if deployed on RHEL nodes. Consequence: topology manager end to end test incorrectly fail (false negative) when running against clusters deployed on RHEL Fix: skip the run (do not fail with false negative) if machine config operator is not detected running Result: false negative failure no longer reported.
Clone Of:
Environment:
Last Closed: 2021-02-24 15:25:25 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift origin pull 25689 0 None closed Bug 1887509: e2e: node: filter out worker nodes without MCD 2021-02-16 15:51:22 UTC
Red Hat Product Errata RHSA-2020:5633 0 None None None 2021-02-24 15:25:56 UTC

Description Joseph Goergen 2020-10-12 16:25:41 UTC
Description of problem:
Openshift conformance test fails on clusters without machine-config-daemon pod. The openshift-tests needs to be reconfigured skip or run a test if the Machine Config Operator is not installed on it.

Version-Release number of selected component (if applicable):


How reproducible:
Any 4.5 cluster without the Machine Config Operator installed on it.
run any of the TopologyManager conformance tests.

Steps to Reproduce:
1. Deploy 4.5 machine without the Machine Config Operator installed on it.
2. Run `openshift-tests run-test "[Serial][sig-node][Feature:TopologyManager] Configured cluster with gu workload attached to SRIOV networks should let resource-aligned PODs have working SRIOV network interface [Suite:openshift/conformance/serial]"`


Actual results:

fail [github.com/openshift/origin/test/extended/topology_manager/utils.go:96]: Unexpected error:
    <*errors.errorString | 0xc0001c3a20>: {
        s: "failed to get machine-config-daemon pod for the node \"10.94.78.132\"",
    }
    failed to get machine-config-daemon pod for the node "10.94.78.132"
occurred


Expected results:
Skip.

Additional info:

Comment 1 Sinny Kumari 2020-10-13 13:33:39 UTC
machine-config-daemon must be running on an OCP cluster in order to manage underlying RHCOS nodes. Not sure from where this test is coming coming from. It would be more appropriate to move this bug to component which is running this Openshift conformance test.

@Jospeh Do you know which component in OpenShift is running thsi conformance tests?

Comment 2 Joseph Goergen 2020-10-13 15:53:03 UTC
This is under IBM clouds openshift offering, I believe we are using RHEL instead of RHCOS and we do not require the use of the machine-config-daemon.

Here are the list of tests that fails from this problem
[Serial][sig-node][Feature:TopologyManager] Configured cluster with gu workload attached to SRIOV networks should let resource-aligned PODs have working SRIOV network interface [Suite:openshift/conformance/serial]
[Serial][sig-node][Feature:TopologyManager] Configured cluster with gu workload saturating NUMA nodes should allow a pod requesting as many cores as a full NUMA node have [Suite:openshift/conformance/serial]
[Serial][sig-node][Feature:TopologyManager] Configured cluster with gu workload saturating NUMA nodes should guarantee correct allocation with concurrent creation [Suite:openshift/conformance/serial]
[Serial][sig-node][Feature:TopologyManager] Configured cluster with gu workload saturating NUMA nodes should reject pod requesting more cores than a single NUMA node have [Suite:openshift/conformance/serial]
[Serial][sig-node][Feature:TopologyManager] Configured cluster with gu workload should guarantee NUMA-aligned cpu cores in gu pods with multiple pods, each with a single container requesting 1 core, 1 device [Suite:openshift/conformance/serial]
[Serial][sig-node][Feature:TopologyManager] Configured cluster with gu workload should guarantee NUMA-aligned cpu cores in gu pods with multiple pods, each with a single container requesting 2 core, 1 device [Suite:openshift/conformance/serial]
[Serial][sig-node][Feature:TopologyManager] Configured cluster with gu workload should guarantee NUMA-aligned cpu cores in gu pods with multiple pods, each with multiple containers requesting 1 core, 1 device [Suite:openshift/conformance/serial]
[Serial][sig-node][Feature:TopologyManager] Configured cluster with gu workload should guarantee NUMA-aligned cpu cores in gu pods with multiple pods, each with multiple containers requesting 1 core, only one requesting 1 device [Suite:openshift/conformance/serial]
[Serial][sig-node][Feature:TopologyManager] Configured cluster with gu workload should guarantee NUMA-aligned cpu cores in gu pods with single pod, multiple containers requesting 1 core, 1 device each [Suite:openshift/conformance/serial]
[Serial][sig-node][Feature:TopologyManager] Configured cluster with gu workload should guarantee NUMA-aligned cpu cores in gu pods with single pod, single container requesting 1 core, 1 device [Suite:openshift/conformance/serial]
[Serial][sig-node][Feature:TopologyManager] Configured cluster with gu workload should guarantee NUMA-aligned cpu cores in gu pods with single pod, single container requesting 4 cores, 1 device [Suite:openshift/conformance/serial]
[Serial][sig-node][Feature:TopologyManager] Configured cluster with non-gu workload should run with no regressions with single pod, single container requesting 1 core [Suite:openshift/conformance/serial]
[Serial][sig-node][Feature:TopologyManager] Configured cluster with non-gu workload should run with no regressions with single pod, single container requesting multiple cores [Suite:openshift/conformance/serial]

A couple of the tests that I looked at seem to test high cpu usage on a container/pod? I don't know everything put into it, that's just what I'm guessing at a quick glance

Comment 3 Kirsten Garrison 2020-10-13 18:56:04 UTC
The MCO doesn't own these tests, they seem to be in node,  so passing to them in case they know who to modify them? But again as Sinny said, OCP clusters are generally assumed to be running the MCO & RHCOS, so I'm not really sure if it's even optimal to change the tests as opposed to you writing your own tests.

Seem to indicate sig-node in the above post, so passing over to node since MCO doesn't own/maintain these tests, I'll let them decide what to do or who to pass this onto if they don't maintain them.

Comment 4 Francesco Romani 2020-10-14 16:09:35 UTC
Hi! co-maintainer of the TopologyManager tests here!
The reason why the Topology Manager tests query the machine-config-daemon pod is indeed the one stated in https://bugzilla.redhat.com/show_bug.cgi?id=1887509#c1 .
However, the tests themself don't really depend on MCD features, hence we can work on a fix to make them skip if MCD is not here, or probably even run anyway rearchitecturing them to some extent. A fix will land in 4.7.

Comment 9 Francesco Romani 2020-11-16 08:45:00 UTC
This is actually on tests, but "topology-manager" seems the closest in the current sub-component list. Reassigned.

Comment 20 Walid A. 2021-01-12 16:09:43 UTC
Hitting issues with MCD is in drain loop on one of the 2 RHEL nodes added to a upi on aws OCP 4.7fc2 cluster.
We are hitting https://bugzilla.redhat.com/show_bug.cgi?id=1913582 when trying to apply kubelet config to configure CPU Manager and Topology Manager, unable to verify this BZ now
One of the two RHEL nodes is stuck in SchedulingDisabled state:

ip-10-0-53-28.us-east-2.compute.internal    Ready                      worker   12h   v1.20.0+394a5a3
ip-10-0-55-236.us-east-2.compute.internal   Ready,SchedulingDisabled   worker   12h   v1.20.0+394a5a3


@rphillip looked at my cluster:


oc --kubeconfig kubeconfig_4.7fc2upi-rhel.txt debug node/ip-10-0-55-236.us-east-2.compute.internal


I0112 14:54:39.750805    2507 update.go:1462] Writing systemd unit dropin "mco-disabled.conf"
I0112 14:54:39.755381    2507 update.go:1534] Could not reset unit preset for zincati.service, skipping. (Error msg: error running preset on unit: Failed to execute operation: No such file or directory
)
I0112 14:54:39.885569    2507 update.go:1429] Enabled systemd units: [kubelet.service machine-config-daemon-firstboot.service machine-config-daemon-pull.service node-valid-hostname.service openvswitch.service ovs-configuration.service ovsdb-server.service]
I0112 14:54:40.015536    2507 update.go:1440] Disabled systemd units [nodeip-configuration.service]
I0112 14:54:40.015591    2507 update.go:1280] Deleting stale data
E0112 14:54:40.015670    2507 writer.go:135] Marking Degraded due to: failed to log pending config: logger: unrecognized option '--journald'
Usage:
 logger [options] [message]
Options:
 -T, --tcp             use TCP only
 -d, --udp             use UDP only
 -i, --id              log the process ID too
 -f, --file <file>     log the contents of this file
 -h, --help            display this help text and exit
 -S, --size <num>      maximum size for a single message (default 1024)
 -n, --server <name>   write to this remote syslog server
 -P, --port <port>     use this port for UDP or TCP connection
 -p, --priority <prio> mark given message with this priority
 -s, --stderr          output message to standard error as well
 -t, --tag <tag>       mark every line with this tag
 -u, --socket <socket> write to this Unix socket
 -V, --version         output version information and exit
: exit 
status 1

Comment 21 Francesco Romani 2021-01-12 16:16:23 UTC
@walid if you have the chance to verify that the tests still work as expected when run against OCP on RHCOS, that's already valuable verification (non-regression) and brings us almost half way to verify this BZ.

Comment 22 Sinny Kumari 2021-01-12 17:58:04 UTC
(In reply to Walid A. from comment #20)

> E0112 14:54:40.015670    2507 writer.go:135] Marking Degraded due to: failed
> to log pending config: logger: unrecognized option '--journald'

It is a known issue on RHEL7 and fix is on the way. It is tracked in https://bugzilla.redhat.com/show_bug.cgi?id=1913582

Comment 23 Walid A. 2021-01-19 21:38:37 UTC
Re-ran the Topology Manager E2E tests with RHEL worker nodes on a UPI on AWS OCP 4.7fc3 cluster.

This was a single NUMA setup with no SRIOV cards but the tests were skipped and did not see any E2E failures:

[root@ip-172-31-45-145 openshift]#  $OPENSHIFT_TESTS run openshift/conformance --dry-run | grep -E "TopologyManager" | $OPENSHIFT_TESTS run -f -
openshift-tests version: v4.1.0-3477-g800c265
I0119 21:22:32.963126   56083 test_context.go:457] Tolerating taints "node-role.kubernetes.io/master" when considering if nodes are ready
I0119 21:22:33.022480   56085 test_context.go:457] Tolerating taints "node-role.kubernetes.io/master" when considering if nodes are ready
started: (0/1/11) "[Serial][sig-node][Feature:TopologyManager] Configured cluster with gu workload should guarantee NUMA-aligned cpu cores in gu pods with multiple pods, each with a single container requesting 2 core, 1 device [Suite:openshift/conformance/serial]"

skip [github.com/openshift/origin/test/extended/topology_manager/resourcealign.go:115]: multi-NUMA node system not found in the cluster

skipped: (17.6s) 2021-01-19T21:22:50 "[Serial][sig-node][Feature:TopologyManager] Configured cluster with gu workload should guarantee NUMA-aligned cpu cores in gu pods with multiple pods, each with a single container requesting 2 core, 1 device [Suite:openshift/conformance/serial]"

started: (0/2/11) "[Serial][sig-node][Feature:TopologyManager] Configured cluster with gu workload saturating NUMA nodes should reject pod requesting more cores than a single NUMA node have [Suite:openshift/conformance/serial]"

skip [github.com/openshift/origin/test/extended/topology_manager/resourcealign.go:115]: multi-NUMA node system not found in the cluster

skipped: (17.8s) 2021-01-19T21:23:08 "[Serial][sig-node][Feature:TopologyManager] Configured cluster with gu workload saturating NUMA nodes should reject pod requesting more cores than a single NUMA node have [Suite:openshift/conformance/serial]"

started: (0/3/11) "[Serial][sig-node][Feature:TopologyManager] Configured cluster with gu workload should guarantee NUMA-aligned cpu cores in gu pods with multiple pods, each with multiple containers requesting 1 core, 1 device [Suite:openshift/conformance/serial]"

skip [github.com/openshift/origin/test/extended/topology_manager/resourcealign.go:115]: multi-NUMA node system not found in the cluster

skipped: (17.5s) 2021-01-19T21:23:25 "[Serial][sig-node][Feature:TopologyManager] Configured cluster with gu workload should guarantee NUMA-aligned cpu cores in gu pods with multiple pods, each with multiple containers requesting 1 core, 1 device [Suite:openshift/conformance/serial]"

started: (0/4/11) "[Serial][sig-node][Feature:TopologyManager] Configured cluster with gu workload should guarantee NUMA-aligned cpu cores in gu pods with single pod, single container requesting 4 cores, 1 device [Suite:openshift/conformance/serial]"

skip [github.com/openshift/origin/test/extended/topology_manager/resourcealign.go:115]: multi-NUMA node system not found in the cluster

skipped: (17.5s) 2021-01-19T21:23:43 "[Serial][sig-node][Feature:TopologyManager] Configured cluster with gu workload should guarantee NUMA-aligned cpu cores in gu pods with single pod, single container requesting 4 cores, 1 device [Suite:openshift/conformance/serial]"

started: (0/5/11) "[Serial][sig-node][Feature:TopologyManager] Configured cluster with gu workload saturating NUMA nodes should guarantee correct allocation with concurrent creation [Suite:openshift/conformance/serial]"

skip [github.com/openshift/origin/test/extended/topology_manager/resourcealign.go:115]: multi-NUMA node system not found in the cluster

skipped: (17.9s) 2021-01-19T21:24:01 "[Serial][sig-node][Feature:TopologyManager] Configured cluster with gu workload saturating NUMA nodes should guarantee correct allocation with concurrent creation [Suite:openshift/conformance/serial]"

started: (0/6/11) "[Serial][sig-node][Feature:TopologyManager] Configured cluster with gu workload saturating NUMA nodes should allow a pod requesting as many cores as a full NUMA node have [Suite:openshift/conformance/serial]"

skip [github.com/openshift/origin/test/extended/topology_manager/resourcealign.go:115]: multi-NUMA node system not found in the cluster

skipped: (17.5s) 2021-01-19T21:24:18 "[Serial][sig-node][Feature:TopologyManager] Configured cluster with gu workload saturating NUMA nodes should allow a pod requesting as many cores as a full NUMA node have [Suite:openshift/conformance/serial]"

started: (0/7/11) "[Serial][sig-node][Feature:TopologyManager] Configured cluster with gu workload attached to SRIOV networks should let resource-aligned PODs have working SRIOV network interface [Suite:openshift/conformance/serial]"

skip [github.com/openshift/origin/test/extended/topology_manager/resourcealign.go:115]: multi-NUMA node system not found in the cluster

skipped: (17.5s) 2021-01-19T21:24:36 "[Serial][sig-node][Feature:TopologyManager] Configured cluster with gu workload attached to SRIOV networks should let resource-aligned PODs have working SRIOV network interface [Suite:openshift/conformance/serial]"

started: (0/8/11) "[Serial][sig-node][Feature:TopologyManager] Configured cluster with gu workload should guarantee NUMA-aligned cpu cores in gu pods with multiple pods, each with multiple containers requesting 1 core, only one requesting 1 device [Suite:openshift/conformance/serial]"

skip [github.com/openshift/origin/test/extended/topology_manager/resourcealign.go:115]: multi-NUMA node system not found in the cluster

skipped: (17.5s) 2021-01-19T21:24:53 "[Serial][sig-node][Feature:TopologyManager] Configured cluster with gu workload should guarantee NUMA-aligned cpu cores in gu pods with multiple pods, each with multiple containers requesting 1 core, only one requesting 1 device [Suite:openshift/conformance/serial]"

started: (0/9/11) "[Serial][sig-node][Feature:TopologyManager] Configured cluster with gu workload should guarantee NUMA-aligned cpu cores in gu pods with single pod, multiple containers requesting 1 core, 1 device each [Suite:openshift/conformance/serial]"

skip [github.com/openshift/origin/test/extended/topology_manager/resourcealign.go:115]: multi-NUMA node system not found in the cluster

skipped: (17.6s) 2021-01-19T21:25:11 "[Serial][sig-node][Feature:TopologyManager] Configured cluster with gu workload should guarantee NUMA-aligned cpu cores in gu pods with single pod, multiple containers requesting 1 core, 1 device each [Suite:openshift/conformance/serial]"

started: (0/10/11) "[Serial][sig-node][Feature:TopologyManager] Configured cluster with gu workload should guarantee NUMA-aligned cpu cores in gu pods with multiple pods, each with a single container requesting 1 core, 1 device [Suite:openshift/conformance/serial]"

skip [github.com/openshift/origin/test/extended/topology_manager/resourcealign.go:115]: multi-NUMA node system not found in the cluster

skipped: (17.5s) 2021-01-19T21:25:29 "[Serial][sig-node][Feature:TopologyManager] Configured cluster with gu workload should guarantee NUMA-aligned cpu cores in gu pods with multiple pods, each with a single container requesting 1 core, 1 device [Suite:openshift/conformance/serial]"

started: (0/11/11) "[Serial][sig-node][Feature:TopologyManager] Configured cluster with gu workload should guarantee NUMA-aligned cpu cores in gu pods with single pod, single container requesting 1 core, 1 device [Suite:openshift/conformance/serial]"

skip [github.com/openshift/origin/test/extended/topology_manager/resourcealign.go:115]: multi-NUMA node system not found in the cluster

skipped: (17.5s) 2021-01-19T21:25:46 "[Serial][sig-node][Feature:TopologyManager] Configured cluster with gu workload should guarantee NUMA-aligned cpu cores in gu pods with single pod, single container requesting 1 core, 1 device [Suite:openshift/conformance/serial]"


Timeline:

Jan 19 21:22:33.597 I ns/e2e-test-topology-manager-4kz8q namespace/e2e-test-topology-manager-4kz8q reason/CreatedSCCRanges created SCC ranges
Jan 19 21:22:33.619 I ns/e2e-test-topology-manager-4kz8q namespace/e2e-test-topology-manager-4kz8q reason/CreatedSCCRanges created SCC ranges (2 times)
Jan 19 21:22:51.091 I ns/e2e-test-topology-manager-dh4cc namespace/e2e-test-topology-manager-dh4cc reason/CreatedSCCRanges created SCC ranges
Jan 19 21:22:51.119 I ns/e2e-test-topology-manager-dh4cc namespace/e2e-test-topology-manager-dh4cc reason/CreatedSCCRanges created SCC ranges (2 times)
Jan 19 21:23:08.879 I ns/e2e-test-topology-manager-x8t5q namespace/e2e-test-topology-manager-x8t5q reason/CreatedSCCRanges created SCC ranges
Jan 19 21:23:26.355 I ns/e2e-test-topology-manager-bbvsk namespace/e2e-test-topology-manager-bbvsk reason/CreatedSCCRanges created SCC ranges
Jan 19 21:23:29.669 I ns/openshift-marketplace pod/redhat-operators-rmpxl node/ reason/Created
Jan 19 21:23:29.669 I ns/openshift-marketplace pod/community-operators-b7p5f node/ reason/Created
Jan 19 21:23:29.681 I ns/openshift-marketplace pod/redhat-operators-rmpxl node/ip-10-0-63-79.us-east-2.compute.internal reason/Scheduled
Jan 19 21:23:29.684 I ns/openshift-marketplace pod/community-operators-b7p5f node/ip-10-0-63-79.us-east-2.compute.internal reason/Scheduled
Jan 19 21:23:29.705 I ns/openshift-marketplace pod/redhat-marketplace-v7z9f node/ reason/Created
Jan 19 21:23:29.708 I ns/openshift-marketplace pod/qe-app-registry-cbz5q node/ reason/Created
Jan 19 21:23:29.716 I ns/openshift-marketplace pod/redhat-marketplace-v7z9f node/ip-10-0-63-79.us-east-2.compute.internal reason/Scheduled
Jan 19 21:23:29.723 I ns/openshift-marketplace pod/qe-app-registry-cbz5q node/ip-10-0-50-24.us-east-2.compute.internal reason/Scheduled
Jan 19 21:23:31.623 I ns/openshift-marketplace pod/qe-app-registry-cbz5q reason/AddedInterface Add eth0 [10.130.2.21/23]
Jan 19 21:23:31.775 I ns/openshift-marketplace pod/community-operators-b7p5f reason/AddedInterface Add eth0 [10.128.2.79/23]
Jan 19 21:23:31.796 I ns/openshift-marketplace pod/qe-app-registry-cbz5q node/ip-10-0-50-24.us-east-2.compute.internal container/registry-server reason/Pulling image/quay.io/openshift-qe-optional-operators/ocp4-index:latest
Jan 19 21:23:32.023 I ns/openshift-marketplace pod/redhat-operators-rmpxl reason/AddedInterface Add eth0 [10.128.2.80/23]
Jan 19 21:23:32.124 I ns/openshift-marketplace pod/community-operators-b7p5f node/ip-10-0-63-79.us-east-2.compute.internal container/registry-server reason/Pulling image/registry.redhat.io/redhat/community-operator-index:latest
Jan 19 21:23:32.382 I ns/openshift-marketplace pod/redhat-operators-rmpxl node/ip-10-0-63-79.us-east-2.compute.internal container/registry-server reason/Pulling image/registry.redhat.io/redhat/redhat-operator-index:v4.6
Jan 19 21:23:32.440 I ns/openshift-marketplace pod/redhat-marketplace-v7z9f reason/AddedInterface Add eth0 [10.128.2.81/23]
Jan 19 21:23:32.759 I ns/openshift-marketplace pod/redhat-marketplace-v7z9f node/ip-10-0-63-79.us-east-2.compute.internal container/registry-server reason/Pulling image/registry.redhat.io/redhat/redhat-marketplace-index:v4.6
Jan 19 21:23:33.012 I ns/openshift-marketplace pod/qe-app-registry-cbz5q node/ip-10-0-50-24.us-east-2.compute.internal container/registry-server reason/Pulled image/quay.io/openshift-qe-optional-operators/ocp4-index:latest
Jan 19 21:23:33.183 I ns/openshift-marketplace pod/qe-app-registry-cbz5q node/ip-10-0-50-24.us-east-2.compute.internal container/registry-server reason/Created
Jan 19 21:23:33.219 I ns/openshift-marketplace pod/qe-app-registry-cbz5q node/ip-10-0-50-24.us-east-2.compute.internal container/registry-server reason/Started
Jan 19 21:23:33.287 I ns/openshift-marketplace pod/redhat-operators-rmpxl node/ip-10-0-63-79.us-east-2.compute.internal container/registry-server reason/Pulled image/registry.redhat.io/redhat/redhat-operator-index:v4.6
Jan 19 21:23:33.473 I ns/openshift-marketplace pod/redhat-operators-rmpxl node/ip-10-0-63-79.us-east-2.compute.internal container/registry-server reason/Created
Jan 19 21:23:33.515 I ns/openshift-marketplace pod/redhat-operators-rmpxl node/ip-10-0-63-79.us-east-2.compute.internal container/registry-server reason/Started
Jan 19 21:23:33.765 I ns/openshift-marketplace pod/community-operators-b7p5f node/ip-10-0-63-79.us-east-2.compute.internal container/registry-server reason/Pulled image/registry.redhat.io/redhat/community-operator-index:latest
Jan 19 21:23:33.946 I ns/openshift-marketplace pod/community-operators-b7p5f node/ip-10-0-63-79.us-east-2.compute.internal container/registry-server reason/Created
Jan 19 21:23:33.976 I ns/openshift-marketplace pod/community-operators-b7p5f node/ip-10-0-63-79.us-east-2.compute.internal container/registry-server reason/Started
Jan 19 21:23:37.639 I ns/openshift-marketplace pod/redhat-marketplace-v7z9f node/ip-10-0-63-79.us-east-2.compute.internal container/registry-server reason/Pulled image/registry.redhat.io/redhat/redhat-marketplace-index:v4.6
Jan 19 21:23:37.795 I ns/openshift-marketplace pod/redhat-marketplace-v7z9f node/ip-10-0-63-79.us-east-2.compute.internal container/registry-server reason/Created
Jan 19 21:23:37.822 I ns/openshift-marketplace pod/redhat-marketplace-v7z9f node/ip-10-0-63-79.us-east-2.compute.internal container/registry-server reason/Started
Jan 19 21:23:40.402 I ns/openshift-marketplace pod/qe-app-registry-cbz5q node/ip-10-0-50-24.us-east-2.compute.internal container/registry-server reason/Ready
Jan 19 21:23:40.417 W ns/openshift-marketplace pod/qe-app-registry-cbz5q node/ip-10-0-50-24.us-east-2.compute.internal reason/GracefulDelete in 0s
Jan 19 21:23:40.422 W ns/openshift-marketplace pod/qe-app-registry-cbz5q node/ip-10-0-50-24.us-east-2.compute.internal reason/Deleted
Jan 19 21:23:40.425 I ns/openshift-marketplace pod/qe-app-registry-cbz5q node/ip-10-0-50-24.us-east-2.compute.internal container/registry-server reason/Killing
Jan 19 21:23:40.431 I ns/openshift-marketplace pod/qe-app-registry-cbz5q node/ip-10-0-50-24.us-east-2.compute.internal container/registry-server reason/Killing
Jan 19 21:23:42.070 I ns/openshift-marketplace pod/community-operators-b7p5f node/ip-10-0-63-79.us-east-2.compute.internal container/registry-server reason/Ready
Jan 19 21:23:42.087 W ns/openshift-marketplace pod/community-operators-b7p5f node/ip-10-0-63-79.us-east-2.compute.internal reason/GracefulDelete in 0s
Jan 19 21:23:42.092 W ns/openshift-marketplace pod/community-operators-b7p5f node/ip-10-0-63-79.us-east-2.compute.internal reason/Deleted
Jan 19 21:23:42.095 I ns/openshift-marketplace pod/community-operators-b7p5f node/ip-10-0-63-79.us-east-2.compute.internal container/registry-server reason/Killing
Jan 19 21:23:42.101 I ns/openshift-marketplace pod/community-operators-b7p5f node/ip-10-0-63-79.us-east-2.compute.internal container/registry-server reason/Killing
Jan 19 21:23:42.781 I ns/openshift-marketplace pod/redhat-operators-rmpxl node/ip-10-0-63-79.us-east-2.compute.internal container/registry-server reason/Ready
Jan 19 21:23:42.798 W ns/openshift-marketplace pod/redhat-operators-rmpxl node/ip-10-0-63-79.us-east-2.compute.internal reason/GracefulDelete in 0s
Jan 19 21:23:42.803 W ns/openshift-marketplace pod/redhat-operators-rmpxl node/ip-10-0-63-79.us-east-2.compute.internal reason/Deleted
Jan 19 21:23:42.806 I ns/openshift-marketplace pod/redhat-operators-rmpxl node/ip-10-0-63-79.us-east-2.compute.internal container/registry-server reason/Killing
Jan 19 21:23:42.821 I ns/openshift-marketplace pod/redhat-operators-rmpxl node/ip-10-0-63-79.us-east-2.compute.internal container/registry-server reason/Killing
Jan 19 21:23:43.534 I ns/openshift-marketplace pod/redhat-marketplace-v7z9f node/ip-10-0-63-79.us-east-2.compute.internal container/registry-server reason/Ready
Jan 19 21:23:43.584 W ns/openshift-marketplace pod/redhat-marketplace-22z2m node/ip-10-0-63-79.us-east-2.compute.internal reason/GracefulDelete in 0s
Jan 19 21:23:43.593 W ns/openshift-marketplace pod/redhat-marketplace-22z2m node/ip-10-0-63-79.us-east-2.compute.internal reason/Deleted
Jan 19 21:23:43.594 I ns/openshift-marketplace pod/redhat-marketplace-22z2m node/ip-10-0-63-79.us-east-2.compute.internal container/registry-server reason/Killing
Jan 19 21:23:43.606 I ns/openshift-marketplace pod/redhat-marketplace-22z2m node/ip-10-0-63-79.us-east-2.compute.internal container/registry-server reason/Killing
Jan 19 21:23:43.924 I ns/e2e-test-topology-manager-5zbl5 namespace/e2e-test-topology-manager-5zbl5 reason/CreatedSCCRanges created SCC ranges
Jan 19 21:24:01.743 I ns/e2e-test-topology-manager-p47qf namespace/e2e-test-topology-manager-p47qf reason/CreatedSCCRanges created SCC ranges
Jan 19 21:24:19.314 I ns/e2e-test-topology-manager-dwlrt namespace/e2e-test-topology-manager-dwlrt reason/CreatedSCCRanges created SCC ranges
Jan 19 21:24:36.870 I ns/e2e-test-topology-manager-vl8kp namespace/e2e-test-topology-manager-vl8kp reason/CreatedSCCRanges created SCC ranges
Jan 19 21:24:54.409 I ns/e2e-test-topology-manager-cx8t5 namespace/e2e-test-topology-manager-cx8t5 reason/CreatedSCCRanges created SCC ranges
Jan 19 21:25:11.988 I ns/e2e-test-topology-manager-6n2zj namespace/e2e-test-topology-manager-6n2zj reason/CreatedSCCRanges created SCC ranges
Jan 19 21:25:29.503 I ns/e2e-test-topology-manager-qhd2v namespace/e2e-test-topology-manager-qhd2v reason/CreatedSCCRanges created SCC ranges
Jan 19 21:25:29.528 I ns/e2e-test-topology-manager-qhd2v namespace/e2e-test-topology-manager-qhd2v reason/CreatedSCCRanges created SCC ranges (2 times)

Synthetic test results:
passed: [sig-node] kubelet terminates kube-apiserver gracefully
passed: [sig-node] pods should never transition back to pending
passed: [sig-node] pods should not fail on systemd timeouts
passed: [sig-network] pods should successfully create sandboxes by reading container
passed: [sig-network] pods should successfully create sandboxes by not timing out
passed: [sig-network] pods should successfully create sandboxes by writing network status
passed: [sig-network] pods should successfully create sandboxes by getting pod
passed: [sig-network] pods should successfully create sandboxes by writing child
passed: [sig-network] pods should successfully create sandboxes by other
passed: [sig-api-machinery] kube-apiserver-new-connection should be available
passed: [sig-api-machinery] openshift-apiserver-new-connection should be available
passed: [sig-api-machinery] oauth-apiserver-new-connection should be available
passed: [sig-api-machinery] kube-apiserver-reused-connection should be available
passed: [sig-api-machinery] openshift-apiserver-reused-connection should be available
passed: [sig-api-machinery] oauth-apiserver-reused-connection should be available
(duplicates are used to mark some failures as flake and not to fail the whole suite

0 pass, 11 skip (3m14s)

@Francesco Romani is this sufficient to verify this BZ ?

Comment 24 Francesco Romani 2021-01-21 16:13:32 UTC
@walid the test cluster looks good. It skips later in the flow, suggesting MCO is running in the cluster, and thus MCD in the nodes. Is this the case?

Comment 25 Francesco Romani 2021-01-21 16:41:20 UTC
(In reply to Francesco Romani from comment #24)
> @walid the test cluster looks good. It skips later in the flow, suggesting
> MCO is running in the cluster, and thus MCD in the nodes. Is this the case?

Anyway, it seems good enough. We passed the specific code point and the behaviour we seen is consistent with the expectations.

Comment 27 errata-xmlrpc 2021-02-24 15:25:25 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:5633


Note You need to log in before you can comment on or make changes to this bug.