Created attachment 1769566 [details] functests-cn-ran-du.log Description of problem: Performance Topology Manager test is failing when running cnf-tests in discovery mode. Version-Release number of selected component (if applicable): 4.8.0-0.nightly-2021-04-03-092337 CNF tests image: registry-proxy.engineering.redhat.com/rh-osbs/openshift4-cnf-tests:v4.8.0-19 How reproducible: Run containerised tests in discovery mode against existing 4.8 clusters. Steps to Reproduce: Run containerised tests in discovery mode against existing 4.8 clusters for performance addon operator feature: podman run -v /root/cnf_ci/cnf-internal-deploy/_cache/:/kubeconfig:Z -v /tmp/artifacts/cn-ran-du:/reports:Z -e CLEAN_PERFORMANCE_PROFILE=false -e CNF_TESTS_IMAGE=openshift4-cnf-tests:v4.8.0-19 -e DPDK_TESTS_IMAGE=dpdk-base:v4.8.0-2 -e IMAGE_REGISTRY=registry-proxy.engineering.redhat.com/rh-osbs/ -e KUBECONFIG=/kubeconfig/kubeconfig -e SCTPTEST_HAS_NON_CNF_WORKERS=false -e DISCOVERY_MODE=true -e NODES_SELECTOR=node-role.kubernetes.io/worker-duprofile= -e ROLE_WORKER_CNF=worker-duprofile -e LATENCY_TEST_RUN=false -e LATENCY_TEST_RUNTIME=600 -e OSLAT_MAXIMUM_LATENCY=200 registry-proxy.engineering.redhat.com/rh-osbs/openshift4-cnf-tests:v4.8.0-19 /usr/bin/test-run.sh -ginkgo.focus 'performance|ptp|sriov|sctp|dpdk|ovn' -junit /reports/ -report /reports/ Actual results: • Failure [0.246 seconds] [rfe_id:27350][performance]Topology Manager /remote-source/app/vendor/github.com/openshift-kni/performance-addon-operators/functests/1_performance/topology_manager.go:17 [test_id:26932][crit:high][vendor:cnf-qe][level:acceptance] should be enabled with the policy specified in profile [It] /remote-source/app/vendor/github.com/openshift-kni/performance-addon-operators/functests/1_performance/topology_manager.go:36 Topology Manager policy mismatch got "" expected "restricted" Expected <string>: to equal <string>: restricted /remote-source/app/vendor/github.com/openshift-kni/performance-addon-operators/functests/1_performance/topology_manager.go:42 Expected results: The test should pass.
Created attachment 1769567 [details] tests-artifacts
I believe this is related to bug 1946584
Also, undefined Topology Manager policy seems to point to an issue while applying the KubeletConfig. Swati: check MCO labels and KubeletConfig / PerfProfile MCO selector here please.
Created attachment 1770214 [details] PAO must gather
Yep, same issue as https://bugzilla.redhat.com/show_bug.cgi?id=1946584 There are three worker nodes in the cluster cnfdd5 - with labels node-role.kubernetes.io/worker: "" and node-role.kubernetes.io/worker-cnf: "" cnfdd6 - with labels node-role.kubernetes.io/worker: "" and node-role.kubernetes.io/worker-cnf: "" cnfdd7 - with labels node-role.kubernetes.io/worker: "" and node-role.kubernetes.io/worker-duprofile: "" 2 performance profiles perf-example.yaml with nodeSelector as node-role.kubernetes.io/worker-duprofile: "" performance.yaml with nodeSelector as node-role.kubernetes.io/worker-cnf: "" Which means that cnfdd7 should be targeted and profile should be applied to it.
*** This bug has been marked as a duplicate of bug 1946584 ***