Bug 2033215 - Flaky CI: crud/other-routes.spec.ts fails sometimes with an cypress ace/a11y AssertionError: 1 accessibility violation was detected
Summary: Flaky CI: crud/other-routes.spec.ts fails sometimes with an cypress ace/a11y ...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Dev Console
Version: 4.10
Hardware: Unspecified
OS: Unspecified
high
medium
Target Milestone: ---
: 4.10.0
Assignee: Christoph Jerolimov
QA Contact: spathak@redhat.com
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-12-16 09:04 UTC by Christoph Jerolimov
Modified: 2022-03-10 16:34 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-03-10 16:34:29 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift console pull 10674 0 None open Bug 2033215: Do not render an empty button if it has no content to fix a11y check 2021-12-16 09:11:23 UTC
Red Hat Product Errata RHSA-2022:0056 0 None Waiting on Red Hat ACM Channels are unable to pull git repo from Azure Dev Ops Git repository 2022-05-09 06:44:38 UTC

Description Christoph Jerolimov 2021-12-16 09:04:08 UTC
Description of problem:
Our CI tests fail sometimes with a cypress ace/a11y/accessibility error in the crud/other-routes.spec.ts which is not related to the change our devs have done.

This happens because of some race conditions between the test and the prometheus data shown on the admin dashboard.

Version-Release number of selected component (if applicable):
4.10 (did not test old branches)

How reproducible:
Sometimes

Steps to Reproduce:
1. Run a CI test

Actual results:
CI tests fails sometimes with this error:

  Running:  crud/other-routes.spec.ts                                                     (10 of 18)

  Visiting other routes
  Logging in as kubeadmin
1 accessibility violation was detected for [object Object] page
┌─────────┬───────────────┬────────────┬─────────────────────────────────────────┬───────┐
│ (index) │      id       │   impact   │               description               │ nodes │
├─────────┼───────────────┼────────────┼─────────────────────────────────────────┼───────┤
│    0    │ 'button-name' │ 'critical' │ 'Ensures buttons have discernible text' │   4   │
└─────────┴───────────────┴────────────┴─────────────────────────────────────────┴───────┘
1 accessibility violation was detected for [object Object] page
┌─────────┬───────────────┬────────────┬─────────────────────────────────────────┬───────┐
│ (index) │      id       │   impact   │               description               │ nodes │
├─────────┼───────────────┼────────────┼─────────────────────────────────────────┼───────┤
│    0    │ 'button-name' │ 'critical' │ 'Ensures buttons have discernible text' │   4   │
└─────────┴───────────────┴────────────┴─────────────────────────────────────────┴───────┘
    1) successfully displays view for route:  
    ✓ successfully displays view for route:  k8s cluster clusterroles view (2342ms)
    ✓ successfully displays view for route:  k8s cluster nodes (2970ms)
    ✓ successfully displays view for route:  k8s all-namespaces events (3383ms)
    ✓ successfully displays view for route:  k8s all-namespaces import (3290ms)
    ✓ successfully displays view for route:  api-explorer (7525ms)
    ✓ successfully displays view for route:  api-resource ns default core~v1~Pod (1105ms)
    ✓ successfully displays view for route:  api-resource ns default core~v1~Pod schema (2335ms)
    ✓ successfully displays view for route:  api-resource ns default core~v1~Pod instances (2292ms)
    ✓ successfully displays view for route:  api-resource ns default core~v1~Pod access (2540ms)
    ✓ successfully displays view for route:  k8s cluster user.openshift.io~v1~User (1058ms)
    ✓ successfully displays view for route:  k8s ns openshift-machine-api machine.openshift.io~v1beta1~Machine (2649ms)
    ✓ successfully displays view for route:  k8s ns openshift-machine-api machine.openshift.io~v1beta1~MachineSet (2798ms)
    ✓ successfully displays view for route:  k8s ns openshift-machine-api autoscaling.openshift.io~v1beta1~MachineAutoscaler (2391ms)
    ✓ successfully displays view for route:  k8s ns openshift-machine-api machine.openshift.io~v1beta1~MachineHealthCheck (2501ms)
    ✓ successfully displays view for route:  k8s cluster machineconfiguration.openshift.io~v1~MachineConfig (2962ms)
    ✓ successfully displays view for route:  k8s cluster machineconfiguration.openshift.io~v1~MachineConfigPool (2441ms)
    ✓ successfully displays view for route:  k8s all-namespaces monitoring.coreos.com~v1~Alertmanager (2491ms)
    ✓ successfully displays view for route:  k8s ns openshift-monitoring monitoring.coreos.com~v1~Alertmanager main (2692ms)
    ✓ successfully displays view for route:  settings cluster (2474ms)
    ✓ successfully displays view for route:  monitoring query-browser (2436ms)
  Logging out
    ✓ successfully displays view for route:  search all-namespaces?kind=config.openshift.io~v1~Console (2557ms)

  Test perspective query parameters
  Logging in as kubeadmin
    ✓ tests Developer query parameter (10366ms)
  Logging out
    ✓ tests Administrator query parameter (5320ms)


  23 passing (2m)
  1 failing

  1) Visiting other routes
       successfully displays view for route:  :
     AssertionError: 1 accessibility violation was detected: expected 1 to equal 0
      at Context.eval (https://console-openshift-console.apps.ci-op-xk7cycyk-75d12.**********************/__cypress/tests?p=tests/crud/other-routes.spec.ts:10178:20)



Expected results:
At least if the PR doesn't break something this cypress test should pass.

Additional info:

Comment 1 Christoph Jerolimov 2021-12-20 11:24:14 UTC
This change only affects the CI test stability and could not be tested on a cluster. Set this as verified and will continuously watch the CI failure rate.

Comment 5 errata-xmlrpc 2022-03-10 16:34:29 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.10.3 security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:0056


Note You need to log in before you can comment on or make changes to this bug.