The following accessibility violations need to be addressed: https://search.ci.openshift.org/?search=accessibility+violation+was+detected&maxAge=336h&context=1&type=junit&name=pull-ci-openshift-console-master-e2e-gcp-console&excludeName=&maxMatches=5&maxBytes=20971520&groupBy=job ┌────────────────────────────────────────────────────────────────────────────────────────────────┐ │ Cypress: 8.5.0 │ │ Browser: Chrome 94 (headless) │ │ Specs: 1 found (e2e/add-flow-ci.feature) │ │ Searched: features/e2e/add-flow-ci.feature │ └────────────────────────────────────────────────────────────────────────────────────────────────┘ ──────────────────────────────────────────────────────────────────────────────────────────────────── Running: e2e/add-flow-ci.feature (1 of 1) Couldn't determine Mocha version Logging in as kubeadmin Create the different workloads from Add page ✓ Getting started resources on Developer perspective (16626ms) 1 accessibility violation was detected for Developer Catalog ┌─────────┬────────┬───────────┬───────────────────────────────────────────────┬───────┐ │ (index) │ id │ impact │ description │ nodes │ ├─────────┼────────┼───────────┼───────────────────────────────────────────────┼───────┤ │ 0 │ 'list' │ 'serious' │ 'Ensures that lists are structured correctly' │ 1 │ └─────────┴────────┴───────────┴───────────────────────────────────────────────┴───────┘ 1) Deploy Application using Catalog Template "CI/CD": A-01-TC02 (example #1) 1 accessibility violation was detected for Developer Catalog ┌─────────┬────────┬───────────┬───────────────────────────────────────────────┬───────┐ │ (index) │ id │ impact │ description │ nodes │ ├─────────┼────────┼───────────┼───────────────────────────────────────────────┼───────┤ │ 0 │ 'list' │ 'serious' │ 'Ensures that lists are structured correctly' │ 1 │ └─────────┴────────┴───────────┴───────────────────────────────────────────────┴───────┘ 2) Deploy Application using Catalog Template "Databases": A-01-TC02 (example #2) 1 accessibility violation was detected for Developer Catalog ┌─────────┬────────┬───────────┬───────────────────────────────────────────────┬───────┐ │ (index) │ id │ impact │ description │ nodes │ ├─────────┼────────┼───────────┼───────────────────────────────────────────────┼───────┤ │ 0 │ 'list' │ 'serious' │ 'Ensures that lists are structured correctly' │ 1 │ └─────────┴────────┴───────────┴───────────────────────────────────────────────┴───────┘ 3) Deploy Application using Catalog Template "Languages": A-01-TC02 (example #3) 1 accessibility violation was detected for Developer Catalog ┌─────────┬────────┬───────────┬───────────────────────────────────────────────┬───────┐ │ (index) │ id │ impact │ description │ nodes │ ├─────────┼────────┼───────────┼───────────────────────────────────────────────┼───────┤ │ 0 │ 'list' │ 'serious' │ 'Ensures that lists are structured correctly' │ 1 │ └─────────┴────────┴───────────┴───────────────────────────────────────────────┴───────┘ 4) Deploy Application using Catalog Template "Middleware": A-01-TC02 (example #4) 1 accessibility violation was detected for Developer Catalog ┌─────────┬────────┬───────────┬───────────────────────────────────────────────┬───────┐ │ (index) │ id │ impact │ description │ nodes │ ├─────────┼────────┼───────────┼───────────────────────────────────────────────┼───────┤ │ 0 │ 'list' │ 'serious' │ 'Ensures that lists are structured correctly' │ 1 │ └─────────┴────────┴───────────┴───────────────────────────────────────────────┴───────┘ 5) Deploy Application using Catalog Template "Other": A-01-TC02 (example #5) 1 accessibility violation was detected for Deploy Page ┌─────────┬────────┬───────────┬───────────────────────────────────────────────┬───────┐ │ (index) │ id │ impact │ description │ nodes │ ├─────────┼────────┼───────────┼───────────────────────────────────────────────┼───────┤ │ 0 │ 'list' │ 'serious' │ 'Ensures that lists are structured correctly' │ 1 │ └─────────┴────────┴───────────┴───────────────────────────────────────────────┴───────┘ 6) Deploy secure image with Runtime icon from external registry: A-02-TC02 (example #1) 1 accessibility violation was detected for Deploy Page ┌─────────┬────────┬───────────┬───────────────────────────────────────────────┬───────┐ │ (index) │ id │ impact │ description │ nodes │ ├─────────┼────────┼───────────┼───────────────────────────────────────────────┼───────┤ │ 0 │ 'list' │ 'serious' │ 'Ensures that lists are structured correctly' │ 1 │ └─────────┴────────┴───────────┴───────────────────────────────────────────────┴───────┘ 7) Deploy image with Runtime icon from internal registry: A-02-TC03 (example #1) 1 accessibility violation was detected for Deploy Page ┌─────────┬────────┬───────────┬───────────────────────────────────────────────┬───────┐ │ (index) │ id │ impact │ description │ nodes │ ├─────────┼────────┼───────────┼───────────────────────────────────────────────┼───────┤ │ 0 │ 'list' │ 'serious' │ 'Ensures that lists are structured correctly' │ 1 │ └─────────┴────────┴───────────┴───────────────────────────────────────────────┴───────┘ 8) Edit Runtime Icon while Editing Image: A-02-TC05 1 accessibility violation was detected for Developer Catalog ┌─────────┬────────┬───────────┬───────────────────────────────────────────────┬───────┐ │ (index) │ id │ impact │ description │ nodes │ ├─────────┼────────┼───────────┼───────────────────────────────────────────────┼───────┤ │ 0 │ 'list' │ 'serious' │ 'Ensures that lists are structured correctly' │ 1 │ └─────────┴────────┴───────────┴───────────────────────────────────────────────┴───────┘ 9) Create the Database from Add page: A-03-TC01 10) Deploy git workload with devfile from topology page: A-04-TC01 1 accessibility violation was detected for Import from Docker file ┌─────────┬────────┬───────────┬───────────────────────────────────────────────┬───────┐ │ (index) │ id │ impact │ description │ nodes │ ├─────────┼────────┼───────────┼───────────────────────────────────────────────┼───────┤ │ 0 │ 'list' │ 'serious' │ 'Ensures that lists are structured correctly' │ 1 │ └─────────┴────────┴───────────┴───────────────────────────────────────────────┴───────┘ 11) Create a workload from Docker file with "Deployment" as resource type: A-05-TC02 (example #1) 12) Create a workload from YAML file: A-07-TC01 1 accessibility violation was detected for Upload JAR file ┌─────────┬────────┬───────────┬───────────────────────────────────────────────┬───────┐ │ (index) │ id │ impact │ description │ nodes │ ├─────────┼────────┼───────────┼───────────────────────────────────────────────┼───────┤ │ 0 │ 'list' │ 'serious' │ 'Ensures that lists are structured correctly' │ 1 │ └─────────┴────────┴───────────┴───────────────────────────────────────────────┴───────┘ 13) Upload Jar file page details: A-10-TC01 1 passing (3m) 13 failing
From the cypress recordings, it all seems to go south when developer catalog takes some time to load. Let me see if adding a simple wait solves this.
Hi jrichter, I understand the desire to add a simple wait, but it is typically better to strive for a deterministic solution: https://docs.cypress.io/guides/references/best-practices#Unnecessary-Waiting. thanks
(In reply to David Taylor from comment #4) > Hi jrichter, I understand the desire to add a simple wait, but it > is typically better to strive for a deterministic solution: > https://docs.cypress.io/guides/references/best-practices#Unnecessary-Waiting. > thanks The simple part is just adding a conditional wait that we already use to wait for pages to load. The wait itself is not that 'simple', sorry for the confusion :)
Hi, still seeing this flake, CI success rate is currently at 0% partially due to this issue! Indeterminate waiting is not reliable to squash flakes like this. Best to wait on an assertion such as url should equal, or a 'cy.contains' Thanks, Dave # Create the different workloads from Add page.Create the different workloads from Add page Deploy image with Runtime icon from internal registry: A-02-TC03 (example #1) AssertionError: 1 accessibility violation was detected: expected 1 to equal 0 at Context.eval (https://console-openshift-console.apps.ci-op-lb7gpwtl-75d12.**********************/__cypress/tests?p=support/commands/index.ts:10178:20)
(In reply to David Taylor from comment #7) > Hi, still seeing this flake, CI success rate is currently at 0% partially > due to this issue! > Indeterminate waiting is not reliable to squash flakes like this. Best to > wait on an assertion such as url should equal, or a 'cy.contains' > Thanks, > Dave > > # Create the different workloads from Add page.Create the different > workloads from Add page Deploy image with Runtime icon from internal > registry: A-02-TC03 (example #1) > AssertionError: 1 accessibility violation was detected: expected 1 to equal 0 > at Context.eval > (https://console-openshift-console.apps.ci-op-lb7gpwtl-75d12. > **********************/__cypress/tests?p=support/commands/index.ts:10178:20) AFAIK this is a different issue to what was happening before. The developer catalog cases are still nice and green. And the wait is actually not indeterminate: https://github.com/openshift/console/blob/d482d27a0dd8ec5c42521d99f9c9e33a7b7402df/frontend/packages/dev-console/integration-tests/support/pages/app.ts#L15 Now I just need to figure out what's wrong in this picture: https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/pr-logs/pull/openshift_console/10645/pull-ci-openshift-console-master-e2e-gcp-console/1469230738531422208/artifacts/e2e-gcp-console/test/artifacts/gui_test_screenshots/cypress/screenshots/e2e/add-flow-ci.feature/Create%20the%20different%20workloads%20from%20Add%20page%20--%20Edit%20Runtime%20Icon%20while%20Editing%20Image%20A-02-TC05%20(failed).png
Searched for the reported CI issues with the link in the first comment: https://search.ci.openshift.org/?search=accessibility+violation+was+detected&maxAge=336h&context=1&type=junit&name=pull-ci-openshift-console-master-e2e-gcp-console&excludeName=&maxMatches=5&maxBytes=20971520&groupBy=job 620 runs, 43% failed, 0% of failures match = 0% impact Only tests are changed, so it can not affect any user. Mark this as verified.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.10.3 security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:0056