Description of problem: Currently our periodic and pre-submit jobs run and unknowingly there could be core dumps where OVN crashed. We are seeing that in periodic upgrade jobs like: https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-openshift-release-master-ci-4.8-upgrade-from-stable-4.7-e2e-aws-ovn-upgrade/1389623166254452736 We should have an e2e test that checks to see if there are cores after a run and then fail the test if so.
The first step was getting our normal e2e-*-ovn jobs to be generated jobs and not templated so they can use workflows where we have a core dump step. that PR is here: https://github.com/openshift/release/pull/20503
the work to fail on core dumps wont be merged until we get the oustanding core dump resolved in glibc which is tracked in https://bugzilla.redhat.com/show_bug.cgi?id=1957030
The PR to do this is finally merged and working as expected: https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-openshift-release-master-ci-4.10-upgrade-from-stable-4.9-e2e-aws-ovn-upgrade/1507152507262996480 https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/logs/periodic-ci-openshift-release-master-ci-4.10-upgrade-from-stable-4.9-e2e-aws-ovn-upgrade/1507152507262996480/artifacts/e2e-aws-ovn-upgrade/gather-core-dump/build-log.txt