As far as I can tell, these are the same symptoms from bug 1788253. But since that bug is VERIFIED, here's a new bug so we can sort out whether we've regressed, never actually fixed the issue, or have a separate issue that happens to have similar symptoms. Example 4.5.0-0.nightly-2020-03-16-182052 job [1,2]: Mar 16 19:00:25.139: INFO: wait: LatestVersion: 2 STEP: making sure DC can be scaled [AfterEach] adoption /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/test/extended/deployments/deployments.go:1566 Mar 16 19:00:55.197: INFO: Running 'oc --namespace=e2e-test-cli-deployment-6hd86 --kubeconfig=/tmp/configfile536210205 get dc/deployment-simple -o yaml' And here's a 4.4.0-0.nightly-2020-03-15-041004 job [3,4]: Mar 15 05:05:21.499: INFO: wait: LatestVersion: 2 STEP: making sure DC can be scaled [AfterEach] adoption [Conformance] /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/test/extended/deployments/deployments.go:1566 Mar 15 05:05:51.560: INFO: Running 'oc --namespace=e2e-test-cli-deployment-qc6gw --kubeconfig=/tmp/configfile183933056 get dc/deployment-simple -o yaml' [5] has a curl you can run to spit out failing 4.4 and 4.5 jobs, and a bit of context around their "making sure DC can be scaled" log line, in case that helps when triaging. [1]: https://prow.svc.ci.openshift.org/view/gcs/origin-ci-test/logs/release-openshift-ocp-installer-e2e-aws-4.5/421 [2]: https://storage.googleapis.com/origin-ci-test/logs/release-openshift-ocp-installer-e2e-aws-4.5/421/build-log.txt [3]: https://prow.svc.ci.openshift.org/view/gcs/origin-ci-test/logs/release-openshift-ocp-installer-e2e-azure-4.4/1317 [4]: https://storage.googleapis.com/origin-ci-test/logs/release-openshift-ocp-installer-e2e-azure-4.4/1317/build-log.txt [5]: https://bugzilla.redhat.com/show_bug.cgi?id=1788253
@yinzhou Changed a workloads case to you, thanks
@yinzhou Changed a workloads bug to you, thanks
*** Bug 1817473 has been marked as a duplicate of this bug. ***
job failure observed: https://prow.svc.ci.openshift.org/view/gcs/origin-ci-test/logs/release-openshift-ocp-installer-e2e-gcp-rt-4.4/157
Similar CI job failure: https://prow.svc.ci.openshift.org/view/gcs/origin-ci-test/logs/release-openshift-ocp-installer-e2e-aws-4.3/1337
*** Bug 1829300 has been marked as a duplicate of this bug. ***
Adding search results showing this is still happening fairly frequently https://search-clayton-ci-search.apps.build01.ci.devcluster.openshift.com/?search=will+orphan+all+RCs+and+adopt+them+back+when+recreated&maxAge=336h&context=1&type=bug%2Bjunit&name=&maxMatches=5&maxBytes=20971520&groupBy=job Sippy says this fails 3.5% of the time in 4.5.
fixing search link: https://search.apps.build01.ci.devcluster.openshift.com/?search=will+orphan+all+RCs+and+adopt+them+back+when+recreated&maxAge=48h&context=1&type=bug%2Bjunit&name=&maxMatches=5&maxBytes=20971520&groupBy=job 2.53% of runs that fail, have this failure, which is relatively high.
I’m adding UpcomingSprint, because I was occupied by fixing bugs with higher priority/severity, developing new features with higher priority, or developing new features to improve stability at a macro level. I will revisit this bug next sprint.
I'm not seeing it as a top item on that list? And sippy for 4.6 is showing it passing at 98% across 287 runs and it doesn't even show up on the 4.5 sippy page (which means it's passing at a high enough rate that sippy cuts it off, or sippy is bugged) (Still worth fixing, i don't think there's any reason a test like this shouldn't be able to pass at 99.9999%, but it doesn't seem like a major source of CI failures right now, compared to where it used to be)
Example recent failure: https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/release-openshift-ocp-e2e-aws-scaleup-rhel7-4.2/1314177462690648064
(In reply to Benjamin Gilbert from comment #13) > Example recent failure: > https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/release-openshift- > ocp-e2e-aws-scaleup-rhel7-4.2/1314177462690648064 This failure is coming from a 4.2 where the fix is not available. Given that I'm not seeing any recent problem I'm closing this.