Bug 1936859 - ovirt 4.4 -> 4.5 upgrade jobs are permafailing
Summary: ovirt 4.4 -> 4.5 upgrade jobs are permafailing
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 4.4
Hardware: Unspecified
OS: Unspecified
medium
low
Target Milestone: ---
: 4.8.0
Assignee: Gal Zaidman
QA Contact: Gal Zaidman
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-03-09 11:21 UTC by Vadim Rutkovsky
Modified: 2021-07-27 22:52 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
operator conditions monitoring
Last Closed: 2021-07-27 22:51:56 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift release pull 17871 0 None open Bug 1936859: Fix oVirt release 4.4 and 4.5 workflows 2021-04-21 09:53:32 UTC
Red Hat Product Errata RHSA-2021:2438 0 None None None 2021-07-27 22:52:20 UTC

Description Vadim Rutkovsky 2021-03-09 11:21:02 UTC
test:
operator conditions monitoring 

is failing frequently in CI, see search results:
https://search.ci.openshift.org/?maxAge=168h&context=1&type=bug%2Bjunit&name=&maxMatches=5&maxBytes=20971520&groupBy=job&search=operator+conditions+monitoring

4.4.z install on oVirt can't complete as monitoring can't rollout. This blocks 4.4.z -> 4.5 upgrade tests.

example job: https://prow.ci.openshift.org/view/gcs/origin-ci-test/logs/periodic-ci-openshift-release-master-ci-4.5-upgrade-from-stable-4.4-e2e-ovirt-upgrade/1369211579001737216

level=info msg="Cluster operator monitoring Available is False with : "
level=info msg="Cluster operator monitoring Progressing is True with RollOutInProgress: Rolling out the stack."
level=error msg="Cluster operator monitoring Degraded is True with UpdatingPrometheusK8SFailed: Failed to rollout the stack. Error: running task Updating Prometheus-k8s failed: waiting for Prometheus object changes failed: waiting for Prometheus: expected 2 replicas, updated 0 and available 0"
level=fatal msg="failed to initialize the cluster: Cluster operator monitoring is still updating"

Comment 2 Gal Zaidman 2021-03-30 13:34:53 UTC
This is a CI only issue and is in progress
due to capacity constraints we will be revisiting this bug in the upcoming sprint

Comment 3 Douglas Schilling Landgraf 2021-04-20 14:34:36 UTC
Should be the same problem as 4.5 jobs. Please see: https://bugzilla.redhat.com/show_bug.cgi?id=1936857#c5
Moving to Gal.

Comment 6 Gal Zaidman 2021-04-28 07:04:56 UTC
Hi(In reply to Vadim Rutkovsky from comment #5)
> Not yet fixed - see
> https://prow.ci.openshift.org/job-history/gs/origin-ci-test/logs/periodic-ci-
> openshift-release-master-ci-4.5-upgrade-from-stable-4.4-e2e-ovirt-upgrade
> *
> https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-
> openshift-release-master-ci-4.5-upgrade-from-stable-4.4-e2e-ovirt-upgrade/
> 1386606443997696000
> *
> https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-
> openshift-release-master-ci-4.5-upgrade-from-stable-4.4-e2e-ovirt-upgrade/
> 1386244053321912320
> *
> https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-
> openshift-release-master-ci-4.5-upgrade-from-stable-4.4-e2e-ovirt-upgrade/
> 1385881662205726720

Hi those are upgrade jobs and that is a different workflow.
I believe that the upgrade issue was resolved by https://github.com/openshift/release/pull/18091.
But anyway I think you can mark it as verified since 4.4 jobs are passing installation (hitting docker rate limiting on test cases but that is a different problem)

Comment 7 Gal Zaidman 2021-05-20 11:21:52 UTC
Moving to verified now

Comment 9 Vadim Rutkovsky 2021-05-20 12:50:56 UTC
Although, lets create a new issue for that

Comment 15 errata-xmlrpc 2021-07-27 22:51:56 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.8.2 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:2438


Note You need to log in before you can comment on or make changes to this bug.