Bug 1919980

Summary: oVirt installer fails due to terraform error "Failed to wait for Templte(...) to become ok"
Product: OpenShift Container Platform Reporter: Gal Zaidman <gzaidman>
Component: InstallerAssignee: Janos Bonic <jpasztor>
Installer sub component: OpenShift on RHV QA Contact: Michael Burman <mburman>
Status: CLOSED ERRATA Docs Contact:
Severity: low    
Priority: low CC: mburman
Version: 4.8   
Target Milestone: ---   
Target Release: 4.11.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-08-10 10:35:38 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 2082283    
Bug Blocks:    

Description Gal Zaidman 2021-01-25 13:59:59 UTC
Description of problem:

Sometimes oVirt installer fails due to terraform error:
```
level=debug msg=2021-01-22T22:22:44.534Z [DEBUG] plugin.terraform-provider-ovirt: 2021/01/22 22:22:44 [DEBUG] Failed to wait for Templte(efcb7a8f-8f61-46d0-be46-9ba0d1170d82) to become ok: couldn't find resource (21 retries)

level=error msg=Error: couldn't find resource (21 retries)
level=error
level=error msg=  on ../tmp/openshift-install-088314764/template/main.tf line 78, in resource "ovirt_template" "releaseimage_template":
level=error msg=  78: resource "ovirt_template" "releaseimage_template" {
level=error
level=error
level=fatal msg=failed to fetch Cluster: failed to generate asset "Cluster": failed to create cluster: failed to apply Terraform: failed to complete the change
```

I saw it while there was huge pressure on CI, notice that the timeout hire is 4m[1], maybe worth trying to increase it.

happend 4 times past 14 days:
- https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/release-openshift-ocp-installer-e2e-ovirt-4.7/1352739628699881472
- https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/openshift_release/15125/rehearse-15125-pull-ci-openshift-csi-external-snapshotter-release-4.7-e2e-ovirt/1353620724668960768
- https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/openshift_release/15125/rehearse-15125-pull-ci-openshift-csi-livenessprobe-release-4.7-e2e-ovirt/1353620716271964160
- https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/release-openshift-ocp-installer-e2e-ovirt-4.7/1352014442572812288

[1] https://github.com/oVirt/terraform-provider-ovirt/blob/master/ovirt/resource_ovirt_vm_template.go#L419

Comment 1 Gal Zaidman 2021-01-27 12:56:06 UTC
Pushing to next sprint

Comment 2 Gal Zaidman 2021-03-30 14:10:27 UTC
due to capacity constraints, we will be revisiting this bug in the upcoming sprint

Comment 3 Janos Bonic 2021-07-01 11:58:17 UTC
This will be fixed when we switch to the new client library.

Comment 5 Janos Bonic 2022-06-21 07:09:24 UTC
Fixed in 2082283

Comment 6 Michael Burman 2022-06-21 08:25:06 UTC
(In reply to Janos Bonic from comment #5)
> Fixed in 2082283

Fixed in bz 2082283

Comment 7 Michael Burman 2022-06-23 12:36:52 UTC
The issue is gone.
Such terraform errors no longer visible in the installer.

Verified on - 4.11.0-0.nightly-2022-06-22-235234 and rhvm-4.5.1.2-0.11.el8ev.noarch

Comment 9 errata-xmlrpc 2022-08-10 10:35:38 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: OpenShift Container Platform 4.11.0 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:5069