Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1846212

Summary:	After grafana setup, if next engine-setup fails, rollback fails
Product:	[oVirt] ovirt-engine	Reporter:	Yedidyah Bar David <didi>
Component:	Setup.EngineCommon	Assignee:	Yedidyah Bar David <didi>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Pavel Novotny <pnovotny>
Severity:	medium	Docs Contact:
Priority:	high
Version:	4.4.1	CC:	bugs, lleistne
Target Milestone:	ovirt-4.4.1	Flags:	pm-rhel: ovirt-4.4+
Target Release:	4.4.1.4
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	ovirt-engine-4.4.1.4	Doc Type:	No Doc Update
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2020-08-05 06:25:00 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	Metrics	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Yedidyah Bar David 2020-06-11 05:43:35 UTC

Description of problem:

Creation of grafana db user is done in a way that causes pg_dump to dump statements that the dwh db user has no rights to execute, so if we need to rollback, we fail. This is in the setup log:

2020-06-11 08:31:05,641+0300 ERROR otopi.ovirt_engine_setup.engine_common.database database.restore:956 Errors while restoring ovirt_engine_history database, please check the log file for details
2020-06-11 08:31:05,641+0300 DEBUG otopi.ovirt_engine_setup.engine_common.database database.restore:961 Errors unfiltered during restore:

pg_restore: from TOC entry 3234; 826 16411 DEFAULT ACL DEFAULT PRIVILEGES FOR TABLES postgres
pg_restore: error: could not execute query: ERROR:  must be member of role "postgres"
Command was: ALTER DEFAULT PRIVILEGES FOR ROLE postgres IN SCHEMA public REVOKE ALL ON TABLES  FROM postgres;
ALTER DEFAULT PRIVILEGES FOR ROLE postgres IN SCHEMA public GRANT SELECT ON TABLES  TO ovirt_engine_history_grafana;

Version-Release number of selected component (if applicable):
Current master

How reproducible:
Always

Steps to Reproduce:
1. Install and setup engine+dwh+grafana
2. Run engine-setup again, make it fail after it updated dwh db schema
3.

Actual results:
Fails as above

Expected results:
Succeeds, by ignoring these errors - they can be ignored, because we create the grafana db user in a way that will still work even if these statements fail.

Additional info:
Noticed while checking bug 1839676, which is basically a duplicate except for grafana. Decided to also open current to not postpone verification of that one.

Comment 1 Pavel Novotny 2020-07-13 19:12:34 UTC

Verified in
ovirt-engine-4.4.1.8-0.7.el8ev.noarch
otopi-common-1.9.2-1.el8ev.noarch

Verification steps:
1. Install engine+dwh+grafana.
2. Run setup again and fail it after 'Creating/refreshing DWH database schema' step:
   - install otopi-debug-plugins
   - run: OTOPI_FORCE_FAIL_STAGE=STAGE_MISC OTOPI_FORCE_FAIL_PRIORITY=PRIORITY_LOW engine-setup

Result:
Setup fails after refreshing DWH DB schema step.
Rollback of DWH DB then succeeds, error 'pg_restore: error: could not execute query: ERROR:  must be member of role "postgres"' is ignored.

Console output:
-~-
...
[ INFO  ] Backing up database localhost:ovirt_engine_history to '/var/lib/ovirt-engine-dwh/backups/dwh-20200713205557.de65z02w.dump'.
[ INFO  ] Creating/refreshing DWH database schema
[ INFO  ] Configuring WebSocket Proxy
[ INFO  ] Backing up database localhost:engine to '/var/lib/ovirt-engine/backups/engine-20200713205611.dngmii2a.dump'.
[ INFO  ] Creating/refreshing Engine database schema
[ INFO  ] Creating/refreshing Engine 'internal' domain database schema
          Unregistering existing client registration info.
[ ERROR ] Failed to execute stage 'Misc configuration': Force Fail: stage 11 priority 9000
[ INFO  ] DNF Performing DNF transaction rollback
[WARNING] Rollback of DWH database postponed to Stage "Clean up"
[ INFO  ] Rolling back database schema
[ INFO  ] Clearing Engine database engine
[ INFO  ] Restoring Engine database engine
[ INFO  ] Restoring file '/var/lib/ovirt-engine/backups/engine-20200713205611.dngmii2a.dump' to database localhost:engine.
[ INFO  ] Stage: Clean up
          Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-setup-20200713204338-bxekez.log
[ INFO  ] Generating answer file '/var/lib/ovirt-engine/setup/answers/20200713205757-setup.conf'
[WARNING] Rollback of DWH database started
          This might be a long process, but it should be safe to start the engine service before it finishes, if needed.
[ INFO  ] Clearing DWH database ovirt_engine_history
[ INFO  ] Restoring DWH database ovirt_engine_history
[ INFO  ] Restoring file '/var/lib/ovirt-engine-dwh/backups/dwh-20200713205557.de65z02w.dump' to database localhost:ovirt_engine_history.
[ INFO  ] Stage: Pre-termination
[ INFO  ] Stage: Termination
[ ERROR ] Execution of setup failed
-~-

Comment 2 Sandro Bonazzola 2020-08-05 06:25:00 UTC

This bugzilla is included in oVirt 4.4.1 release, published on July 8th 2020.

Since the problem described in this bug report should be resolved in oVirt 4.4.1 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.