Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1848877

Summary:	engine-backup --mode=verify is broken [RHV clone - 4.3.11]
Product:	Red Hat Enterprise Virtualization Manager	Reporter:	RHV bug bot <rhv-bugzilla-bot>
Component:	ovirt-engine	Assignee:	Yedidyah Bar David <didi>
Status:	CLOSED ERRATA	QA Contact:	Guilherme Santos <gdeolive>
Severity:	high	Docs Contact:
Priority:	high
Version:	unspecified	CC:	bugs, didi, lleistne, mtessun, rdlugyhe
Target Milestone:	ovirt-4.3.11	Keywords:	ZStream
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	ovirt-engine-4.3.10.2	Doc Type:	Bug Fix
Doc Text:	In previous versions, `engine-backup --mode=verify` passed even if `pg_restore` emitted errors. The current release fixes this issue. The `engine-backup --mode=verify` command correctly fails if `pg_restore` emits errors.	Story Points:	---
Clone Of:	1832905	Environment:
Last Closed:	2020-09-30 10:07:13 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	Integration	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1565523, 1832905, 1868929
Bug Blocks:

Description RHV bug bot 2020-06-19 06:45:26 UTC

+++ This bug is an upstream to downstream clone. The original bug is: +++
+++   bug 1832905 +++
======================================================================

Description of problem:

engine-backup --mode=verify:

1. Does not pass '-f -' to pg_restore, which makes it fail, because in recent versions it does not default to stdout anymore.

2. Despite (1.), engine-backup does not fail or log this.

Version-Release number of selected component (if applicable):
4.4

How reproducible:
Always

Steps to Reproduce:
1. engine-backup --file=f1
2. engine-backup --mode=verify --file=f1
3.

Actual results:
Passes, although pg_restore failed

Expected results:
Should pass, and if pg_restore still fails should abort and log the errors

Additional info:

I already pushed two separate patches for this to gerrit, then decided to also open this bug, and consider backporting to 4.3, mainly for (1.) - so that if pg_restore fails we notice.

Verifying this with each patch separately was easy, but for QE, who gets a single fixed build, it's a bit harder, as it should not normally fail. If you want negative testing, you'll have to do something like:
1. engine-backup --file=f1
2. mkdir f1.d
3. cd f1.d
4. tar xpf ../f1
5. Somehow cause damage to db/engine_backup.db - e.g.:
dd if=/dev/zero of=db/engine_backup.db bs=100k seek=1 count=0
6. tar cvzf ../f2 .
7. cd ..
8. engine-backup --mode=verify --file=f2

(Originally by didi)

Comment 1 RHV bug bot 2020-06-19 06:45:28 UTC

Sorry, found another place calling pg_restore.

(Originally by didi)

Comment 2 RHV bug bot 2020-06-19 06:45:31 UTC

Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.

(Originally by pm-rhel)

Comment 3 RHV bug bot 2020-06-19 06:45:32 UTC

$ git tag --contains 0b28b1dead18425574957ce40892947a16f32f4b
ovirt-engine-4.3.10.2
ovirt-engine-4.3.10.3
$ git tag --contains 2dba99c0977a9cc26e10d1ddf3357d799bc70a2b
ovirt-engine-4.3.10.2
ovirt-engine-4.3.10.3

(Originally by Sandro Bonazzola)

Comment 4 RHV bug bot 2020-06-19 06:45:34 UTC

https://gerrit.ovirt.org/#/c/109138/  was not backported to 4.3, moving out to 4.3.11

(Originally by Sandro Bonazzola)

Comment 5 RHV bug bot 2020-06-19 06:45:36 UTC

Wrote doc text which applies only for 4.3. In 4.4, also the passing of '-f -' is mandatory. For completeness, and in case we might want to use later PostgreSQL versions also in 4.3, I backported these patches as well.

(Originally by didi)

Comment 7 Guilherme Santos 2020-07-23 14:35:19 UTC

Verified on:
ovirt-engine-4.3.11.2-0.1.el7.noarch

Steps:
1. the ones described on description for QE
2. engine-backup --mode=verify --file=f1

Results
It fails for f2, aborting the process and logging the errors: 
# cat /var/log/ovirt-engine-backup/ovirt-engine-verify-20200723173325.log
2020-07-23 17:33:25 47495: Start of engine-backup mode verify scope all file f2
2020-07-23 17:33:25 47495: OUTPUT: Start of engine-backup with mode 'verify'
2020-07-23 17:33:25 47495: OUTPUT: scope: all
2020-07-23 17:33:25 47495: OUTPUT: archive file: f2
2020-07-23 17:33:25 47495: OUTPUT: log file: /var/log/ovirt-engine-backup/ovirt-engine-verify-20200723173325.log
2020-07-23 17:33:25 47495: Setting scl env for rh-postgresql10
2020-07-23 17:33:26 47495: OUTPUT: Verifying:
2020-07-23 17:33:26 47495: OUTPUT: - Unpacking file 'f2'
2020-07-23 17:33:26 47495: Opening tarball f2 to /tmp/engine-backup.H5aPSmBlCL
2020-07-23 17:33:26 47495: Verifying hash
2020-07-23 17:33:26 47495: FATAL: Checksum verification failed

But it succeeds for f1

Comment 12 errata-xmlrpc 2020-09-30 10:07:13 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat Virtualization Engine security, bug fix 4.3.11), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:4112