Bug 1340810 - 4.0 engine-backup should hide reports backup/restore
Summary: 4.0 engine-backup should hide reports backup/restore
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: Backup-Restore.Engine
Version: 4.0.0
Hardware: Unspecified
OS: Unspecified
medium
medium vote
Target Milestone: ovirt-4.0.5
: 4.0.5
Assignee: Yedidyah Bar David
QA Contact: Lucie Leistnerova
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-05-30 10:48 UTC by Jiri Belka
Modified: 2017-01-18 07:35 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Enhancement
Doc Text:
Feature: engine-backup --mode=restore does not restore reports db/conf even if found in backup. Reason: In 4.0 Reports is not packaged/supported anymore. engine-backup of 4.0 supports restoring a 3.6 backup, which might include Reports data. Result: If Reports db dump is found in a backup, engine-backup notifies the user that it will not be restored, and does not restore Reports db/conf.
Clone Of:
Environment:
Last Closed: 2017-01-18 07:35:51 UTC
oVirt Team: Integration
rule-engine: ovirt-4.0.z+
rule-engine: exception+
rule-engine: planning_ack+
dfediuck: devel_ack+
rule-engine: testing_ack+


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
oVirt gerrit 60472 master MERGED packaging: engine-backup: Do not backup/restore Reports 2016-08-09 08:48:30 UTC
oVirt gerrit 60493 master MERGED packaging: engine-backup: Drop Reports from --help 2016-08-09 08:49:16 UTC
oVirt gerrit 62100 ovirt-engine-4.0 MERGED packaging: engine-backup: Do not backup/restore Reports 2016-08-09 08:51:22 UTC
oVirt gerrit 62101 ovirt-engine-4.0 MERGED packaging: engine-backup: Drop Reports from --help 2016-08-09 08:51:28 UTC
oVirt gerrit 62102 ovirt-engine-4.0.2 MERGED packaging: engine-backup: Do not backup/restore Reports 2016-08-09 08:51:41 UTC
oVirt gerrit 62103 ovirt-engine-4.0.2 MERGED packaging: engine-backup: Drop Reports from --help 2016-08-09 08:51:52 UTC
oVirt gerrit 62913 master MERGED packaging: engine-backup: Exclude reports files on restore 2016-09-06 16:20:49 UTC
oVirt gerrit 63450 ovirt-engine-4.0 MERGED packaging: engine-backup: Exclude reports files on restore 2016-09-07 07:47:32 UTC
oVirt gerrit 63452 ovirt-engine-4.0.4 MERGED packaging: engine-backup: Exclude reports files on restore 2016-09-07 07:49:44 UTC
oVirt gerrit 64116 master NEW packaging: engine-backup: Exclude engine reports conf on restore 2016-09-19 08:36:20 UTC
oVirt gerrit 64416 ovirt-engine-4.0 NEW packaging: engine-backup: Exclude engine reports conf on restore 2016-09-25 12:00:43 UTC

Description Jiri Belka 2016-05-30 10:48:14 UTC
Description of problem:

not sure if 4.0 engine-backup should restore 'reports' (files, db) in --scope=all.

Version-Release number of selected component (if applicable):
ovirt-engine-tools-backup-4.0.0-0.7.master.el7ev.noarch

How reproducible:
100%

Steps to Reproduce:
1. have 3.6 backup with --scope=all (incl dwh/reports)
2. engine-backup --mode=restore --scope=all --file=<backup> --log=restore.log --restore-permissions --provision-db --provision-dwh-db
3.

Actual results:
4.0 engine-backup's --scope=all restores reports files/db

Expected results:
unsure what we want here :)

Additional info:

Comment 1 Yedidyah Bar David 2016-05-30 11:47:01 UTC
(In reply to Jiri Belka from comment #0)
> Expected results:
> unsure what we want here :)

I'd say we definitely want the current default/documented flow to succeed.

Meaning the resulting restored machine will have all the files and databases, including reports, but no reports rpms (app and setup). engine-setup will finish successfully, and leave the reports database untouched.

Moving to QE. If this fails, or if something else should happen, please move back to assigned.

Comment 2 Jiri Belka 2016-05-30 14:05:41 UTC
ok, works fine, ie. 4.0.0-10 engine-backup can restore reports files/db fine.

Comment 3 Yedidyah Bar David 2016-06-20 10:28:58 UTC
Julie, this is the flow I mentioned today.

You might want to mention it somewhere, although it's not that important (as imo the behavior is intuitive).

Comment 4 Sandro Bonazzola 2016-07-05 07:38:29 UTC
oVirt 4.0.0 has been released, closing current release.

Comment 5 Yaniv Lavi 2016-07-10 12:00:35 UTC
This should not be the flow. We should ignore the reports in the restore to 4.0, since it's not included anymore. Reopening to correct.

Comment 6 Martin Perina 2016-07-21 11:29:05 UTC
(In reply to Yaniv Dary from comment #5)
> This should not be the flow. We should ignore the reports in the restore to
> 4.0, since it's not included anymore. Reopening to correct.

IMO this is not correct fix for the issue:

1. Restore tool should restore what was backed up by backup tool and it should not decide automatically to skip some content of the backup

2. This does not solve the issue for upstream, where oVirt engine 3.6 can be installed on Centos 7 or Fedora and in this case users will perform in-place upgrade

Removal of existing reports configuration should be part of engine-setup which performs upgrade from 3.6 to 4.0

Btw leaving unwanted reports configuration in 4.0 can cause unexpected issues like the one in BZ1358160.

Comment 7 Yedidyah Bar David 2016-07-31 09:56:29 UTC
Yaniv - please see Martin's comment 6. How should I continue?

Comment 8 Yaniv Lavi 2016-07-31 12:00:37 UTC
(In reply to Yedidyah Bar David from comment #7)
> Yaniv - please see Martin's comment 6. How should I continue?

We should handle removal of report in 4.0. Please use BZ1358160 to track fixing this.
The restore tool should not restore data of deprecated reports and we should not require a DB for this (there is no point of it). 3.6 tool can still restore reports if the user wants this.

Comment 9 Martin Perina 2016-08-04 06:51:12 UTC
(In reply to Yaniv Dary from comment #8)
> (In reply to Yedidyah Bar David from comment #7)
> > Yaniv - please see Martin's comment 6. How should I continue?
> 
> We should handle removal of report in 4.0.

Right, but we want to fix all scenarios (include in-place upgrade upstream), we need to do that inside engine-setup

> Please use BZ1358160 to track fixing this.

This bug is dedicated to fix reports proxy part inside webadmin UI and not removing reports configuration.

> The restore tool should not restore data of deprecated reports and we should
> not require a DB for this (there is no point of it). 3.6 tool can still
> restore reports if the user wants this.

Once again I disagree with that for following reason:

1. Restore tool should not contain any automatic logic which will decide what part of backed up data will be excluded during restore. Only user may be allowed to specify that using command line options

2. Not restoring reports configuration during restore solves the issue only for upgrade flow from "engine 3.6 on EL6" to "engine 4.0 on EL7". It does not solve in-place upgrades for engine 3.6 on Centos 7 or Fedora, those can be handled only by plugin in engine-setup -> we should not have same logic on two places, so IMO only correct solution is do that logic only in engine-setup

3. We should remove also reports configuration inside engine, so at least /etc/ovirt-engine/engine.conf.d/10-setup-reports-access.conf but not sure if there's anything else

Comment 11 Martin Perina 2016-08-04 08:28:47 UTC
BZ1363958 was created to track reports configuration removal for in-place upgrades. I still think we should handle removal of reports on once place :-)

Comment 12 Yedidyah Bar David 2016-08-09 09:11:29 UTC
Sample output:

# engine-backup --mode=restore --file=backup-fqdn-didi-centos7.eng.lab.tlv.redhat.com-3.6.7.5-with-reports --log=restore-log-2016-08-09-02 --no-restore-permissions --provision-db --provision-dwh-db
Preparing to restore:
- Unpacking file 'backup-fqdn-didi-centos7.eng.lab.tlv.redhat.com-3.6.7.5-with-reports'
------------------------------------------------------------------------------
Please note:

A Reports dump was found inside the backup.

This version does not support Reports, and the dump and configuration files
will not be restored.
------------------------------------------------------------------------------
Restoring:
- Files
------------------------------------------------------------------------------
Please note:

Operating system is different from the one used during backup.
Current operating system: centos7
Operating system at backup: centos6

Apache httpd configuration will not be restored.
You will be asked about it on the next engine-setup run.
------------------------------------------------------------------------------
Provisioning PostgreSQL users/databases:
- user 'engine', database 'engine'
- user 'ovirt_engine_history', database 'ovirt_engine_history'
Restoring:
- Engine database 'engine'
  - Cleaning up temporary tables in engine database 'engine'
  - Resetting DwhCurrentlyRunning in dwh_history_timekeeping in engine database
------------------------------------------------------------------------------
Please note:

The engine database was backed up at 2016-08-09 10:38:52.000000000 +0300 .

Objects that were added, removed or changed after this date, such as virtual
machines, disks, etc., are missing in the engine, and will probably require
recovery or recreation.
------------------------------------------------------------------------------
- DWH database 'ovirt_engine_history'
You should now run engine-setup.
Done.

Comment 13 Jiri Belka 2016-08-11 13:20:46 UTC
fail, 

after restore there are no reports files/db but reports related files are still present.

[root@livecd ~]# tar tzvf brq-setup_backup_201608110000.tar.gz 
drwxr-xr-x root/root         0 2016-08-11 00:00 ./
-rw-r--r-- root/root         8 2016-08-11 00:00 ./os_version
-rw-r--r-- root/root         8 2016-08-11 00:00 ./version
-rw-r--r-- root/root    234768 2016-08-11 00:00 ./files
-rw-r--r-- root/root       328 2016-08-11 00:00 ./md5sum
-rw-r--r-- root/root       150 2016-08-11 00:00 ./config
drwxr-xr-x root/root         0 2016-08-11 00:00 ./db/
-rw-r--r-- root/root   3412054 2016-08-11 00:00 ./db/engine_backup.db
-rw-r--r-- root/root  50103024 2016-08-11 00:00 ./db/dwh_backup.db
-rw-r--r-- root/root   7472381 2016-08-11 00:00 ./db/reports_backup.db
[root@livecd ~]# su - postgres -c "psql -c '\l'"
                                          List of databases
         Name         |     Owner      | Encoding |   Collate   |    Ctype    |   Access privileges   
----------------------+----------------+----------+-------------+-------------+-----------------------
 engine               | engine         | UTF8     | en_US.UTF-8 | en_US.UTF-8 | 
 ovirt_engine_history | engine_history | UTF8     | en_US.UTF-8 | en_US.UTF-8 | 
 postgres             | postgres       | UTF8     | en_US.UTF-8 | en_US.UTF-8 | 
 template0            | postgres       | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
                      |                |          |             |             | postgres=CTc/postgres
 template1            | postgres       | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
                      |                |          |             |             | postgres=CTc/postgres
(5 rows)

But...

# find /etc/ovirt-engine-reports/
/etc/ovirt-engine-reports/
/etc/ovirt-engine-reports/ovirt-engine-reports.conf.d
/etc/ovirt-engine-reports/ovirt-engine-reports.conf.d/10-setup-java.conf
/etc/ovirt-engine-reports/ovirt-engine-reports.conf.d/10-setup-jboss.conf.20151222130932
/etc/ovirt-engine-reports/ovirt-engine-reports.conf.d/10-setup-database.conf.20150107143414
/etc/ovirt-engine-reports/ovirt-engine-reports.conf.d/20-rhevm-reports-license.conf
/etc/ovirt-engine-reports/ovirt-engine-reports.conf.d/10-setup-jboss.conf
/etc/ovirt-engine-reports/ovirt-engine-reports.conf.d/10-setup-protocols.conf
/etc/ovirt-engine-reports/ovirt-engine-reports.conf.d/10-setup-database.conf
/etc/ovirt-engine-reports/sso.properties
/etc/ovirt-engine-reports/sso.properties.20150107143414
...

# diff -uNp /usr/share/ovirt-engine/bin/engine-backup.sh{.orig,}
--- /usr/share/ovirt-engine/bin/engine-backup.sh.orig   2016-08-11 11:07:55.870405925 +0200
+++ /usr/share/ovirt-engine/bin/engine-backup.sh        2016-08-11 11:21:18.858405925 +0200
@@ -435,7 +435,10 @@ VALID_BACKUP_RESTORE_PAIRS="3.6 4.0"
 BACKUP_VERSION=
 INSTALLED_VERSION=
 EXCLUDED_FILES_ON_RESTORE="etc/ovirt-engine/engine-config/engine-config.properties
-etc/ovirt-engine-setup.conf.d/*packaging*"
+etc/ovirt-engine-setup.conf.d/*packaging*
+etc/ovirt-engine-reports
+var/lib/ovirt-engine-reports
+etc/httpd/conf.d/z-ovirt-engine-reports-proxy.conf"
 
 source_d defaults
 
# find /etc/ovirt-engine-reports /var/lib/ovirt-engine-reports /etc/httpd/conf.d/z-ovirt-engine-reports-proxy.conf 
find: ‘/etc/ovirt-engine-reports’: No such file or directory
find: ‘/var/lib/ovirt-engine-reports’: No such file or directory
find: ‘/etc/httpd/conf.d/z-ovirt-engine-reports-proxy.conf’: No such file or directory

Comment 14 Red Hat Bugzilla Rules Engine 2016-08-11 13:20:53 UTC
Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.

Comment 15 Jiri Belka 2016-08-11 13:21:51 UTC
ovirt-engine-tools-backup-4.0.2.6-0.1.el7ev.noarch

Comment 16 Lucie Leistnerova 2016-09-16 15:18:00 UTC
after restore found one more reports related file

/etc/ovirt-engine/engine.conf.d/10-setup-reports-access.conf

in ovirt-engine-tools-backup-4.0.4.3-0.1.el7ev.noarch

Comment 17 Red Hat Bugzilla Rules Engine 2016-09-16 15:18:06 UTC
Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.

Comment 18 Yedidyah Bar David 2016-09-18 08:06:25 UTC
(In reply to Lucie Leistnerova from comment #16)
> after restore found one more reports related file
> 
> /etc/ovirt-engine/engine.conf.d/10-setup-reports-access.conf
> 

When last fix was pushed, this file was awaiting needinfo on bug 1363958.

Commented now there as well.

Comment 19 Lucie Leistnerova 2016-09-30 13:45:42 UTC
verified in ovirt-engine-tools-backup-4.0.5-0.1.el7ev.noarch


Note You need to log in before you can comment on or make changes to this bug.