Bug 1340810

(In reply to Jiri Belka from comment #0)
> Expected results:
> unsure what we want here :)

I'd say we definitely want the current default/documented flow to succeed.

Meaning the resulting restored machine will have all the files and databases, including reports, but no reports rpms (app and setup). engine-setup will finish successfully, and leave the reports database untouched.

Moving to QE. If this fails, or if something else should happen, please move back to assigned.

ok, works fine, ie. 4.0.0-10 engine-backup can restore reports files/db fine.

Julie, this is the flow I mentioned today.

You might want to mention it somewhere, although it's not that important (as imo the behavior is intuitive).

oVirt 4.0.0 has been released, closing current release.

This should not be the flow. We should ignore the reports in the restore to 4.0, since it's not included anymore. Reopening to correct.

(In reply to Yaniv Dary from comment #5)
> This should not be the flow. We should ignore the reports in the restore to
> 4.0, since it's not included anymore. Reopening to correct.

IMO this is not correct fix for the issue:

1. Restore tool should restore what was backed up by backup tool and it should not decide automatically to skip some content of the backup

2. This does not solve the issue for upstream, where oVirt engine 3.6 can be installed on Centos 7 or Fedora and in this case users will perform in-place upgrade

Removal of existing reports configuration should be part of engine-setup which performs upgrade from 3.6 to 4.0

Btw leaving unwanted reports configuration in 4.0 can cause unexpected issues like the one in BZ1358160.

Yaniv - please see Martin's comment 6. How should I continue?

(In reply to Yedidyah Bar David from comment #7)
> Yaniv - please see Martin's comment 6. How should I continue?

We should handle removal of report in 4.0. Please use BZ1358160 to track fixing this.
The restore tool should not restore data of deprecated reports and we should not require a DB for this (there is no point of it). 3.6 tool can still restore reports if the user wants this.

(In reply to Yaniv Dary from comment #8)
> (In reply to Yedidyah Bar David from comment #7)
> > Yaniv - please see Martin's comment 6. How should I continue?
> 
> We should handle removal of report in 4.0.

Right, but we want to fix all scenarios (include in-place upgrade upstream), we need to do that inside engine-setup

> Please use BZ1358160 to track fixing this.

This bug is dedicated to fix reports proxy part inside webadmin UI and not removing reports configuration.

> The restore tool should not restore data of deprecated reports and we should
> not require a DB for this (there is no point of it). 3.6 tool can still
> restore reports if the user wants this.

Once again I disagree with that for following reason:

1. Restore tool should not contain any automatic logic which will decide what part of backed up data will be excluded during restore. Only user may be allowed to specify that using command line options

2. Not restoring reports configuration during restore solves the issue only for upgrade flow from "engine 3.6 on EL6" to "engine 4.0 on EL7". It does not solve in-place upgrades for engine 3.6 on Centos 7 or Fedora, those can be handled only by plugin in engine-setup -> we should not have same logic on two places, so IMO only correct solution is do that logic only in engine-setup

3. We should remove also reports configuration inside engine, so at least /etc/ovirt-engine/engine.conf.d/10-setup-reports-access.conf but not sure if there's anything else

BZ1363958 was created to track reports configuration removal for in-place upgrades. I still think we should handle removal of reports on once place :-)

Sample output:

# engine-backup --mode=restore --file=backup-fqdn-didi-centos7.eng.lab.tlv.redhat.com-3.6.7.5-with-reports --log=restore-log-2016-08-09-02 --no-restore-permissions --provision-db --provision-dwh-db
Preparing to restore:
- Unpacking file 'backup-fqdn-didi-centos7.eng.lab.tlv.redhat.com-3.6.7.5-with-reports'
------------------------------------------------------------------------------
Please note:

A Reports dump was found inside the backup.

This version does not support Reports, and the dump and configuration files
will not be restored.
------------------------------------------------------------------------------
Restoring:
- Files
------------------------------------------------------------------------------
Please note:

Operating system is different from the one used during backup.
Current operating system: centos7
Operating system at backup: centos6

Apache httpd configuration will not be restored.
You will be asked about it on the next engine-setup run.
------------------------------------------------------------------------------
Provisioning PostgreSQL users/databases:
- user 'engine', database 'engine'
- user 'ovirt_engine_history', database 'ovirt_engine_history'
Restoring:
- Engine database 'engine'
  - Cleaning up temporary tables in engine database 'engine'
  - Resetting DwhCurrentlyRunning in dwh_history_timekeeping in engine database
------------------------------------------------------------------------------
Please note:

The engine database was backed up at 2016-08-09 10:38:52.000000000 +0300 .

Objects that were added, removed or changed after this date, such as virtual
machines, disks, etc., are missing in the engine, and will probably require
recovery or recreation.
------------------------------------------------------------------------------
- DWH database 'ovirt_engine_history'
You should now run engine-setup.
Done.

fail, 

after restore there are no reports files/db but reports related files are still present.

[root@livecd ~]# tar tzvf brq-setup_backup_201608110000.tar.gz 
drwxr-xr-x root/root         0 2016-08-11 00:00 ./
-rw-r--r-- root/root         8 2016-08-11 00:00 ./os_version
-rw-r--r-- root/root         8 2016-08-11 00:00 ./version
-rw-r--r-- root/root    234768 2016-08-11 00:00 ./files
-rw-r--r-- root/root       328 2016-08-11 00:00 ./md5sum
-rw-r--r-- root/root       150 2016-08-11 00:00 ./config
drwxr-xr-x root/root         0 2016-08-11 00:00 ./db/
-rw-r--r-- root/root   3412054 2016-08-11 00:00 ./db/engine_backup.db
-rw-r--r-- root/root  50103024 2016-08-11 00:00 ./db/dwh_backup.db
-rw-r--r-- root/root   7472381 2016-08-11 00:00 ./db/reports_backup.db
[root@livecd ~]# su - postgres -c "psql -c '\l'"
                                          List of databases
         Name         |     Owner      | Encoding |   Collate   |    Ctype    |   Access privileges   
----------------------+----------------+----------+-------------+-------------+-----------------------
 engine               | engine         | UTF8     | en_US.UTF-8 | en_US.UTF-8 | 
 ovirt_engine_history | engine_history | UTF8     | en_US.UTF-8 | en_US.UTF-8 | 
 postgres             | postgres       | UTF8     | en_US.UTF-8 | en_US.UTF-8 | 
 template0            | postgres       | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
                      |                |          |             |             | postgres=CTc/postgres
 template1            | postgres       | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
                      |                |          |             |             | postgres=CTc/postgres
(5 rows)

But...

# find /etc/ovirt-engine-reports/
/etc/ovirt-engine-reports/
/etc/ovirt-engine-reports/ovirt-engine-reports.conf.d
/etc/ovirt-engine-reports/ovirt-engine-reports.conf.d/10-setup-java.conf
/etc/ovirt-engine-reports/ovirt-engine-reports.conf.d/10-setup-jboss.conf.20151222130932
/etc/ovirt-engine-reports/ovirt-engine-reports.conf.d/10-setup-database.conf.20150107143414
/etc/ovirt-engine-reports/ovirt-engine-reports.conf.d/20-rhevm-reports-license.conf
/etc/ovirt-engine-reports/ovirt-engine-reports.conf.d/10-setup-jboss.conf
/etc/ovirt-engine-reports/ovirt-engine-reports.conf.d/10-setup-protocols.conf
/etc/ovirt-engine-reports/ovirt-engine-reports.conf.d/10-setup-database.conf
/etc/ovirt-engine-reports/sso.properties
/etc/ovirt-engine-reports/sso.properties.20150107143414
...

# diff -uNp /usr/share/ovirt-engine/bin/engine-backup.sh{.orig,}
--- /usr/share/ovirt-engine/bin/engine-backup.sh.orig   2016-08-11 11:07:55.870405925 +0200
+++ /usr/share/ovirt-engine/bin/engine-backup.sh        2016-08-11 11:21:18.858405925 +0200
@@ -435,7 +435,10 @@ VALID_BACKUP_RESTORE_PAIRS="3.6 4.0"
 BACKUP_VERSION=
 INSTALLED_VERSION=
 EXCLUDED_FILES_ON_RESTORE="etc/ovirt-engine/engine-config/engine-config.properties
-etc/ovirt-engine-setup.conf.d/*packaging*"
+etc/ovirt-engine-setup.conf.d/*packaging*
+etc/ovirt-engine-reports
+var/lib/ovirt-engine-reports
+etc/httpd/conf.d/z-ovirt-engine-reports-proxy.conf"
 
 source_d defaults
 
# find /etc/ovirt-engine-reports /var/lib/ovirt-engine-reports /etc/httpd/conf.d/z-ovirt-engine-reports-proxy.conf 
find: ‘/etc/ovirt-engine-reports’: No such file or directory
find: ‘/var/lib/ovirt-engine-reports’: No such file or directory
find: ‘/etc/httpd/conf.d/z-ovirt-engine-reports-proxy.conf’: No such file or directory

Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.

ovirt-engine-tools-backup-4.0.2.6-0.1.el7ev.noarch

after restore found one more reports related file

/etc/ovirt-engine/engine.conf.d/10-setup-reports-access.conf

in ovirt-engine-tools-backup-4.0.4.3-0.1.el7ev.noarch

Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.

(In reply to Lucie Leistnerova from comment #16)
> after restore found one more reports related file
> 
> /etc/ovirt-engine/engine.conf.d/10-setup-reports-access.conf
> 

When last fix was pushed, this file was awaiting needinfo on bug 1363958.

Commented now there as well.

verified in ovirt-engine-tools-backup-4.0.5-0.1.el7ev.noarch