Bug 1369757 - engine-backup --mode=backup --restore-permissions should create the missing extra users if possible
Summary: engine-backup --mode=backup --restore-permissions should create the missing e...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: Backup-Restore.Engine
Version: 4.0.3
Hardware: Unspecified
OS: Unspecified
unspecified
high vote
Target Milestone: ovirt-4.0.4
: 4.0.4.4
Assignee: Yedidyah Bar David
QA Contact: Lukas Svaty
URL:
Whiteboard:
Depends On:
Blocks: 1368589 1373320
TreeView+ depends on / blocked
 
Reported: 2016-08-24 10:46 UTC by Sandro Bonazzola
Modified: 2016-09-26 12:32 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: engine-backup --mode=restore --restore-permissions fails if there are grants for extra users, even if passing --provision-*db. Consequence: Fix: Now the above command also creates users for the users given extra grants in the backup, with random passwords. Result: Restore no longer fails. The admin still has to change the passwords if the extra users still need access on the restored system.
Clone Of:
: 1373320 (view as bug list)
Environment:
Last Closed: 2016-09-26 12:32:02 UTC
oVirt Team: Integration
rule-engine: ovirt-4.0.z+
ylavi: exception+
mgoldboi: planning_ack+
sbonazzo: devel_ack+
pstehlik: testing_ack+


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 62823 0 'None' MERGED packaging: setup: Allow provisioning postgresql role 2020-09-08 07:53:56 UTC
oVirt gerrit 62844 0 'None' MERGED packaging: engine-backup: Create extra users on provisioning 2020-09-08 07:53:56 UTC
oVirt gerrit 62849 0 'None' MERGED packaging: setup: Allow provisioning postgresql role 2020-09-08 07:53:56 UTC
oVirt gerrit 62850 0 'None' MERGED packaging: engine-backup: Create extra users on provisioning 2020-09-08 07:53:55 UTC
oVirt gerrit 62886 0 'None' MERGED packaging: engine-backup: clarify random password 2020-09-08 07:53:55 UTC
oVirt gerrit 62915 0 'None' MERGED packaging: engine-backup: clarify random password 2020-09-08 07:53:55 UTC
oVirt gerrit 62991 0 'None' MERGED packaging: engine-backup: clarify random password 2020-09-08 07:53:55 UTC
oVirt gerrit 64133 0 'None' MERGED packaging: setup: postgres applyEnvironment: Allow missing keys 2020-09-08 07:53:55 UTC
oVirt gerrit 64134 0 'None' MERGED packaging: setup: postgres applyEnvironment: Allow missing keys 2020-09-08 07:53:55 UTC

Description Sandro Bonazzola 2016-08-24 10:46:35 UTC
Description of problem:
Bug #1368589 requires some change in backup/ restore too.
Addressing them here.

Comment 1 Yedidyah Bar David 2016-08-25 14:54:17 UTC
Changes in behavior of engine-backup --mode=restore with the currently pending changes:

Per db dump, if the dump contains GRANT statements for extra users (meaning, other than the owner, "postgres" and "PUBLIC"), and if --provision-*db and --restore-permissions are passed, and db host is localhost (not remote), then these extra users are created (if they do not already exist), and their passwords are set to random ones.

Later on, when actually restoring the dump, the extra GRANT statements will succeed.

Reproduction/Verification flow:

1. setup engine+dwh, accept automatic provisioning.
2. Add extra users and grants, e.g. using [1](3.6) or [2](4.0).
3. backup using 'engine-backup --mode=backup --file=file1 --log=log1'
4. Install engine+dwh on a new clean machine and copy there file1
5. on the new machine: engine-backup --mode=restore --file=file1 --log=restorelog1 --provision-db --provision-dwh-db --restore-permissions
6. engine-setup

[1] https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Virtualization/3.6/html/Reports_and_Data_Warehouse_Guide/sect-History_Database.html#Allowing_Read_Only_Access_to_the_History_Database
[2] https://access.redhat.com/documentation/en/red-hat-virtualization/4.0/paged/data-warehouse-guide/22-history-database#Allowing_Read_Only_Access_to_the_History_Database

Step (4.) can be partially simulated using the same machine by doing:
A. engine-cleanup
B. service postgresql stop
C. rm -rf /var/lib/pgsql/data

In reproduction (with a bad version during restore), step (5.) will fail while running grant commands. In verification (with a good version during restore), all will succeed.

backup mode was not changed. In particular:
1. The above works well also when restoring on 4.0 a backup taken on 3.6.
2. Passwords for the extra users are not saved inside the backup, thus can't be restored, so are set to random ones.

Also note:

The fix for this bug is not enough to allow access using the extra users. It's only enough so that restore works. To re-allow access, these steps are needed too:

1. edit /var/lib/pgsql/data/pg_hba.conf :
Find the lines containing 'ovirt_engine_history', there should be two such lines
duplicate them
in the duplicates, replace the second occurrence of ovirt_engine_history with the extra user.
E.g. if the extra user is 'rouser', do this:

sed -i '/ovirt_engine_history *ovirt_engine_history/p; s/^\([^ ]* *ovirt_engine_history *\)ovirt_engine_history/\1rouser/' /var/lib/pgsql/data/pg_hba.conf

2. Grant the user to connect, e.g.:

su - postgres -c 'psql -U postgres -c "GRANT CONNECT ON DATABASE ovirt_engine_history TO rouser;"'

3. change its password, e.g.:

su - postgres -c 'psql -U postgres -c "ALTER role rouser with login encrypted password '\''mypassword'\'';"'

4. service postgresql restart

Then this should work:

psql -h localhost -U rouser ovirt_engine_history
(input mypassword in the prompt)
ovirt_engine_history=> select * from schema_version;

Comment 2 Yaniv Lavi 2016-08-26 15:00:25 UTC
(In reply to Yedidyah Bar David from comment #1)
> 2. Passwords for the extra users are not saved inside the backup, thus can't
> be restored, so are set to random ones.

Does the restore say that?

> 
> Also note:
> 
> The fix for this bug is not enough to allow access using the extra users.
> It's only enough so that restore works. To re-allow access, these steps are
> needed too:
> 
> 1. edit /var/lib/pgsql/data/pg_hba.conf :
> Find the lines containing 'ovirt_engine_history', there should be two such
> lines
> duplicate them
> in the duplicates, replace the second occurrence of ovirt_engine_history
> with the extra user.
> E.g. if the extra user is 'rouser', do this:
> 
> sed -i '/ovirt_engine_history *ovirt_engine_history/p; s/^\([^ ]*
> *ovirt_engine_history *\)ovirt_engine_history/\1rouser/'
> /var/lib/pgsql/data/pg_hba.conf
> 
> 2. Grant the user to connect, e.g.:
> 
> su - postgres -c 'psql -U postgres -c "GRANT CONNECT ON DATABASE
> ovirt_engine_history TO rouser;"'
> 
> 3. change its password, e.g.:
> 
> su - postgres -c 'psql -U postgres -c "ALTER role rouser with login
> encrypted password '\''mypassword'\'';"'
> 
> 4. service postgresql restart
> 
> Then this should work:
> 
> psql -h localhost -U rouser ovirt_engine_history
> (input mypassword in the prompt)
> ovirt_engine_history=> select * from schema_version;

Did you link to any wiki saying you need to do these steps?

Comment 3 Yedidyah Bar David 2016-08-28 09:10:48 UTC
(In reply to Yaniv Dary from comment #2)
> (In reply to Yedidyah Bar David from comment #1)
> > 2. Passwords for the extra users are not saved inside the backup, thus can't
> > be restored, so are set to random ones.
> 
> Does the restore say that?

No, moving back to POST for this.

If you want the passwords to be saved too, so that they can be restored, please open another bug (IMO).

> 
> > 
> > Also note:
> > 
> > The fix for this bug is not enough to allow access using the extra users.
> > It's only enough so that restore works. To re-allow access, these steps are
> > needed too:
> > 
> > 1. edit /var/lib/pgsql/data/pg_hba.conf :
> > Find the lines containing 'ovirt_engine_history', there should be two such
> > lines
> > duplicate them
> > in the duplicates, replace the second occurrence of ovirt_engine_history
> > with the extra user.
> > E.g. if the extra user is 'rouser', do this:
> > 
> > sed -i '/ovirt_engine_history *ovirt_engine_history/p; s/^\([^ ]*
> > *ovirt_engine_history *\)ovirt_engine_history/\1rouser/'
> > /var/lib/pgsql/data/pg_hba.conf
> > 
> > 2. Grant the user to connect, e.g.:
> > 
> > su - postgres -c 'psql -U postgres -c "GRANT CONNECT ON DATABASE
> > ovirt_engine_history TO rouser;"'
> > 
> > 3. change its password, e.g.:
> > 
> > su - postgres -c 'psql -U postgres -c "ALTER role rouser with login
> > encrypted password '\''mypassword'\'';"'
> > 
> > 4. service postgresql restart
> > 
> > Then this should work:
> > 
> > psql -h localhost -U rouser ovirt_engine_history
> > (input mypassword in the prompt)
> > ovirt_engine_history=> select * from schema_version;
> 
> Did you link to any wiki saying you need to do these steps?

No, this bz comment is the only place I wrote it, following Sandro's request.

I do not see much point in proper documentation of a change that is still pending review...

Also, I thought there might still be a slight chance of getting this into 4.0.3, thus tried to do things quickly. If it's for sure 4.0.4, we have a bit more time. I'll add you as reviewer to the patch.

Comment 4 Yedidyah Bar David 2016-08-28 12:20:20 UTC
Prepared a patch for the site [1], will get it merged after all patches are merged.

[1] https://github.com/didib/ovirt-site/commits/engine-backup-restore-extra-users

Comment 5 Yedidyah Bar David 2016-08-29 07:38:39 UTC
Please review https://gerrit.ovirt.org/62886 . Thanks.

Comment 6 Yedidyah Bar David 2016-08-29 07:45:23 UTC
Sandro already merged it. We can push another patch if you want. Cherry-picked to 4.0 for now.

Comment 7 Yedidyah Bar David 2016-09-01 06:56:19 UTC
(In reply to Yedidyah Bar David from comment #4)
> Prepared a patch for the site [1], will get it merged after all patches are
> merged.
> 
> [1]
> https://github.com/didib/ovirt-site/commits/engine-backup-restore-extra-users

Setting needinfo on myself to remember to merge above once 4.0.4 it released.

Comment 8 Lukas Svaty 2016-09-16 09:20:00 UTC
Failed in ovirt-engine-4.0.4.3-0.1.el7ev.noarch

Preparing to restore:
- Unpacking file 'file1'
Restoring:
- Files
Provisioning PostgreSQL users/databases:
- user 'engine', database 'engine'
- user 'ovirt_engine_history', database 'ovirt_engine_history'
- extra user 'history_role' having grants on database ovirt_engine_history, created with a random password
FATAL: Creating user 'history_role' failed, please check the log for details

Log:
...
[ INFO  ] Stage: Closing up
[ INFO  ] Stage: Clean up
          Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-provisiondb-20160916071122-okv138.log
[ INFO  ] Stage: Pre-termination
[ INFO  ] Stage: Termination
[ INFO  ] Execution of provisiondb completed successfully
2016-09-16 07:11:27 7225: OUTPUT: - extra user 'history_role' having grants on database ovirt_engine_history, created with a random password
[ INFO  ] Stage: Initializing
[ INFO  ] Stage: Environment setup
          Configuration files: ['/etc/ovirt-engine-setup.conf.d/10-packaging-wsp.conf', '/etc/ovirt-engine-setup.conf.d/10-packaging.conf', '/etc/ovirt-engine-setup.conf.d/20-setup-ovirt-post.conf']
          Log file: /var/log/ovirt-engine/setup/ovirt-engine-provisiondb-20160916071127-946zmj.log
          Version: otopi-1.5.2 (otopi-1.5.2-1.el7ev)
[ INFO  ] Stage: Environment packages setup
[ INFO  ] Stage: Programs detection
[ INFO  ] Stage: Environment customization
[ ERROR ] Failed to execute stage 'Environment customization': 'OVESETUP_PROVISION_DB/database'
[ INFO  ] Stage: Clean up
          Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-provisiondb-20160916071127-946zmj.log
[ INFO  ] Stage: Pre-termination
[ INFO  ] Stage: Termination
[ ERROR ] Execution of provisiondb failed
2016-09-16 07:11:28 7225: FATAL: Creating user 'history_role' failed, please check the log for details

Comment 9 Red Hat Bugzilla Rules Engine 2016-09-16 09:20:06 UTC
Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.

Comment 10 Yedidyah Bar David 2016-09-18 07:25:45 UTC
(In reply to Lukas Svaty from comment #8)
> Failed in ovirt-engine-4.0.4.3-0.1.el7ev.noarch

rpm -q ovirt-engine-setup-plugin-ovirt-engine-common ?

Should be included in latest 4.0.4 build of it, wasn't in 4.0.3.

Comment 11 Lukas Svaty 2016-09-19 13:09:11 UTC
Indeed it was checked on latest 4.0.4.
Leaving on ASSIGNED as the problem is still present.
Discussed this within direct messaging channels.

Comment 12 Yedidyah Bar David 2016-09-19 13:12:26 UTC
(In reply to Yedidyah Bar David from comment #10)
> (In reply to Lukas Svaty from comment #8)
> > Failed in ovirt-engine-4.0.4.3-0.1.el7ev.noarch
> 
> rpm -q ovirt-engine-setup-plugin-ovirt-engine-common ?
> 
> Should be included in latest 4.0.4 build of it, wasn't in 4.0.3.

To clarify:

Patch [1] was pushed to engine master branch quite many months ago,
without relation to current bug.

Previous patches for current bug were only tested on master builds, not 4.0 ones.

Now cherry-picked [1] to 4.0 and 4.0.4 branches.

[1] https://gerrit.ovirt.org/57967

Comment 13 Lukas Svaty 2016-09-21 08:00:37 UTC
verified in ovirt-engine-4.0.4.4-0.1.el7ev.noarch


Note You need to log in before you can comment on or make changes to this bug.