Bug 1535935 - engine-setup creates bad 10-setup-database.conf if it has to provision ovirt_engine_history_TIMESTAMP
Summary: engine-setup creates bad 10-setup-database.conf if it has to provision ovirt_...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-engine-dwh
Classification: oVirt
Component: Setup
Version: 4.2.0
Hardware: Unspecified
OS: Unspecified
unspecified
urgent
Target Milestone: ovirt-4.2.1
: ---
Assignee: Yedidyah Bar David
QA Contact: Lukas Svaty
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-01-18 09:52 UTC by Yedidyah Bar David
Modified: 2018-02-12 11:48 UTC (History)
6 users (show)

Fixed In Version: ovirt-engine-dwh-4.2.1.2
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1535904
Environment:
Last Closed: 2018-02-12 11:48:52 UTC
oVirt Team: Integration
Embargoed:
rule-engine: ovirt-4.2+
rule-engine: blocker+


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1518802 0 medium CLOSED upgrade 4.1.8 to 4.2 with answer file fails on upgrade postgres issue 2021-02-22 00:41:40 UTC
oVirt gerrit 86527 0 master MERGED packaging: setup: Schedule 10-setup-database.conf 2020-09-11 13:07:42 UTC
oVirt gerrit 86541 0 master MERGED packaging: setup: Provisioning.applyEnvironment only at MISC 2020-09-11 13:07:42 UTC

Internal Links: 1518802

Description Yedidyah Bar David 2018-01-18 09:52:05 UTC
+++ This bug was initially created as a clone of Bug #1535904 +++

Description of problem:

$summary.

Version-Release number of selected component (if applicable):

Current master. Not sure what changed the order of things so that this didn't happen before.

Comment 1 Yedidyah Bar David 2018-01-18 13:45:24 UTC
To reproduce/verify:

Install clean EL7

Install ovirt-engine

/usr/share/ovirt-engine/setup/bin/ovirt-engine-provisiondb --otopi-environment="OVESETUP_PROVISION_DB/database=str:ovirt_engine_history OVESETUP_PROVISION_DB/user=str:ovirt_engine_history OVESETUP_PROVISION_DB/password=str:secret"

su - postgres -c "scl enable rh-postgresql95 \"psql ovirt_engine_history -c 'create table tab1(a int);'\""

engine-setup

Check dwhd log to see if it connected to its database

Comment 2 Red Hat Bugzilla Rules Engine 2018-01-23 08:03:28 UTC
This bug report has Keywords: Regression or TestBlocker.
Since no regressions or test blockers are allowed between releases, it is also being identified as a blocker for this release. Please resolve ASAP.

Comment 3 Yedidyah Bar David 2018-01-24 08:39:22 UTC
I didn't intend to have 86541 merged for current bug, because it has a somewhat significant impact on engine-setup, and I wanted to postpone it to 4.3. However, I didn't clarify this well-enough in gerrit, and was on PTO, and it was merged. The parallel patch for the engine was not merged yet, so the following applies only to DWH (for now). It should still be safe, so I suggest to keep it.

Patch 86541 makes it so that nothing in engine-setup knows about the db and user names we intend to create until right before actually creating them. In practice, the only place I currently know about is the code displaying to the user "--== CONFIGURATION PREVIEW ==--", then a list of items, then asking "Please confirm installation settings (OK, Cancel) [OK]:". However, I might be wrong, so:

QE: Please verify various relevant flows with the version that fixes this bug, including e.g.:

- Clean engine+dwh setup, both automatic and manual db provisioning
- Clean engine-only setup, then 'engine-setup --reconfigure-optional-components' and choose to provision dwh - both automatic and manual
- engine and dwh on separate machines

etc.

Comment 4 Lukas Svaty 2018-01-27 17:37:57 UTC
Hi, didi can you confirm/align these verification steps?

1. Fresh rhel7
2. Install ovirt-engine with default db names
3. /usr/share/ovirt-engine/setup/bin/ovirt-engine-provisiondb --otopi-environment="OVESETUP_PROVISION_DB/database=str:ovirt_engine_history OVESETUP_PROVISION_DB/user=str:ovirt_engine_history OVESETUP_PROVISION_DB/password=str:PWD"

Output:
[ INFO  ] Stage: Initializing
[ INFO  ] Stage: Environment setup
          Configuration files: ['/etc/ovirt-engine-setup.conf.d/10-packaging-wsp.conf', '/etc/ovirt-engine-setup.conf.d/10-packaging.conf', '/etc/ovirt-engine-setup.conf.d/20-setup-ovirt-post.conf']
          Log file: /var/log/ovirt-engine/setup/ovirt-engine-provisiondb-20180127183412-x81hzq.log
          Version: otopi-1.7.6 (otopi-1.7.6-1.el7ev)
[ INFO  ] Stage: Environment packages setup
[ INFO  ] Stage: Programs detection
[ INFO  ] Stage: Environment customization
[ INFO  ] Stage: Setup validation
[ INFO  ] Stage: Transaction setup
[ INFO  ] Stage: Misc configuration
[ INFO  ] Stage: Package installation
[ INFO  ] Stage: Misc configuration
[ INFO  ] Creating PostgreSQL 'ovirt_engine_history' database
[ INFO  ] Configuring PostgreSQL
[ ERROR ] Failed to execute stage 'Misc configuration': Existing resources found, new ones created: 
         database ovirt_engine_history_20180127183414 user ovirt_engine_history_20180127183414
[ INFO  ] Stage: Clean up
          Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-provisiondb-20180127183412-x81hzq.log
[ INFO  ] Stage: Pre-termination
[ INFO  ] Stage: Termination
[ ERROR ] Execution of provisiondb failed

Traceback from log:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/otopi/context.py", line 133, in _executeMethod
    method['method']()
  File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-provisiondb/ovirt-engine-common/provisioning/database.py", line 112, in _misc
    oprovisioncons.ProvDBEnv.USER
RuntimeError: Existing resources found, new ones created: 
database ovirt_engine_history_20180127183414 user ovirt_engine_history_20180127183414
2018-01-27 18:34:17,974+0100 ERROR otopi.context context._executeMethod:152 Failed to execute stage 'Misc configuration': Existing resources found, new ones created: 
database ovirt_engine_history_20180127183414 user ovirt_engine_history_20180127183414


I believe the script should finish successfully.


I tried creation of dwh_db_TIMESTAMP with engine-setup which updates all the config files successfully.

Comment 5 Yedidyah Bar David 2018-01-28 07:11:42 UTC
(In reply to Lukas Svaty from comment #4)
> Hi, didi can you confirm/align these verification steps?
> 
> 1. Fresh rhel7

OK

> 2. Install ovirt-engine with default db names

Let me clarify: Here, and IIRC almost everywhere, when I say "Install engine" I mean "yum install ovirt-engine", but do not setup. Setup is either a separate step, or I write "Install and Setup" (or just "Setup", implying install first).

> 3. /usr/share/ovirt-engine/setup/bin/ovirt-engine-provisiondb
> --otopi-environment="OVESETUP_PROVISION_DB/database=str:ovirt_engine_history
> OVESETUP_PROVISION_DB/user=str:ovirt_engine_history
> OVESETUP_PROVISION_DB/password=str:PWD"

So you ran this command after setup finished, and requested it to create same user/db that setup already created.

> 
> Output:
> [ INFO  ] Stage: Initializing
> [ INFO  ] Stage: Environment setup
>           Configuration files:
> ['/etc/ovirt-engine-setup.conf.d/10-packaging-wsp.conf',
> '/etc/ovirt-engine-setup.conf.d/10-packaging.conf',
> '/etc/ovirt-engine-setup.conf.d/20-setup-ovirt-post.conf']
>           Log file:
> /var/log/ovirt-engine/setup/ovirt-engine-provisiondb-20180127183412-x81hzq.
> log
>           Version: otopi-1.7.6 (otopi-1.7.6-1.el7ev)
> [ INFO  ] Stage: Environment packages setup
> [ INFO  ] Stage: Programs detection
> [ INFO  ] Stage: Environment customization
> [ INFO  ] Stage: Setup validation
> [ INFO  ] Stage: Transaction setup
> [ INFO  ] Stage: Misc configuration
> [ INFO  ] Stage: Package installation
> [ INFO  ] Stage: Misc configuration
> [ INFO  ] Creating PostgreSQL 'ovirt_engine_history' database
> [ INFO  ] Configuring PostgreSQL
> [ ERROR ] Failed to execute stage 'Misc configuration': Existing resources
> found, new ones created: 
>          database ovirt_engine_history_20180127183414 user
> ovirt_engine_history_20180127183414
> [ INFO  ] Stage: Clean up
>           Log file is located at
> /var/log/ovirt-engine/setup/ovirt-engine-provisiondb-20180127183412-x81hzq.
> log
> [ INFO  ] Stage: Pre-termination
> [ INFO  ] Stage: Termination
> [ ERROR ] Execution of provisiondb failed
> 
> Traceback from log:
> Traceback (most recent call last):
>   File "/usr/lib/python2.7/site-packages/otopi/context.py", line 133, in
> _executeMethod
>     method['method']()
>   File
> "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-provisiondb/ovirt-
> engine-common/provisioning/database.py", line 112, in _misc
>     oprovisioncons.ProvDBEnv.USER
> RuntimeError: Existing resources found, new ones created: 
> database ovirt_engine_history_20180127183414 user
> ovirt_engine_history_20180127183414
> 2018-01-27 18:34:17,974+0100 ERROR otopi.context context._executeMethod:152
> Failed to execute stage 'Misc configuration': Existing resources found, new
> ones created: 
> database ovirt_engine_history_20180127183414 user
> ovirt_engine_history_20180127183414
> 
> 
> I believe the script should finish successfully.

You are welcome to open a bug about this, if you want. That's not current bug. Also, I disagree, and I made it fail deliberately, unlike engine-setup which does not fail and uses the _TIMESTAMP names.

provisiondb was meant to be used mostly for 'engine-backup --mode=restore --provision*db', and I felt it's too risky to let it continue with the new names, even if I could make it work in the simple flow. Restore should normally be done to a clean and nice system, and if you need anything else, you should not use --provision*db (and probably do use --change-*db-credentials, after creating db/user as you wish).

> 
> 
> I tried creation of dwh_db_TIMESTAMP with engine-setup which updates all the
> config files successfully.

Please try again the flow in Comment 1, where 'Install' is just 'yum install'. Thanks. Also, the provisiondb command was just suggested as a simple means to create a db/user, you are welcome to do that yourself manually if you want - that's not part of current bug.

Comment 6 Lukas Svaty 2018-01-29 11:44:15 UTC
Thank you for explaining, I agree with the failure, as it states the issue pretty clearly.

tested in: ovirt-engine-dwh-4.2.1.2-1.el7ev.noarch
DWH is running, database conf is correctly set up.

Comment 7 Sandro Bonazzola 2018-02-12 11:48:52 UTC
This bugzilla is included in oVirt 4.2.1 release, published on Feb 12th 2018.

Since the problem described in this bug report should be
resolved in oVirt 4.2.1 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.