engine-setup currently agrees to use a non-empty remote database. This can cause problems and should be avoided. A simple example - if a future version adds a new table which already existed in this database. engine-backup --mode=restore already does that. setup should do the same.
Setting target release to current version for consideration and review. please do not push non-RFE bugs to an undefined target release to make sure bugs are reviewed for relevancy, fix, closure, etc.
Sometimes we saw cases where customer actually did restore database before setup of the product after the crash. If they were forced to do the restore after the setup it could cause i.e. overwriting stored procedures. It seems like refusing to use such DB will just set us to fragile conditions of restoring something that should not be restored.
(In reply to Tomas Dosek from comment #2) > Sometimes we saw cases where customer actually did restore database before > setup of the product after the crash. > > If they were forced to do the restore after the setup it could cause i.e. > overwriting stored procedures. It seems like refusing to use such DB will > just set us to fragile conditions of restoring something that should not be > restored. I do not follow this statement... we have the following cases: 1. fresh product setup - database must be empty (or created by setup as empty). 2. upgrade product - database must be valid per previous installation. 3. product database (only) restore - database must be empty as backup file contains the entire database. 4. product restore (complete) - database must be empty as backup file contains the entire database. What is the problem in forcing (3) and (4) to be empty? It is a validation so resulting database will be consistent.
5. Product version say X.X.Y Upgrade available X.X.Z X.X.Y crashed in a way it's easier to set up and restore database We can face following two scenarios A) setup restore database (if there was change in postgres stored in procedures between X.X.Y and X.X.Z we will lose it). B) restore database setup (would fail according to this bug)
(In reply to Tomas Dosek from comment #4) > 5. Product version say X.X.Y > Upgrade available X.X.Z > X.X.Y crashed in a way it's easier to set up and restore database > We can face following two scenarios I do not understand why X.X.Z is relevant. > A) setup > restore database (if there was change in postgres stored in procedures > between X.X.Y and X.X.Z we will lose it). Unfortunately, you cannot assume there is no database change between z-streams, restore must use the same version of engine, then use engine-setup to upgrade. > B) restore database > setup (would fail according to this bug) Right, as restore database cannot be done to none empty database, all you need to do is to empty database or drop/create it and then restore. Restore in postgresql is just execute sql statements in accumulative method, to begin with... the fact that we call it restore is incorrect... as you expect that: sys1 == sys1.restore() The only way to achieve the above statement is to run these sql statements within empty database. But why don't you use the engine-backup utility and drop the setup phase, it should do it all...
The X.X.Z is relevant in case that X.X.Y installation (with exception of database) is corrupted and you need to do cleanup and setup. In such a case you will most likely get the Z version and will do the restore there. In such a way it makes sense, but we really need this to be well defined and most importantly well documented process. Also some people in the field use complete pg_dump instead of backup utility as it's commonly used backup method for opensource backup solutions like amanda.
(In reply to Tomas Dosek from comment #6) > Also some people in the field use complete pg_dump instead of backup utility > as it's commonly used backup method for opensource backup solutions like > amanda. No problem to use pg_dump... as long as restore is done into empty database, this is the best approach and is transparent to setup.
Verified with ovirt-engine-3.4.0-0.11.beta3.el6.noarch. 1) installed engine with remote db on machine1 2) added some elements to environment (dc, clust, host) 3) tried yo install engine with the same remote db and same credentials on machine 2 4) engine-setup didn't try to use the same non-empty db 5) this scenario introduced the following bz - https://bugzilla.redhat.com/show_bug.cgi?id=1070160
Doesn't work. Does not get all tables - e.g. if table was created by user postgres, it will not be returned from select * from information_schema.tables where table_schema = 'public'; searched and found: select * from pg_catalog.pg_tables where schemaname='public'; What do you say?
(In reply to Yedidyah Bar David from comment #9) > Doesn't work. Does not get all tables - e.g. if table was created by user > postgres, it will not be returned from > > select * from information_schema.tables where table_schema = 'public'; > > searched and found: > > select * from pg_catalog.pg_tables where schemaname='public'; > > What do you say? OK, so please use the 2nd one , good catch
(In reply to Eli Mesika from comment #10) > (In reply to Yedidyah Bar David from comment #9) > > Doesn't work. Does not get all tables - e.g. if table was created by user > > postgres, it will not be returned from > > > > select * from information_schema.tables where table_schema = 'public'; > > > > searched and found: > > > > select * from pg_catalog.pg_tables where schemaname='public'; > > > > What do you say? > > OK, so please use the 2nd one , good catch Good catch indeed, but not mine - reported by Sefi while trying to verify.
This is an automated message. Re-targeting all non-blocker bugs still open on 3.4.0 to 3.4.1.
It seems that second installed engine just backs up and overwrites the remote database of the first engine, without any warning Here is steps which I took 1) installed rhevm on server called DB and installed it with local database (in orded to get postgress and everything configured easily) 2) enabled remote access to postgres on server DB 3) on server DB: su - postgress psql create database engine_remote owner engine template template0 encoding 'UTF8' lc_collate 'en_US.UTF-8' lc_ctype 'en_US.UTF-8'; 4) grep -i pass /etc/ovirt-engine/engine.conf.d/10-setup-database.conf ENGINE_DB_PASSWORD="*******" 5) installed rhevm on server called engine1: --== CONFIGURATION PREVIEW ==-- Engine database name : engine_remote Engine database secured connection : False Engine database host : db Engine database user name : engine Engine database host name validation : False Engine database port : 5432 .... [ INFO ] Creating Engine database schema .... [ INFO ] Execution of setup completed successfully 6) installed rhevm on server called engine2: --== CONFIGURATION PREVIEW ==-- Engine database name : engine_remote Engine database secured connection : False Engine database host : db Engine database user name : engine Engine database host name validation : False Engine database port : 5432 .... [ INFO ] Backing up database db:engine_remote to '/var/lib/ovirt-engine/backups/engine-20140328155404.cDjUjk.sql'. [ INFO ] Updating Engine database schema .... [ INFO ] Execution of setup completed successfully logs from all 3 machines attached Didi can you confirm that this is how it is expected to work?
Created attachment 879896 [details] engine1_logs
Created attachment 879897 [details] engine2_logs
Created attachment 879898 [details] db_logs
Yes, I do not think it is actually changed, although the check empty now is ok, and worked in this case, it still tries to upgrade the database: 2014-03-28 15:54:05 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine.db.schema plugin.executeRaw:366 execute: ['/usr/share/ovirt-engine/dbscripts/upgrade.sh' ... this is the same behavior as we always had, if database exists preserve its contents, assuming it was restored and user wishes to use previous backed up database.
(In reply to Alon Bar-Lev from comment #17) > Yes, I do not think it is actually changed, although the check empty now is > ok, and worked in this case, it still tries to upgrade the database: > > 2014-03-28 15:54:05 DEBUG > otopi.plugins.ovirt_engine_setup.ovirt_engine.db.schema > plugin.executeRaw:366 execute: > ['/usr/share/ovirt-engine/dbscripts/upgrade.sh' ... > > this is the same behavior as we always had, if database exists preserve its > contents, assuming it was restored and user wishes to use previous backed up > database. Alon - I did not understand your reply - do you think current behavior is good or not? IMO it's not. We fixed the function that checked if a database is empty, but I do not think we should agree to use an existing non-empty database. In 3.3 we had an "action" - "upgrade" - which we dropped now. Now we simply check everywhere if "NEW_DATABASE". So now that the function correctly says it's not "new", we simply do an upgrade. Not sure it was a good idea to drop "upgrade" - logic should have been: if upgrade: if new_database: something is wrong, alert and exit else: upgrade elif setup: if new_database: ok, setup else: something is wrong, alert and exit But now that we do not have "upgrade", if we want to implement this logic, it should be done earlier: if we have existing credentials: upgrade? else: ask for credentials if not new_database: alert Or something like that. Should consider all options including answer-file-supplied credentials etc. Currently setting to ASSIGNED. Alon - please comment.
The flow that was always supported in product, and I think has a value: 1. setup product. 2. backup database. 3. reinstall host 4. restore only database. 5. setup product with database from (4). 6. working environment. It is also very important for developers, to cleanup installation and reuse existing database. If a user manually specified existing database (not in provisioning), it is his responsibility to avoid conflicts.
(In reply to Alon Bar-Lev from comment #19) > The flow that was always supported in product, and I think has a value: > > > 1. setup product. > 2. backup database. > 3. reinstall host > 4. restore only database. > 5. setup product with database from (4). > 6. working environment. I wasn't aware of this flow. I thought that people that want something like this do (4) after (5). That is, do a clean installation, then drop the resulting db and restore their own. > > It is also very important for developers, to cleanup installation and reuse > existing database. > > If a user manually specified existing database (not in provisioning), it is > his responsibility to avoid conflicts. So you basically claim CLOSE NOTABUG?
(In reply to Yedidyah Bar David from comment #20) > > > > It is also very important for developers, to cleanup installation and reuse > > existing database. > > > > If a user manually specified existing database (not in provisioning), it is > > his responsibility to avoid conflicts. > > So you basically claim CLOSE NOTABUG? there was a bug I stopped tracking the number, that was fixed that even in provisioning mode, if previous database was created using postgres user we could not detect non empty database, this was a real bug and is resolved now. the above sequence presented at comment#13 of manual specifying same database is currently by design and I think it is required. this has nothing to do with database restore, as in that case database must be empty before restore.
Moving to 3.4.1 since 3.4.0 has been released
This is an automated message. oVirt 3.4.1 has been released. This issue has been retargeted to 3.5.0 since it has not been marked as high priority or severity issue, please retarget if needed.