These are the steps I got to work:

On 5.10 database appliance:

- `systemctl stop evmserverd`
- `pg_dumpall -c --if-exists -f vmdb-5.10.pg`

On newly deployed 5.11 appliance:

- Configure a new internal database (appliance console option 7).
- Fetch v2_key from one of the 5.10 appliance.
- Select the same region number and database password as the old installation
- wait for the application to come up

- `systemctl stop evmserverd`
- Copy the GUID file from the 5.10 appliance
- Copy the database dump from the 5.10 appliance
- `psql -f vmdb-5.10.pg postgres`
- `vmdb`
- `bin/rake db:migrate`
- `bin/rake evm:automate:reset`
- `systemctl start evmserverd`

I based this on the last time we needed to "lift-and-shift" the environment for an upgrade which was 5.4 to 5.5 (https://access.redhat.com/articles/2076193)
I imagine this will need some more detail to deal with external databases and additional appliances, but this is what worked for me in a single appliance environment and is probably a good place to start.

We should also do some testing to see just how slow things will be for a large database. We can probably compress the dump file before writing it to disk or even stream the data over to the new database with something like this:

`pg_dumpall -c --if-exists | psql -d postgres -h <5.11-database-hostname> -U root`

But I haven't gotten a chance to test that yet.

All of the stuff about resizing the disk space and dealing with repos can be removed since we're deploying new appliances.

Since we're also re-creating the databases both HA and region-to-region replication will need to be re-initialized so we can also remove all the HA and region migration ordering stuff. Additionally the guidance should be to remove all the subscriptions from the global region before migrating the global database since the subscriptions will have to be dropped and recreated anyway (it doesn't make sense to move all that data to just delete it and re-replicate it).

Also, I think we will only support upgrades from the most recent prior version (so 4.7).

I think that will be a good start and we can iterate from there.

I tried the steps above, but got

[root@dhcp-8-197-160 vmdb]# psql -f ~/59.nostop.dumpall 
psql: FATAL:  database "root" does not exist


which stops after

[root@dhcp-8-197-160 vmdb]# createdb root

Also, as I said in https://bugzilla.redhat.com/show_bug.cgi?id=1734309#c7

* It seems the documentation should tell to stop evmserverd before making any dump.

(In reply to Jaroslav Henner from comment #9)
> I tried the steps above, but got
> 
> [root@dhcp-8-197-160 vmdb]# psql -f ~/59.nostop.dumpall 
> psql: FATAL:  database "root" does not exist
> 
> 
> which stops after
> 
> [root@dhcp-8-197-160 vmdb]# createdb root

You did not execute the steps correctly the command I listed is `psql -f vmdb-5.10.pg postgres`, you're missing the "postgres" at the end which would solve the error you hit.

(In reply to Jaroslav Henner from comment #10)
> Also, as I said in https://bugzilla.redhat.com/show_bug.cgi?id=1734309#c7
> 
> * It seems the documentation should tell to stop evmserverd before making
> any dump.

This is already the first step in both sections in comment 7

Setting urgent because to ensure good testing of upgrades I need to have good information about how it is supposed to work.

Note that the db dump that is created on CFME 5.10 using appliance console can be restored with different command:

pg_restore -v --dbname=vmdb_production the_dump_file

This should probably be either written in the doc or the db dump command should be updated in cfme 5.10 appliance console.

seems we forgot to mention the step to copy the /var/www/miq/vmdb/config/database.yml between the machines as it holds encryption password

note this doc https://access.redhat.com/documentation/en-us/red_hat_cloudforms/5.0-beta/html-single/high_availability_guide/index#updating-ha is not up-to-date as it talks about yum update, but we cannot do yum update to update to 5.11

I thikn the document should contain reciepes how to do upgrades of
 * Single instance deployment
 * Deployment with external database
 * HA deployment
 * Centralized control deployment (global and remote replication)

I tried update of the deployment with two appliances with one DB where one is configured to use the db of the other one, both running evmserved (the Deployment with external database). I discovered there is a problem that I don't have any Zone.

I reported a bug about it:
https://bugzilla.redhat.com/show_bug.cgi?id=1749694