Bug 1045262

Summary: 2.0 upgrade mongo migrations fail with replica set
Product: OpenShift Container Platform Reporter: Luke Meyer <lmeyer>
Component: Cluster Version OperatorAssignee: Luke Meyer <lmeyer>
Status: CLOSED CURRENTRELEASE QA Contact: libra bugs <libra-bugs>
Severity: high Docs Contact:
Priority: high    
Version: 2.0.0CC: bleanhar, jialiu, libra-onpremise-devel
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-02-04 14:52:35 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Luke Meyer 2013-12-20 03:31:05 UTC
Description of problem:
/usr/lib/ruby/site_ruby/1.8/ose-upgrade/broker/upgrades/2/pre/04-broker-migrate-datastore-prerelease fails with a replica set.

Version-Release number of selected component (if applicable):

Steps to Reproduce:
1. Install 1.2 HA infrastructure
2. Run ose-upgrade begin pre

Results:

ERROR: run_script
/usr/lib/ruby/site_ruby/1.8/ose-upgrade/broker/upgrades/2/pre/04-broker-migrate-datastore-prerelease had errors:
--BEGIN OUTPUT--
+ /usr/libexec/openshift/ose-upgrade-migrate-datastore --prerelease
/opt/rh/ruby193/root/usr/local/share/gems/gems/mongo-1.8.1/lib/mongo/networking.rb:84:in `send_message_with_gle': 10058: not master (Mongo::ConnectionFailure)
        from /opt/rh/ruby193/root/usr/local/share/gems/gems/mongo-1.8.1/lib/mongo/collection.rb:1100:in `block in insert_documents'
        from /opt/rh/ruby193/root/usr/local/share/gems/gems/mongo-1.8.1/lib/mongo/util/logging.rb:33:in `block in instrument'
        from /opt/rh/ruby193/root/usr/local/share/gems/gems/mongo-1.8.1/lib/mongo/util/logging.rb:65:in `instrument'
        from /opt/rh/ruby193/root/usr/local/share/gems/gems/mongo-1.8.1/lib/mongo/util/logging.rb:32:in `instrument'
        from /opt/rh/ruby193/root/usr/local/share/gems/gems/mongo-1.8.1/lib/mongo/collection.rb:1098:in `insert_documents'
        from /opt/rh/ruby193/root/usr/local/share/gems/gems/mongo-1.8.1/lib/mongo/collection.rb:1050:in `generate_indexes'
        from /opt/rh/ruby193/root/usr/local/share/gems/gems/mongo-1.8.1/lib/mongo/collection.rb:540:in `create_index'
        from /usr/libexec/openshift/ose-upgrade-migrate-datastore:567:in `block in ensure_members_index'
        from /usr/libexec/openshift/ose-upgrade-migrate-datastore:566:in `each'
        from /usr/libexec/openshift/ose-upgrade-migrate-datastore:566:in `ensure_members_index'
        from /usr/libexec/openshift/ose-upgrade-migrate-datastore:408:in `migrate'
        from /usr/libexec/openshift/ose-upgrade-migrate-datastore:11:in `block in migrate'
        from /usr/libexec/openshift/ose-upgrade-migrate-datastore:9:in `each'
        from /usr/libexec/openshift/ose-upgrade-migrate-datastore:9:in `migrate'
        from /usr/libexec/openshift/ose-upgrade-migrate-datastore:1452:in `<main>'
Starting prerelease migrations.
Running migration Migrate_2_0_32...

Comment 3 Jason DeTiberus 2013-12-20 21:27:44 UTC
The broker was only configured for one mongo host, so the DataStore class was using MongoClient and connecting to the configured host (which was a slave).

Comment 4 Luke Meyer 2013-12-22 15:35:01 UTC
https://github.com/openshift/openshift-extras/pull/257
https://github.com/openshift/openshift-extras/commit/2e06aea6c15d3f58af1991d2ddc8061f1d964848
... and similar for 1.2. openshift.sh was not configuring the broker correctly when the mongodb replicants were installed locally. I would have noticed this broke the brokers too if I had tried a different broker. Not an upgrade bug at all.

Comment 5 Johnny Liu 2013-12-27 15:05:40 UTC
# wget https://raw.github.com/openshift/openshift-extras/enterprise-1.2/enterprise/install-scripts/generic/openshift.sh

Modify openshift.sh:
CONF_DATASTORE_REPLICANTS="datastore01.example.com:27017,datastore02.example.com:27017,datastore03.example.com:27017"

According to the role of every vm, modify CONF_INSTALL_COMPONENTS.


After installation, check broker.conf:
MONGO_HOST_PORT="datastore01.example.com:27017,datastore02.example.com:27017,datastore03.example.com:27017"

Do upgrade process, it is working well.

# ose-upgrade pre
INFO: OpenShift broker installed.
INFO: Setting broker step 'pre' status to UPGRADING
INFO: running /usr/lib/ruby/site_ruby/1.8/ose-upgrade/broker/upgrades/2/step_lock oseupgrade_2_pre
INFO: /usr/lib/ruby/site_ruby/1.8/ose-upgrade/broker/upgrades/2/step_lock oseupgrade_2_pre ran without error:
--BEGIN OUTPUT--

--END /usr/lib/ruby/site_ruby/1.8/ose-upgrade/broker/upgrades/2/step_lock oseupgrade_2_pre OUTPUT--
INFO: Running upgrade scripts in /usr/lib/ruby/site_ruby/1.8/ose-upgrade/broker/upgrades/2/pre
INFO: running /usr/lib/ruby/site_ruby/1.8/ose-upgrade/broker/upgrades/2/pre/02-broker-backup-conf-files
INFO: /usr/lib/ruby/site_ruby/1.8/ose-upgrade/broker/upgrades/2/pre/02-broker-backup-conf-files ran without error:
--BEGIN OUTPUT--
+ mkdir -p /etc/openshift/upgrade/2/
+ for file in /etc/mcollective/client.cfg
+ command cp -f /etc/mcollective/client.cfg /etc/openshift/upgrade/2/
+ cp -f /etc/mcollective/client.cfg /etc/openshift/upgrade/2/

--END /usr/lib/ruby/site_ruby/1.8/ose-upgrade/broker/upgrades/2/pre/02-broker-backup-conf-files OUTPUT--
INFO: running /usr/lib/ruby/site_ruby/1.8/ose-upgrade/broker/upgrades/2/pre/02-broker-clear-most-pending-ops
INFO: /usr/lib/ruby/site_ruby/1.8/ose-upgrade/broker/upgrades/2/pre/02-broker-clear-most-pending-ops ran without error:
--BEGIN OUTPUT--
+ oo-admin-clear-pending-ops
0 applications were cleaned up. 0 users were cleaned up. 0 domains were cleaned up.

--END /usr/lib/ruby/site_ruby/1.8/ose-upgrade/broker/upgrades/2/pre/02-broker-clear-most-pending-ops OUTPUT--
INFO: running /usr/lib/ruby/site_ruby/1.8/ose-upgrade/broker/upgrades/2/pre/04-broker-migrate-datastore-prerelease
INFO: /usr/lib/ruby/site_ruby/1.8/ose-upgrade/broker/upgrades/2/pre/04-broker-migrate-datastore-prerelease ran without error:
--BEGIN OUTPUT--
+ /usr/libexec/openshift/ose-upgrade-migrate-datastore --prerelease
Starting prerelease migrations.
Running migration Migrate_2_0_32...
  Loading user names into memory
Domain members updated: 1
  Loading domain ownership into memory
Application members updated: 1
Migration Migrate_2_0_32 finished.
Running migration Migrate_2_0_36...
Migration Migrate_2_0_36 finished.

Migrations complete.
Done!

--END /usr/lib/ruby/site_ruby/1.8/ose-upgrade/broker/upgrades/2/pre/04-broker-migrate-datastore-prerelease OUTPUT--
INFO: running /usr/lib/ruby/site_ruby/1.8/ose-upgrade/broker/upgrades/2/pre/05-broker-add-authorizations-indexes
INFO: /usr/lib/ruby/site_ruby/1.8/ose-upgrade/broker/upgrades/2/pre/05-broker-add-authorizations-indexes ran without error:
--BEGIN OUTPUT--

--END /usr/lib/ruby/site_ruby/1.8/ose-upgrade/broker/upgrades/2/pre/05-broker-add-authorizations-indexes OUTPUT--
INFO: running /usr/lib/ruby/site_ruby/1.8/ose-upgrade/broker/upgrades/2/step_lock oseupgrade_2_pre done
INFO: /usr/lib/ruby/site_ruby/1.8/ose-upgrade/broker/upgrades/2/step_lock oseupgrade_2_pre done ran without error:
--BEGIN OUTPUT--

--END /usr/lib/ruby/site_ruby/1.8/ose-upgrade/broker/upgrades/2/step_lock oseupgrade_2_pre done OUTPUT--
INFO: Setting broker step 'pre' status to COMPLETE
INFO: Next step is 'outage'