Bug 856551

Summary:

Upgrade failed with many errors

Product:

OKD

Reporter:

Jianwei Hou <jhou>

Component:

Pod

Assignee:

Dan McPherson <dmcphers>

Status:

CLOSED NOTABUG

QA Contact:

libra bugs <libra-bugs>

Severity:

high

Docs Contact:

Priority:

high

Version:

2.x

Target Milestone:

---

Target Release:

---

Hardware:

Unspecified

OS:

Unspecified

Whiteboard:

Fixed In Version:

Doc Type:

Bug Fix

Doc Text:

Story Points:

---

Clone Of:

Environment:

Last Closed:

2012-09-12 14:02:29 UTC

Type:

Bug

Regression:

---

Mount Type:

---

Documentation:

---

CRM:

Verified Versions:

Category:

---

oVirt Team:

---

RHEL 7.3 requirements from Atomic Host:

Cloudforms Team:

---

Target Upstream Version:

Embargoed:

Attachments:

Description	Flags
logs in upgrade	none

Description Jianwei Hou 2012-09-12 09:54:30 UTC

Created attachment 612040 [details]
logs in upgrade

Description of problem:
Upgrade is broken, when doing upgrade, some errors are displayed. And after upgrade, "Cannot connect to a replica set using seeds localhost:27017, :, :" is caught when trying to do some operations from client.

Version-Release number of selected component (if applicable):
Upgraded from devenv-stage_234

How reproducible:
Always

Steps to Reproduce:
1. Launch an older instance, and prepare all cartridge applications
2. modify /etc/yum.repos.d/li.repo, replace 'stage' with 'candidate' in order to upgrade to latest devenv, then do server upgrade
   yum upgrade
3. rhc-admin-migrate --version 2.0.17
  
Actual results:
When upgrading, There are some "FAILED" keywords displayed, the log is in attachment.
And operations from client are broken.
Migration can not be completed.

[root@ip-10-191-45-94 ~]# rhc-admin-migrate --version 2.0.17
/usr/lib/ruby/gems/1.8/gems/mongo-1.5.2/lib/mongo/util/pool_manager.rb:282:in `get_valid_seed_node': Cannot connect to a replica set using seeds localhost:27017 (Mongo::ConnectionFailure)
	from /usr/lib/ruby/gems/1.8/gems/mongo-1.5.2/lib/mongo/util/pool_manager.rb:158:in `connect_to_members'
	from /usr/lib/ruby/gems/1.8/gems/mongo-1.5.2/lib/mongo/util/pool_manager.rb:30:in `connect'
	from /usr/lib/ruby/gems/1.8/gems/mongo-1.5.2/lib/mongo/repl_set_connection.rb:160:in `connect'
	from /usr/lib/ruby/gems/1.8/gems/mongo-1.5.2/lib/mongo/repl_set_connection.rb:500:in `setup'
	from /usr/lib/ruby/gems/1.8/gems/mongo-1.5.2/lib/mongo/repl_set_connection.rb:144:in `initialize'
	from /var/www/stickshift/broker/config/initializers/broker.rb:12:in `new'
	from /var/www/stickshift/broker/config/initializers/broker.rb:12
	from /usr/lib/ruby/gems/1.8/gems/activesupport-3.0.13/lib/active_support/dependencies.rb:236:in `load'
	from /usr/lib/ruby/gems/1.8/gems/activesupport-3.0.13/lib/active_support/dependencies.rb:236:in `load'
	from /usr/lib/ruby/gems/1.8/gems/activesupport-3.0.13/lib/active_support/dependencies.rb:227:in `load_dependency'
	from /usr/lib/ruby/gems/1.8/gems/activesupport-3.0.13/lib/active_support/dependencies.rb:236:in `load'
	from /usr/lib/ruby/gems/1.8/gems/railties-3.0.13/lib/rails/engine.rb:201
	from /usr/lib/ruby/gems/1.8/gems/railties-3.0.13/lib/rails/engine.rb:200:in `each'
	from /usr/lib/ruby/gems/1.8/gems/railties-3.0.13/lib/rails/engine.rb:200
	from /usr/lib/ruby/gems/1.8/gems/railties-3.0.13/lib/rails/initializable.rb:25:in `instance_exec'
	from /usr/lib/ruby/gems/1.8/gems/railties-3.0.13/lib/rails/initializable.rb:25:in `run'
	from /usr/lib/ruby/gems/1.8/gems/railties-3.0.13/lib/rails/initializable.rb:50:in `run_initializers'
	from /usr/lib/ruby/gems/1.8/gems/railties-3.0.13/lib/rails/initializable.rb:49:in `each'
	from /usr/lib/ruby/gems/1.8/gems/railties-3.0.13/lib/rails/initializable.rb:49:in `run_initializers'
	from /usr/lib/ruby/gems/1.8/gems/railties-3.0.13/lib/rails/application.rb:134:in `initialize!'
	from /usr/lib/ruby/gems/1.8/gems/railties-3.0.13/lib/rails/application.rb:77:in `send'
	from /usr/lib/ruby/gems/1.8/gems/railties-3.0.13/lib/rails/application.rb:77:in `method_missing'
	from /var/www/stickshift/broker/config/environment.rb:5
	from /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require'
	from /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require'
	from /usr/bin/rhc-admin-migrate:558


hjw@my scripts$ rhc app show -a zend1 -d
Password: ******

Submitting form:
debug: true
rhlogin: jhou
Contacting https://ec2-184-73-150-200.compute-1.amazonaws.com
Problem reported from server. Response code was 500.

DEBUG:
Cannot connect to a replica set using seeds localhost:27017, :, :/usr/lib/ruby/gems/1.8/gems/mongo-1.5.2/lib/mongo/util/pool_manager.rb:282:in `get_valid_seed_node'
/usr/lib/ruby/gems/1.8/gems/mongo-1.5.2/lib/mongo/util/pool_manager.rb:158:in `connect_to_members'
/usr/lib/ruby/gems/1.8/gems/mongo-1.5.2/lib/mongo/util/pool_manager.rb:30:in `connect'
/usr/lib/ruby/gems/1.8/gems/mongo-1.5.2/lib/mongo/repl_set_connection.rb:160:in `connect'
/usr/lib/ruby/gems/1.8/gems/mongo-1.5.2/lib/mongo/repl_set_connection.rb:500:in `setup'
/usr/lib/ruby/gems/1.8/gems/mongo-1.5.2/lib/mongo/repl_set_connection.rb:144:in `initialize'

Exit Code: 1
broker_c: namespacerhloginsshapp_uuiddebugaltercartridgecart_typeactionapp_nameapi
api_c: placeholder
API version:    1.1.3

RESULT:
Cannot connect to a replica set using seeds localhost:27017, :, :


Expected results:
Upgrade should completed successfully.

Additional info:

Comment 1 Jianwei Hou 2012-09-12 10:13:45 UTC

In the log, there was one line:
Stopping mongod: [FAILED]

Tried to stop and start mongod manually, it was OK
[root@ip-10-191-45-94 ~]# /etc/init.d/mongod stop
Stopping mongod:                                           [  OK  ]
[root@ip-10-191-45-94 ~]# /etc/init.d/mongod start
Starting mongod:                                           [  OK  ]

Comment 2 Dan McPherson 2012-09-12 14:02:29 UTC

Sounds like mongo was upgraded during the yum update.  This error is expected if that is the case.  Or at least it's common practice.