Bug 1202515 - GEM_HOME can break "gear" commands
Summary: GEM_HOME can break "gear" commands
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Containers
Version: 2.2.0
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
: ---
Assignee: Brenton Leanhardt
QA Contact: libra bugs
Depends On: 1187829
TreeView+ depends on / blocked
Reported: 2015-03-16 19:10 UTC by Brenton Leanhardt
Modified: 2019-03-22 07:40 UTC (History)
12 users (show)

Fixed In Version: rubygem-openshift-origin-node-
Doc Type: Bug Fix
Doc Text:
In certain situations, such as pre- and post-receive hooks, the presence of gems in a gear's $GEM_HOME environment variable previously could break /usr/bin/gear subcommands. This was due to the user-supplied gems in $GEM_HOME overriding the OpenShift-provided ones, potentially causing exceptions. This bug fix updates the node logic to explicitly set GEM_HOME to '' for the /usr/bin/gear command and Ruby SDK to protect them from running with user-supplied gems. As a result, exceptions are no longer raised during a "git push" in this scenario.
Clone Of: 1187829
Last Closed: 2015-04-06 17:06:51 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Private Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 1337873 0 None None None Never
Red Hat Product Errata RHBA-2015:0779 0 normal SHIPPED_LIVE Red Hat OpenShift Enterprise 2.2.5 bug fix and enhancement update 2015-04-06 21:05:45 UTC

Description Brenton Leanhardt 2015-03-16 19:10:32 UTC
+++ This bug was initially created as a clone of Bug #1187829 +++

Description of problem:

In situations such as pre- and post-receive hooks, presence of gems in a gear's $GEM_HOME can break /usr/bin/gear subcommands.  For example, in a gear with activemodel-4.0.5 installed, this exception is raised by require 'openshift-origin-node/model/application_container':

NameError: uninitialized constant ActiveModel::Observing
	from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-common-1.29.1/lib/openshift-origin-common/models/model.rb:13:in `<class:Model>'
	from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-common-1.29.1/lib/openshift-origin-common/models/model.rb:6:in `<module:OpenShift>'
	from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-common-1.29.1/lib/openshift-origin-common/models/model.rb:5:in `<top (required)>'
	from /opt/rh/ruby193/root/usr/share/rubygems/rubygems/custom_require.rb:55:in `require'
	from /opt/rh/ruby193/root/usr/share/rubygems/rubygems/custom_require.rb:55:in `require'
	from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-common-1.29.1/lib/openshift-origin-common.rb:22:in `<top (required)>'
	from /opt/rh/ruby193/root/usr/share/rubygems/rubygems/custom_require.rb:55:in `require'
	from /opt/rh/ruby193/root/usr/share/rubygems/rubygems/custom_require.rb:55:in `require'
	from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.34.1/lib/openshift-origin-node/model/frontend_proxy.rb:20:in `<top (required)>'
	from /opt/rh/ruby193/root/usr/share/rubygems/rubygems/custom_require.rb:55:in `require'
	from /opt/rh/ruby193/root/usr/share/rubygems/rubygems/custom_require.rb:55:in `require'
	from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.34.1/lib/openshift-origin-node/model/application_container.rb:18:in `<top (required)>'
	from /opt/rh/ruby193/root/usr/share/rubygems/rubygems/custom_require.rb:60:in `require'
	from /opt/rh/ruby193/root/usr/share/rubygems/rubygems/custom_require.rb:60:in `rescue in require'
	from /opt/rh/ruby193/root/usr/share/rubygems/rubygems/custom_require.rb:35:in `require'
	from (irb):11
	from /opt/rh/ruby193/root/usr/bin/irb:12:in `<main>'irb(main):012:0> 

Version-Release number of selected component (if applicable):

--- Additional comment from Andy Grimm on 2015-02-02 11:59:37 EST ---

For one user who experienced this issue, we have temporarily worked around it by changing his gear's git hooks to:

GEM_HOME= gear prereceive


GEM_HOME= gear postreceive

Unfortunately, this is not something a user can fix themselves.

--- Additional comment from Andy Goldstein on 2015-02-02 15:14:25 EST ---


1) Modify all git repositories' git hooks to unset GEM_HOME. This would require a migration. And it's not a complete solution (see below).

2) Modify the 'gear' script to add ENV['GEM_HOME']='' at the top, before the 'require' lines. Modify /usr/lib/openshift/cartridge_sdk/ruby/sdk.rb to do the same as well, because methods from this file are invoked as part of the build and deployment process from various cartridges from the 'control' script, which receives as its env the combination of the app, cart, and user env vars.

#1 isn't a complete solution because it won't fix the calls made to /usr/lib/openshift/cartridge_sdk/ruby/sdk.rb.

For #2, I haven't looked yet to see what the impact would be, if any, of modifying the ruby sdk.rb file.

I think #2 is our best option, if we want to proceed, assuming it doesn't have adverse side effects.

--- Additional comment from Andy Goldstein on 2015-02-02 15:43:21 EST ---

PR: https://github.com/openshift/origin-server/pull/6065

Running extended gear and node tests now. Also creating a fork_ami for QE to test.

--- Additional comment from Andy Grimm on 2015-02-02 15:43:42 EST ---

@agolste, I completely agree with you.  I was not suggesting that my workaround was the right fix.  That was just the easiest thing I could do without modifying things _outside_ the affected user's gear.  And you are right that "gear deploy" and other commands are probably still failing for them; the user has not gotten back to me yet.

--- Additional comment from Meng Bo on 2015-02-03 04:58:45 EST ---

We have done the testing of the changes. Since the fork_ami build failed (online-broker cannot be started), I applied the changes to devenv_5410 to test it.

After the changes are made, the installed conflicting gems via SSH into the gear will not block the non-scale git push, scaled apps git push and scale up. (Which can be reproduced w/o the fix)
And we also do regression testing on this changes, there is no regression issue found.

--- Additional comment from Andy Goldstein on 2015-02-03 09:26:20 EST ---

PR is in the merge queue

--- Additional comment from Meng Bo on 2015-02-09 01:06:13 EST ---

The PR has been merged in to devenv_5424, and the issue was fixed.

Move the bug to verified.

Comment 3 Anping Li 2015-03-17 11:17:27 UTC
Verified and pass on 
1. The error can be reproduced without the fix., and no such issue on puddle-2-2-2015-03-16

Recreate step as below:
 echo 'echo $GEM_HOME' > .openshift/action_hooks
 echo 'gem install activemodel' >> .openshift/action_hooks
2.git push the change

3. the same error was catched in output
remote: Installing RDoc documentation for thread_safe-0.3.5...
remote: Installing RDoc documentation for tzinfo-1.2.2...
remote: Installing RDoc documentation for minitest-5.5.1...
remote: Installing RDoc documentation for activesupport-4.2.0...
remote: Installing RDoc documentation for builder-3.2.2...
remote: Installing RDoc documentation for activemodel-4.2.0...
remote: Building Ruby cartridge
remote: Preparing build for deployment
remote: Deployment id is 1ddace62
remote: Activating deployment
remote: Compilation of assets is disabled or assets not detected.
remote: Starting Ruby cartridge
remote: /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-common- `<class:Model>': uninitialized constant ActiveModel::Observing (NameError)
remote: 	from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-common- `<module:OpenShift>'

Comment 5 errata-xmlrpc 2015-04-06 17:06:51 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.