Bug 1063265

Compare thor gem package between ose env and online devenv.

Online devenv:
# rpm -qa|grep thor
rubygem-thor-0.18.1-1.el6oso.noarch
ruby193-rubygem-thor-0.15.4-2.el6.noarch


OSE env:
# rpm -qa|grep thor
ruby193-rubygem-thor-0.15.4-2.el6.noarch
rubygem-thor-0.14.6-2.el6op.noarch


Obviously in ose env, thor gem package is older then online devenv, so I install the latest one using gem command.
# gem install thor
Fetching: thor-0.18.1.gem (100%)
Successfully installed thor-0.18.1
1 gem installed
Installing ri documentation for thor-0.18.1...
Installing RDoc documentation for thor-0.18.1...


Then create redmine-quickstart app again, it succeed. So OSE need ship newer package for thor gem.

Thanks for tracking this down.  This should be easy to fix since we ship the ruby 1.8 gem in our channels.

I have tested this against an OSE 2.0 instance installed from RHSM and against a recent 2.1 instance, and cannot reproduce the error.

In addition, I tested the OSE 2.0 case further by pushing rev 5509ec57fb651838603a99d90e36125ee4e6d9c7 for the initial deployment instead of the most recent revision, and also didn't see the error. This would have been the revision of the quickstart that was current at the time the ticket was created.

Ah, maybe I miss some info. 

Try to create a ruby-1.8 (NOT ruby-1.9) to reproduce this issue, I just tried it yesterday on ose-2.0 production env using rhsm, it still could be reproduced.

I already update bug summary to reflect the key point to reproduce this issue.

Was not able to reproduce, but did get a failure on initial git push; I imagine it's unrelated but preserving here just in case:

-bash-4.1# git push
Counting objects: 3269, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (2356/2356), done.
Writing objects: 100% (3259/3259), 5.14 MiB, done.
Total 3259 (delta 794), reused 3247 (delta 787)
remote: Stopping Ruby cartridge
remote: [Thu Mar 13 17:15:22 2014] [warn] PassEnv variable SHELL was undefined
remote: [Thu Mar 13 17:15:22 2014] [warn] PassEnv variable USER was undefined
remote: [Thu Mar 13 17:15:22 2014] [warn] PassEnv variable LOGNAME was undefined
remote: Waiting for stop to finish
remote: Stopping MySQL cartridge
remote: Building git ref 'master', commit 64ccf86
remote: Building Ruby cartridge
remote: Bundling RubyGems based on Gemfile/Gemfile.lock to repo/vendor/bundle with 'bundle install --deployment'
remote: Fetching source index for https://rubygems.org/
remote: Installing rake (10.1.1) 
remote: Installing i18n (0.6.9) 
remote: Installing multi_json (1.8.4) 
remote: Installing activesupport (3.2.16) 
remote: Installing builder (3.0.0) /var/lib/openshift/53221f926892df69dd00008d/ruby//lib/ruby_context: line 7:  1318 Killed                  bundle install --deployment
remote: An error occurred executing 'gear postreceive' (exit code: 137)
remote: Error message: Failed to execute: 'control build' for /var/lib/openshift/53221f926892df69dd00008d/ruby
remote: 
remote: For more details about the problem, try running the command again with the '--trace' option.
To ssh://53221f926892df69dd00008d.com/~/git/redmine.git/
   803c7b2..64ccf86  master -> master
-bash-4.1#

About comments 6, that issue is blocking you to reproduce this bug. So I think you need fix the "Installing builder" failure firstly. At my side, I could not see this failure, pls check comment 7.

Could you give me the address of a VM where I can reproduce the issue?

On your broker, I still can't reproduce the issue, it's blocking at the rmagick gem:

[jolamb@aries redminejwl (master)]$ git push
warning: push.default is unset; its implicit value is changing in
Git 2.0 from 'matching' to 'simple'. To squelch this message
and maintain the current behavior after the default changes, use:

  git config --global push.default matching

To squelch this message and adopt the new behavior now, use:

  git config --global push.default simple

See 'git help config' and search for 'push.default' for further information.
(the 'simple' mode was introduced in Git 1.7.11. Use the similar mode
'current' instead of 'simple' if you sometimes use older versions of Git)

^C
[jolamb@aries redminejwl (master)]$ git commit -amx
On branch master
Your branch is up-to-date with 'origin/master'.

Untracked files:
        .openshift/markers/force_clean_build

nothing added to commit but untracked files present
[jolamb@aries redminejwl (master)]$ git add .
[jolamb@aries redminejwl (master)]$ git commit -amx
[master c3c4238] x
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 .openshift/markers/force_clean_build
[jolamb@aries redminejwl (master)]$ git push
warning: push.default is unset; its implicit value is changing in
Git 2.0 from 'matching' to 'simple'. To squelch this message
and maintain the current behavior after the default changes, use:

  git config --global push.default matching

To squelch this message and adopt the new behavior now, use:

  git config --global push.default simple

See 'git help config' and search for 'push.default' for further information.
(the 'simple' mode was introduced in Git 1.7.11. Use the similar mode
'current' instead of 'simple' if you sometimes use older versions of Git)

Counting objects: 7, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (4/4), done.
Writing objects: 100% (4/4), 395 bytes | 0 bytes/s, done.
Total 4 (delta 2), reused 0 (delta 0)
remote: Stopping Ruby cartridge
remote: Stopping MySQL 5.1 cartridge
remote: Repairing links for 1 deployments
remote: Force clean build enabled - cleaning dependencies
remote: Building git ref 'master', commit c3c4238
remote: Building Ruby cartridge
remote: bundle install --deployment --path ./app-root/repo/vendor/bundle
remote: NOTE: You can prevent installing certain Gemfile group using: rhc env set BUNDLE_WITHOUT=groupname
remote: Fetching source index for https://rubygems.org/
remote: Installing rake (10.1.1)
remote: Installing i18n (0.6.9)
remote: Installing multi_json (1.8.4)
remote: Installing activesupport (3.2.16)
remote: Installing builder (3.0.0)
remote: Installing activemodel (3.2.16)
remote: Installing erubis (2.7.0)
remote: Installing journey (1.0.4)
remote: Installing rack (1.4.5)
remote: Installing rack-cache (1.2)
remote: Installing rack-test (0.6.2)
remote: Installing hike (1.2.3)
remote: Installing tilt (1.4.1)
remote: Installing sprockets (2.2.2)
remote: Installing actionpack (3.2.16)
remote: Installing mime-types (1.25.1)
remote: Installing polyglot (0.3.3)
remote: Installing treetop (1.4.15)
remote: Installing mail (2.5.4)
remote: Installing actionmailer (3.2.16)
remote: Installing arel (3.0.3)
remote: Installing tzinfo (0.3.38)
remote: Installing activerecord (3.2.16)
remote: Installing activeresource (3.2.16)
remote: Installing coderay (1.1.0)
remote: Installing fastercsv (1.5.5)
remote: Installing rack-ssl (1.3.3)
remote: Installing json (1.8.1) with native extensions
remote: ...
remote: .
remote:
remote: ...
remote: .
remote:
remote: Installing rdoc (3.12.2)
remote: Installing thor (0.18.1)
remote: Installing railties (3.2.16)
remote: Installing jquery-rails (2.0.3)
remote: Installing mysql (2.8.1) with native extensions .
remote: ...............
remote: .
remote:
remote: Installing mysql2 (0.3.14) with native extensions ...
remote: ..........
remote: .
remote:
remote: Installing net-ldap (0.3.1)
remote: Installing pg (0.17.1) with native extensions ...........
remote: .............................
remote: .
remote:
remote: Installing ruby-openid (2.3.0)
remote: Installing rack-openid (1.4.1)
remote: Using bundler (1.0.21)
remote: Installing rails (3.2.16)
remote: Installing rmagick (2.13.2) with native extensions .....
remote:
remote: Gem::Installer::ExtensionBuildError: ERROR: Failed to build gem native extension.
remote:
remote:         /usr/bin/ruby extconf.rb
remote: checking for Ruby version >= 1.8.5... yes
remote: checking for gcc... yes
remote: checking for Magick-config... no
remote: Can't install RMagick 2.13.2. Can't find Magick-config in /var/lib/openshift/53231afcd9fa3c18aa00010c/.gem/bin:/bin:/usr/bin:/usr/sbin
remote:
remote: *** extconf.rb failed ***
remote: Could not create Makefile due to some reason, probably lack of
remote: necessary libraries and/or headers.  Check the mkmf.log file for more
remote: details.  You may need configuration options.
remote:
remote: Provided configuration options:
remote:         --with-opt-dir
remote:         --without-opt-dir
remote:         --with-opt-include
remote:         --without-opt-include=${opt-dir}/include
remote:         --with-opt-lib
remote:         --without-opt-lib=${opt-dir}/lib
remote:         --with-make-prog
remote:         --without-make-prog
remote:         --srcdir=.
remote:         --curdir
remote:         --ruby=/usr/bin/ruby
remote:
remote:
remote: Gem files will remain installed in /var/lib/openshift/53231afcd9fa3c18aa00010c/app-root/runtime/repo/vendor/bundle/ruby/1.8/gems/rmagick-2.13.2 for inspection.
remote: Results logged to /var/lib/openshift/53231afcd9fa3c18aa00010c/app-root/runtime/repo/vendor/bundle/ruby/1.8/gems/rmagick-2.13.2/ext/RMagick/gem_make.out
remote: An error occured while installing rmagick (2.13.2), and Bundler cannot continue.
remote: Make sure that `gem install rmagick -v '2.13.2'` succeeds before bundling.
remote: An error occurred executing 'gear postreceive' (exit code: 5)
remote: Error message: CLIENT_ERROR: Failed to execute: 'control build' for /var/lib/openshift/53231afcd9fa3c18aa00010c/ruby
remote:
remote: For more details about the problem, try running the command again with the '--trace' option.
To ssh://53231afcd9fa3c18aa00010c.com.cn/~/git/redminejwl.git/
   cb5f897..c3c4238  master -> master

Created attachment 874641 [details]
"successful" git push, with weird stack traces

I was finally able to reproduce this by doubling the memory limits for the default node profile. I 'fixed' it by ssh-ing into the gear, moving the Gemfile.lock to Gemfile.lock.orig, and running 'bundle install --no-deployment'. After that, I added the force_clean_build marker to the git repo, ran git push, and the app seems to have successfully deployed. If you look at the attached log ( https://bugzilla.redhat.com/attachment.cgi?id=874641 ), however, you'll see that there are several stack dumps in the output from git push. I haven't had enough time to look into these now, but offhand it looks like the ruby 1.9.3 interpreter is trying to load libraries compiled for ruby-1.8...

(In reply to John W. Lamb from comment #11)
> On your broker, I still can't reproduce the issue, it's blocking at the
> rmagick gem:
> 
The version of ose I gave you is 2.1, there are two nodes, one node is small on which recommended metapackages is installed. Another is medium on which optinal metapackages is installed. 

By default, gear size is small, will be deployed to small node, so it is blocking at the rmagick gem. 

I think you just need add "-g medium" to deploy it on medium node, then will not encounter the rmagick gem dependency issue, and you will reproduce this bug.

Referenced gems in Gemfile.lock.ruby-1.8 seemed to be activating the incorrect version of thor - updated Gemfile.lock.ruby-1.8 in https://github.com/openshift/openshift-redmine-quickstart/pull/3 fixes the issue. PR has already been merged into https://github.com/openshift/openshift-redmine-quickstart

Retest this bug with latest redmine quickstart, still failed.

1. Set up ose-2.1 env, make sure recommended + optional metapackage are installed on node.
2. rhc app create redmine ruby-1.8 mysql-5.1 -g medium
3. cd redmine; git remote add upstream -m master git://github.com/openshift/openshift-redmine-quickstart.git; git pull -s recursive -X theirs upstream master; git push

---OUTPUT---
<--snip-->
remote: Installing rmagick (2.13.2) with native extensions ................................................................................
remote: ..................
remote: .
remote: 
remote: Installing sqlite3 (1.3.9) with native extensions ....
remote: ....................
remote: .
remote: 
remote: Installing yard (0.8.7.3) 
remote: Your bundle is complete! It was installed into ./vendor/bundle
remote: Preparing build for deployment
remote: Deployment id is 05d30228
remote: Activating deployment
remote: Starting MySQL 5.1 cartridge
remote: Compilation of assets is disabled or assets not detected.
remote: Generating secret token
remote: /usr/lib/ruby/gems/1.8/gems/bundler-1.0.21/lib/bundler/runtime.rb:31:in `setup': You have already activated thor 0.14.6, but your Gemfile requires thor 0.18.1. Using bundle exec may solve this. (Gem::LoadError)
remote: 	from /usr/lib/ruby/gems/1.8/gems/bundler-1.0.21/lib/bundler/runtime.rb:17:in `setup'
remote: 	from /usr/lib/ruby/gems/1.8/gems/bundler-1.0.21/lib/bundler.rb:110:in `setup'
remote: 	from /usr/lib/ruby/gems/1.8/gems/bundler-1.0.21/lib/bundler/cli.rb:340:in `exec'
remote: 	from /usr/lib/ruby/gems/1.8/gems/thor-0.14.6/lib/thor/task.rb:22:in `send'
remote: 	from /usr/lib/ruby/gems/1.8/gems/thor-0.14.6/lib/thor/task.rb:22:in `run'
remote: 	from /usr/lib/ruby/gems/1.8/gems/thor-0.14.6/lib/thor/invocation.rb:118:in `invoke_task'
remote: 	from /usr/lib/ruby/gems/1.8/gems/thor-0.14.6/lib/thor.rb:263:in `dispatch'
remote: 	from /usr/lib/ruby/gems/1.8/gems/thor-0.14.6/lib/thor/base.rb:389:in `start'
remote: 	from /usr/lib/ruby/gems/1.8/gems/bundler-1.0.21/bin/bundle:13
remote: 	from /usr/bin/bundle:19:in `load'
remote: 	from /usr/bin/bundle:19
remote: -------------------------
remote: Git Post-Receive Result: failure
remote: Activation status: failure
remote: Activation failed for the following gears:
remote: 5329316f8fcf31688200013b (Error activating gear: CLIENT_ERROR: Failed to execute: 'control start' for /var/lib/openshift/5329316f8fcf31688200013b/ruby)
remote: Deployment completed with status: failure
remote: postreceive failed
To ssh://5329316f8fcf31688200013b.com.cn/~/git/redmine.git/
   72665f0..2d8936c  master -> master

And check Gemfile.lock.ruby-1.8, thor version is not updated, still activating the incorrect version of thor.

https://github.com/openshift/openshift-redmine-quickstart/blob/master/Gemfile.lock.ruby-1.8#L89

Should actually be fixed now with https://github.com/openshift/openshift-redmine-quickstart/pull/4

Verified this bug with the above quick again 2.1/2014-03-20.2 puddle and online devenv_4545, redmine app is deployed successfully.