Bug 1010406

Summary:	removal of lib/bundler/vendor in rubygem-bundler produces a broken bundler gem in 1.3+
Product:	[Fedora] Fedora EPEL	Reporter:	Allen Hewes <allen>
Component:	rubygem-bundler	Assignee:	Vít Ondruch <vondruch>
Status:	CLOSED NOTABUG	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	el5	CC:	bkabrda, jstribny, mtasaka, shreyankg, vondruch
Target Milestone:	---
Target Release:	---
Hardware:	i386
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2013-10-01 20:26:34 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:

Description Allen Hewes 2013-09-20 17:33:19 UTC

Description of problem:
Bundler can't run more complex commands because lib/bundler/vendor is being removed during the packaging process

Version-Release number of selected component (if applicable):
so far, every version from 1.3+. But I am sure it happens in any version of bunder where lib/bundler/vendor has been removed during packaging.


How reproducible:
100%


Steps to Reproduce:
1. install rubygem-bundler as packaged by Fedora
2. run bundle command with arguments --gemfile, --system, --without
3.

Actual results:
bundle install --gemfile /home/apps/cap_deploys/app/portal/releases/20130920170819/Gemfile --system --without development test
/usr/share/gems/gems/bundler-1.3.1/lib/bundler/cli.rb:20:in `<class:CLI>': undefined method `stop_on_unknown_option!' for Bundler::CLI:Class (NoMethodError)
from /usr/share/gems/gems/bundler-1.3.1/lib/bundler/cli.rb:6:in `<module:Bundler>'
from /usr/share/gems/gems/bundler-1.3.1/lib/bundler/cli.rb:5:in `<top (required)>'
from /usr/share/rubygems/rubygems/custom_require.rb:36:in `require'
from /usr/share/rubygems/rubygems/custom_require.rb:36:in `require'
from /usr/share/gems/gems/bundler-1.3.1/bin/bundle:18:in `<top (required)>'
from /usr/bin/bundle:23:in `load'
from /usr/bin/bundle:23:in `<main>'



Expected results:
bundle install --gemfile /home/apps/cap_deploys/app/portal/releases/20130920171832/Gemfile --system --without development test
Using rake (10.1.0)
Using i18n (0.6.1)
Using multi_json (1.7.7)
Using activesupport (3.2.13)
Using builder (3.0.4)
Using activemodel (3.2.13)
Using erubis (2.7.0)
Using journey (1.0.4)
Using rack (1.4.5)
Using rack-cache (1.2)
Using rack-test (0.6.2)
Using hike (1.2.2)
Using tilt (1.4.1)
Using sprockets (2.2.2)
Using actionpack (3.2.13)
Using mime-types (1.23)
Using polyglot (0.3.3)
Using treetop (1.4.12)
Using mail (2.5.3)
Using actionmailer (3.2.13)
Using arel (3.0.2)
Using tzinfo (0.3.37)
Using activerecord (3.2.13)
Using <REDACTED GEM>
Using activeresource (3.2.13)
Using phoney (0.1.1)
Using <REDACTED GEM>
Using <REDACTED GEM>
Using chunky_png (1.2.5)
Using coffee-script-source (1.2.0)
Using execjs (1.3.2)
Using coffee-script (2.2.0)
Using rack-ssl (1.3.3)
Using json (1.8.0)
Using rdoc (3.12.2)
Using thor (0.14.6)
Using railties (3.2.13)
Using coffee-rails (3.2.2)
Using fssm (0.2.8.1)
Using sass (3.1.15)
Using compass (0.12.1)
Using bundler (1.3.1)
Using rails (3.2.13)
Using <REDACTED GEM>
Using daemons (1.1.9)
Using <REDACTED GEM>
Using values (1.5.0)
Using <REDACTED GEM>
Using dynamic_form (1.1.4)
Using eventmachine (1.0.3)
Using excon (0.15.4)
Using flash_hash_request_uuid (0.0.2)
Using formatador (0.2.3)
Using net-ssh (2.5.2)
Using net-scp (1.0.4)
Using nokogiri (1.5.9)
Using ruby-hmac (0.4.0)
Using fog (1.5.0)
Using geokit (1.6.5)
Using geokit-rails3 (0.1.5)
Using haml (3.2.0.beta.1)
Using highline (1.6.11)
Using <REDACTED GEM>
Using <REDACTED GEM>
Using jquery-rails (2.0.2)
Using jquery-ui-rails (0.2.2)
Using liquid (2.3.0)
Using <REDACTED GEM>
Using mail_safe (0.3.1)
Using <REDACTED GEM>
Using memcache-client (1.8.5)
Using minitest (2.12.1)
Using newrelic_rpm (3.6.4.122)
Using will_paginate (3.0.3)
Using paginated_table (0.0.6)
Using progressbar (0.10.0)
Using prototype-rails (3.2.1)
Using rack-protection (1.2.0)
Using rails-i18n (0.7.4)
Using redis (2.2.2)
Using redis-namespace (1.0.3)
Using ref (1.0.0)
Using sinatra (1.3.2)
Using vegas (0.1.11)
Using resque (1.20.0)
Using rufus-scheduler (2.0.16)
Using resque-scheduler (1.9.9)
Using safe_yaml (0.9.1)
Using sanitize (2.0.3)
Using sass-rails (3.2.5)
Using <REDACTED GEM>
Using <REDACTED GEM>
Using <REDACTED GEM>
Using strong_parameters (0.2.0)
Using tenacity (0.6.0)
Using therubyracer (0.11.0beta8)
Using thin (1.5.0)
Using <REDACTED GEM>
Using <REDACTED GEM>
Using typhoeus (0.3.3)
Using uglifier (1.2.4)
Using <REDACTED GEM>
Using <REDACTED GEM>
Your bundle is complete! Use `bundle show [gemname]` to see where a bundled gem is installed.


Additional info:
I understand the issues at work here; but the alternative is a much better solution: stop producing an RPM of bundler that's broken when folks think they have a working bundler installed. Install and use a bundler that is installed via gem and not yum/rpm.

Comment 1 Mamoru TASAKA 2013-09-20 20:10:58 UTC

Umm? F-18 latest rubygem-bundler is 1.1.4-2.fc18. Is this really against F-18 rubygem-bundler report?

(The reason for my question is that your log shows:
/usr/share/gems/gems/bundler-1.3.1/lib/bundler/cli.rb:20:in `<class:CLI>': undefined method `stop_on_unknown_option!' for Bundler::CLI:Class (NoMethodError)
)

Comment 2 Allen Hewes 2013-09-20 20:16:33 UTC

Hi Mamoru,

Yeah, no. Technically, it's RHEL 5. I just choose F18 because that's my work machine and I didn't know what else to create it under. But OS, doesn't change the fact that the removal of "lib/bundler/vendor" produces a broken bundler.

I use Fedora 18 with mock to "produce" my packages, then I use Koji running on RHEL 5 to cook them and mash them. My production servers are all RHEL 5.

Comment 3 Mamoru TASAKA 2013-09-20 20:23:17 UTC

Then this bug report must be against RHEL (or EPEL) 5, not for Fedora 18. Your report does not mean that "Fedora's" bundler is broken.

I don't know the current status for RHEL 5 / EPEL 5, however anyway switching target.

Comment 4 Mamoru TASAKA 2013-09-20 20:30:30 UTC

Well, I may be misunderstanding, would you explain your situation clearer?

- So is it correct that you are not running Fedora 18 when you use bundler
  but using RHEL5?
- and then what rubygem-bundler rpm you are actually using? And is the rpm
  really targetted for the OS / version you are using?

Comment 5 Mamoru TASAKA 2013-09-20 20:34:14 UTC

(The reason I am asking this is that I cannot find rubygem-bundler build for EPEL 5:
http://koji.fedoraproject.org/koji/packageinfo?packageID=11569 )

Comment 6 Mamoru TASAKA 2013-09-20 20:40:50 UTC

For now one more question: "what" rubygem-thor rpm is installed?

Comment 7 Allen Hewes 2013-09-20 22:00:51 UTC

Hi Mamoru,

I think that debian bug report is it. I went up to 0.16 or some such for thor and it still failed for me. I went back to using rubygem-thor-0.14.6 after I changed my rubygem-bundler-1.3.1 to leave the "lib/bundler/vendor" directory in tact. I have a hard requirement to use bundler 1.3+, so it's been a few months since I looked at this.

I suspect that I completely missed the thor update in Feodra two days prior to bundler being bumped to 1.3.1.

I will update my stuff and let you guys know. I know for sure that I didn't try thor 0.17+.

I think I'm gonna get some egg on my face...

Comment 8 Allen Hewes 2013-09-24 19:45:30 UTC

Hi Mamoru,

Thanks for the pointer. I spent some time reading all of the commits and found what you (Vit on ruby-sig) have been talking about. I actually think I found another problem; net-http-persistent. I've had HTTP connect issues on and off with gem.fury.io and other SSL enabled bundler endpoints. Ugh.

Should the bundler packaging and gemspec be modified to require the proper "gemed" versions of thor and net-http-persistent? Maybe even patch the bundler files?

Thanks again, I'm sorry for wasting your time.

Comment 9 Mamoru TASAKA 2013-09-25 01:44:27 UTC

(In reply to Allen Hewes from comment #8)
> Should the bundler packaging and gemspec be modified to require the proper
> "gemed" versions of thor and net-http-persistent? Maybe even patch the
> bundler files?

My usual thought for this is that
- If the lowest EVR (Epoch-Version-Relase) of the package "foo" shipped on a
  stable branch satisfies the actual dependency for the package "bar",
  there is no need for writing explicit version dependency for "foo"
  on "bar" spec file, otherwise explicit version dependency is needed.

So for example 
- the _highest_ EVR of rubygem-bundler currently being shipped on F-19 
  is rubygem-bundler-1.3.1-1.fc19. which requires 
  thor >= 0.17.0(?) and net-http-persistent (the least version I don't know).
- The _lowest_ EVR of rubygem-thor being shipped on F-19 is
  rubygem-thor-0.17.0-1.fc19, which satisfies thor >= 0.17.0, so
  _IN MY OPINION_ explicit version dependency need not be written on
  rubygem-bundler spec file.

Note that there is Fedora packaging guideline
https://fedoraproject.org/wiki/Packaging:Guidelines?rd=Packaging/Guidelines#Explicit_Requires
, especially the line

'''
For instance in the example above, when no current Fedora release shipped with libfubar < 1.2.3-7, it is no longer necessary to list the explicit, versioned requirement. 
'''
, which reads that if the lowest EVR of _the lowest stable branch_ (currently
F-18) of the package "foo" does not satisfy the requirement for package bar currently shipped on stable branch, explicit version Requires is needed, however
in my opinion this is overkill. Users are not expected to use mixed rpms from both F-18 and F-19, for example.

Comment 10 Allen Hewes 2013-09-30 19:17:43 UTC

Hi Mamoru,

Yep, I know about the guidelines.

I should have asked if there are any plans to update bundler within Fedora releases. Which would make the position I was taking valid; bundler updates within a Fedora release (should) might have to have hard Requires/BuildRequires because of what's going on upstream.

Here's what I found for bundler 1.3.1 onwards:

bundler 1.3.1 onwards
  thor 0.16.0 with
    efc387cb726f3563244a880401b7b7f0c64fad07
    88b9f4bb78625eb1c6bf3bf287c551a4759ef9d2
  net-http-persistent 2.8 with
    8db9c0a63d2b804509225696be9c102b3d50c825

So it looks like a gemed version of thor that is at least 0.17.0 will satisfy bundler 1.3.1-1.3.5 versions, bundler 1.4 is just around the corner.

The net-http-persistent commit is for "cleanup trailing whitespace".

So this build never made it:
http://koji.fedoraproject.org/koji/buildinfo?buildID=394719

I don't know enough about Fedora's koji tags to know what "f19-rebuild" means, i.e. if this package landed in the normal yum repositories.

Comment 11 Mamoru TASAKA 2013-10-01 01:22:16 UTC

Well, koji list tag shows:

$ TZ=GMT koji list-tag-history --build=rubygem-thor-0.16.0-3.fc19
Mon Feb 18 06:19:53 2013: rubygem-thor-0.16.0-3.fc19 tagged into f19-rebuild by ausil [still active]
Tue Feb 19 13:40:47 2013: rubygem-thor-0.16.0-3.fc19 tagged into f19 by ausil
Wed Mar 20 18:25:19 2013: rubygem-thor-0.16.0-3.fc19 untagged from f19 by ausil

So I guess rubygem-thor-0.16.0-3.fc19 was in F-19 buildroot from
2013-02-19 to 2013-03-05 (then next thor was built), at this stage
F-19 was not released, so for now I say it is okay.

Then:
(In reply to Allen Hewes from comment #10)
> Hi Mamoru, 
> Yep, I know about the guidelines.
> 
> I should have asked if there are any plans to update bundler within Fedora
> releases. Which would make the position I was taking valid; bundler updates
> within a Fedora release (should) might have to have hard
> Requires/BuildRequires because of what's going on upstream.
> 
> Here's what I found for bundler 1.3.1 onwards:
> 
> bundler 1.3.1 onwards
>   thor 0.16.0 with
>     efc387cb726f3563244a880401b7b7f0c64fad07
>     88b9f4bb78625eb1c6bf3bf287c551a4759ef9d2
>   net-http-persistent 2.8 with
>     8db9c0a63d2b804509225696be9c102b3d50c825
> 
> So it looks like a gemed version of thor that is at least 0.17.0 will
> satisfy bundler 1.3.1-1.3.5 versions, bundler 1.4 is just around the corner.

This is up to how Vít thinks, however generally speaking breaking API / ABI
on stable branch is forbidden, so (unless it is guaranteed or maintainer
verifies that it won't change API or so), a kind of upgrade from 1.3.x to
1.4 is not allowed:

http://fedoraproject.org/wiki/Updates_Policy#Philosophy

(Note that some upstream care for versioning not so much and just increase
 version like 1.8 -> 1.9 -> 2.0 -> 2.1 ... , so what "major version upgrade"
 means must be judged per each package)

> The net-http-persistent commit is for "cleanup trailing whitespace".
> 
> So this build never made it:
> http://koji.fedoraproject.org/koji/buildinfo?buildID=394719
> 
> I don't know enough about Fedora's koji tags to know what "f19-rebuild"
> means, i.e. if this package landed in the normal yum repositories.

f19-rebuild (only) tagged packages never landed into yum repositories.
Once mass rebuild was done, a responsible Fedora member tagged 
all f19-rebuild tagged packages as f19 also, then the packages appeared
in F-19 (development) yum repository.

Comment 12 Allen Hewes 2013-10-01 20:26:34 UTC

Mamoru,

Thanks for all the information (and patience)!

Closing.