Bug 1130964 - wget dependency is not described
Summary: wget dependency is not described
Status: CLOSED ERRATA
Alias: None
Product: Fedora EPEL
Classification: Fedora
Component: heat-cfntools
Version: el6
Hardware: All
OS: Linux
unspecified
low
Target Milestone: ---
Assignee: Jeff Peeler
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Keywords:
Depends On:
Blocks: 1131151 1136986 1136987 1138864 1138865 1140412 1147088
TreeView+ depends on / blocked
 
Reported: 2014-08-18 09:52 UTC by Yukinori Sagara
Modified: 2014-12-13 09:39 UTC (History)
4 users (show)

(edit)
Clone Of:
: 1131151 (view as bug list)
(edit)
Last Closed: 2014-09-24 03:43:37 UTC


Attachments (Terms of Use)

Description Yukinori Sagara 2014-08-18 09:52:48 UTC
heat-cfntools uses 'wget' command in cfn_helper.py, but wget dependency is not described in spec file.
wget is not installed sometimes. (e.g. in the case of a minimum OS installation.) We should write it.

The following is the relevant section of the code.
* cfntools/cfn_helper.py
1066         url = 'http://169.254.169.254/openstack/2012-08-10/meta_data.json'
1067         if not os.path.exists(cache_path):
1068             CommandRunner('wget -O %s %s' % (cache_path, url)).run()
1069         try:

* heat-cfntools.spec
  8 Source0: https://pypi.python.org/packages/source/h/heat-cfntools/%{name}-%{version}.tar.gz
  9 
 10 BuildArch: noarch
 11 BuildRequires: python2-devel
 12 BuildRequires: python-setuptools
 13 BuildRequires: python-pbr >= 0.5.20
 14 
 15 Requires: python-boto >= 2.5.2
 16 Requires: python-psutil
 17 Requires: python-argparse
 18 
 19 %description

Comment 1 Yukinori Sagara 2014-08-18 09:56:51 UTC
I missed to write version. I verified it on heat-cfntools-1.2.6-2.el6.src.rpm.

Comment 2 Jeff Peeler 2014-08-18 15:57:55 UTC
I guess curl also needs to be added as well. Is there anything else missing? I'll try and check soon.

Comment 3 Steven Dake 2014-08-18 16:29:51 UTC
I didn't see curl in the sourcetree, but I did notice /usr/bin/gem is ran which means the requirements for rubygems package.  I'm not sure how onerous its dep tree is.

Regards
-steve

Comment 4 Yukinori Sagara 2014-08-18 21:00:31 UTC
Thanks Steven and Jeff. I examined also. Please check it.

Require

* python-boto
* wget
* unzip

Require, but not need explicit Require description, because of basic tools

* coreutils
  for su command.
* rpm
* yum
* python-setuptools
* gzip
* bzip2
* tar
* initscripts, chkconfig / systemd
* shadow-utils
  for useradd, groupadd command.

Optional, I could not determine. Please judge.

* dpkg
  heat_cfntools maybe supports multi-distribution, but most user of Red Hat instance doesn't use it.
* rubygems
  rubygems depends on ruby package. and everyone does not require necessarily.
  When you include this, ruby is introduced to all instances using heat.

Not for Red Hat

* apt
  Not for Red Hat, and not packaged yet in EPEL.

> I didn't see curl in the sourcetree

me, too.

Comment 5 Jeff Peeler 2014-08-19 20:58:38 UTC
Thanks for looking further. This is the curl instance I was referring to:

https://github.com/openstack/heat-cfntools/blob/4ece05a829845ccdbf1e78b030719614255bfc28/bin/cfn-signal#L101

I'm not going to include the init system given that the source covers both types on RPM based systems. However, other RPM distros may want to include zypper if it's not installed by default.

Again, I don't think we'd have to worry about native package management tools being present, but Debian distros may want to depend on dpkg and apt-get.

And lastly, support for installing gems is in the source. I think that rather than surprising somebody with yet more unmet dependencies (isn't that what this bug is about?), those who don't want ruby can force uninstall it if they need to trim their image.

Does all this sound reasonable?

Comment 6 Fedora Update System 2014-08-20 16:58:34 UTC
heat-cfntools-1.2.6-3.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/heat-cfntools-1.2.6-3.fc20

Comment 7 Fedora Update System 2014-08-20 16:59:07 UTC
heat-cfntools-1.2.6-3.el6 has been submitted as an update for Fedora EPEL 6.
https://admin.fedoraproject.org/updates/heat-cfntools-1.2.6-3.el6

Comment 8 Zane Bitter 2014-08-20 18:40:11 UTC
I'd actually be OK with leaving out the gems dependency... not all users are going to want to use it. If you're planning to use Ruby in your image you probably will install gems anyway, but if you're not you probably don't want Ruby itself to get pulled in.

Do we know the effect on image size (if any) for e.g. the Fedora Cloud image?

Comment 9 Yukinori Sagara 2014-08-20 21:32:52 UTC
> Thanks for looking further. This is the curl instance I was referring to:
> 
> https://github.com/openstack/heat-cfntools/blob/4ece05a829845ccdbf1e78b030719614255bfc28/bin/cfn-signal#L101

I have missed, sorry.

That means, heat_cfntools currently depends on both wget and curl. It's not good.
I will report OpenStack community.

> I'm not going to include the init system given that the source covers both types on RPM based systems. > > However, other RPM distros may want to include zypper if it's not installed by default.
> 
> Again, I don't think we'd have to worry about native package management tools being present, but Debian > distros may want to depend on dpkg and apt-get.

I agree, too.

> And lastly, support for installing gems is in the source. I think that rather than surprising somebody with yet more unmet dependencies (isn't that what this bug is about?), those who don't want ruby can force uninstall it if they need to trim their image.

This is a little bit difficult. I think both of them (Jeff and Zane wrote) correct.
I have 3 point of view.

1. EPEL packaging policy
   In other package, how treated this problem. Does it needs to contain all dependencies?
   I often find binary software has not linked library some extra feature.

2. Distribution policy
   I use RHEL/CentOS6 usually. ruby is not a mandatory package.
   I don't know recent Fedora. Is ruby used by system tools in recent distribution.

3. heat_cfntools specific problem
   heat_cfntools package install feature (rpm, yum, easy_install, gem, ..)  used in early stages of
   instance setup.  If rubygem is not installed, we need to write manually install command about
   ruby library packages in 'UserData' section. it's dirty to manage heat template.
   (or need to create custom image, which is ruby gem installed)

Comment 10 Yukinori Sagara 2014-08-20 21:52:07 UTC
> Do we know the effect on image size (if any) for e.g. the Fedora Cloud image?

My environment is CentOS6, so It may not be helpful for you.

================================================================================
 Package                Arch         Version                Repository     Size
================================================================================
Installing:
 rubygems               noarch       1.3.7-5.el6            base          207 k
Installing for dependencies:
 compat-readline5       x86_64       5.2-17.1.el6           base          130 k
 ruby                   x86_64       1.8.7.352-13.el6       updates       534 k
 ruby-irb               x86_64       1.8.7.352-13.el6       updates       314 k
 ruby-libs              x86_64       1.8.7.352-13.el6       updates       1.6 M
 ruby-rdoc              x86_64       1.8.7.352-13.el6       updates       377 k

Comment 11 Fedora Update System 2014-08-20 22:59:24 UTC
Package heat-cfntools-1.2.6-3.el6:
* should fix your issue,
* was pushed to the Fedora EPEL 6 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=epel-testing heat-cfntools-1.2.6-3.el6'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-EPEL-2014-2259/heat-cfntools-1.2.6-3.el6
then log in and leave karma (feedback).

Comment 12 Yukinori Sagara 2014-08-20 23:10:12 UTC
> That means, heat_cfntools currently depends on both wget and curl. It's not good.
> I will report OpenStack community.

I reported OpenStack community.

https://bugs.launchpad.net/heat-cfntools/+bug/1359430

Comment 13 Jeff Peeler 2014-08-21 14:41:36 UTC
Zane here's what yum reports using the F20 cloud image:

================================================================================
 Package                 Arch        Version                 Repository    Size
================================================================================
Installing:
 rubygems                noarch      2.1.11-115.fc20         updates      224 k
Installing for dependencies:
 ruby                    x86_64      2.0.0.353-16.fc20       updates       65 k
 ruby-irb                noarch      2.0.0.353-16.fc20       updates       86 k
 ruby-libs               x86_64      2.0.0.353-16.fc20       updates      2.8 M
 rubygem-bigdecimal      x86_64      1.2.0-16.fc20           updates       77 k
 rubygem-io-console      x86_64      0.4.2-16.fc20           updates       48 k
 rubygem-json            x86_64      1.7.7-101.fc20          fedora        60 k
 rubygem-psych           x86_64      2.0.0-16.fc20           updates       75 k
 rubygem-rdoc            noarch      4.0.1-2.fc20            fedora       288 k
 rubypick                noarch      1.1.1-1.fc20            updates      6.3 k

Transaction Summary
================================================================================
Install  1 Package (+9 Dependent packages)

Total download size: 3.7 M
Installed size: 13 M


I'm unsure if this is considered large or not. I still feel like it's up to the image maintainer to trim their image appropriately and we can even explicitly inform the fedora cloud sig to do so if needed, right? Otherwise, I need to rebuild and cancel the above updates.

Comment 14 Steven Dake 2014-08-21 14:48:35 UTC
Since rubygems has a bunch of deps and is pretty large (13mb is pretty large) one option is to change the cfntools to install rubygems if the gem type is presented.  It is unlikely the gem install will be requested, so this would only be slow in the case of rubygems.

Regards
-steve

Comment 15 Jeff Peeler 2014-08-21 15:36:30 UTC
I asked for feedback on the cloud list:

https://lists.fedoraproject.org/pipermail/cloud/2014-August/004176.html

Comment 16 Zane Bitter 2014-08-21 16:05:27 UTC
(In reply to Steven Dake from comment #14)
> Since rubygems has a bunch of deps and is pretty large (13mb is pretty
> large) one option is to change the cfntools to install rubygems if the gem
> type is presented.  It is unlikely the gem install will be requested, so
> this would only be slow in the case of rubygems.

I like this idea in principle... except that it would require distro-specific information to be included in the upstream code (not just which installer to use, but the package name).

Probably the best way is to just require the user to specify the gems package in the yum/apt section whenever they're also specifying gems to install (unless, of course, they have already built an image that includes it). yum/apt packages are guaranteed to be installed before gems, so this should work now.

The aws-cfn-bootstrap package supplied by Amazon (of which heat-cfntools is a clone) does *not* have an explicit Requires on rubygems, so there's no need to add it for compatibility, and in fact precedent for not having it.

Comment 17 Jeff Peeler 2014-08-21 20:16:39 UTC
mattdm responded on IRC and is in favor of not including ruby, assuming that the failure conditions are not catastrophic.  Also, he wasn't even a fan of including wget, so heat-cfntools will need to be changed to use curl instead, which Yukinori seems to be on top of.

I'll go ahead and cancel the package updates I had and get everything sorted out.

Comment 18 Jeff Peeler 2014-08-28 14:25:08 UTC
Waiting for this upstream review to be approved:

https://review.openstack.org/#/c/116143/

Comment 19 Fedora Update System 2014-08-29 17:32:02 UTC
heat-cfntools-1.2.6-4.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/heat-cfntools-1.2.6-4.fc20

Comment 20 Fedora Update System 2014-08-29 17:35:06 UTC
heat-cfntools-1.2.6-4.el6 has been submitted as an update for Fedora EPEL 6.
https://admin.fedoraproject.org/updates/heat-cfntools-1.2.6-4.el6

Comment 22 Fedora Update System 2014-09-02 03:55:16 UTC
Package heat-cfntools-1.2.6-4.el6:
* should fix your issue,
* was pushed to the Fedora EPEL 6 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=epel-testing heat-cfntools-1.2.6-4.el6'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-EPEL-2014-2373/heat-cfntools-1.2.6-4.el6
then log in and leave karma (feedback).

Comment 23 Fedora Update System 2014-09-10 13:31:18 UTC
heat-cfntools-1.2.6-4.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 24 Fedora Update System 2014-09-24 03:43:37 UTC
heat-cfntools-1.2.6-4.el6 has been pushed to the Fedora EPEL 6 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 25 Colin Walters 2014-11-06 18:11:07 UTC
Can you push an update for F21 as well?

Comment 26 Fedora Update System 2014-11-06 19:24:22 UTC
heat-cfntools-1.2.8-1.fc21 has been submitted as an update for Fedora 21.
https://admin.fedoraproject.org/updates/heat-cfntools-1.2.8-1.fc21

Comment 27 Colin Walters 2014-11-06 20:31:25 UTC
See followup in bug 1161290

Comment 28 Fedora Update System 2014-11-11 13:43:25 UTC
heat-cfntools-1.2.8-2.fc21 has been submitted as an update for Fedora 21.
https://admin.fedoraproject.org/updates/heat-cfntools-1.2.8-2.fc21

Comment 29 Fedora Update System 2014-12-13 09:39:21 UTC
heat-cfntools-1.2.8-2.fc21 has been pushed to the Fedora 21 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.