Bug 698405 - Review Request: mcollective - A framework to build server orchestration or parallel job execution systems
Summary: Review Request: mcollective - A framework to build server orchestration or pa...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: Package Review
Version: rawhide
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Steve Traylen
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On: 760357
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-04-20 20:07 UTC by Jeffrey C. Ollie
Modified: 2012-05-18 21:38 UTC (History)
5 users (show)

Fixed In Version: mcollective-2.0.0-1.el5
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2012-04-19 05:20:01 UTC
Type: ---
Embargoed:
steve.traylen: fedora-review+
gwync: fedora-cvs+


Attachments (Terms of Use)

Description Jeffrey C. Ollie 2011-04-20 20:07:34 UTC
Spec URL: http://jcollie.fedorapeople.org/mcollective.spec
SRPM URL: http://jcollie.fedorapeople.org/mcollective-1.1.3-3.fc15.src.rpm
Description:
The Marionette Collective is a framework to build server orchestration
or parallel job execution systems.

Comment 1 Steve Traylen 2011-08-18 19:18:28 UTC
rpmlint errors as below, with the exception of "no-manual-page-for-binary" I expect the rest can 
possibly be fixed.

I'll look some more now.


SPECS/mcollective.spec:7: W: non-standard-group System Tools
SPECS/mcollective.spec:36: W: non-standard-group System Tools
SPECS/mcollective.spec:50: W: non-standard-group System Tools
SPECS/mcollective.spec:121: W: mixed-use-of-spaces-and-tabs (spaces: line 73, tab: line 121)

mcollective.noarch: W: non-standard-group System Tools
mcollective.noarch: W: invalid-license Apache License, Version 2
mcollective.noarch: E: non-readable /etc/mcollective/server.cfg 0440L
mcollective.noarch: W: no-manual-page-for-binary mcollectived
mcollective-client.noarch: W: non-standard-group System Tools
mcollective-client.noarch: W: invalid-license Apache License, Version 2
mcollective-client.noarch: W: conffile-without-noreplace-flag /etc/mcollective/rpc-help.erb
mcollective-client.noarch: W: no-manual-page-for-binary mc-controller
mcollective-client.noarch: W: no-manual-page-for-binary mc-rpc
mcollective-client.noarch: W: no-manual-page-for-binary mc-find-hosts
mcollective-client.noarch: W: no-manual-page-for-binary mc-inventory
mcollective-client.noarch: W: no-manual-page-for-binary mc-facts
mcollective-client.noarch: W: no-manual-page-for-binary mc-call-agent
mcollective-client.noarch: W: no-manual-page-for-binary mc-ping
mcollective-common.noarch: W: non-standard-group System Tools
mcollective-common.noarch: W: invalid-license Apache License, Version 2
mcollective.src: W: non-standard-group System Tools
mcollective.src: W: invalid-license Apache License, Version 2
mcollective.src:121: W: mixed-use-of-spaces-and-tabs (spaces: line 73, tab: line

Comment 2 Steve Traylen 2011-08-18 19:45:00 UTC
1) You have a few mentions of '/etc/init.d' which should be /etc/rc.d/init.d
   and in fact should be %{_initrddir}. You have to set this if you 
   are targeting EPEL5 as well (yes please).

2) Given that mcollective and mcollective-client requires mcollective-common
   I see no need to duplicate the 

    Requires: ruby
    Requires: rubygems
    Requires: rubygem-stomp

   on these packages.

3) Version 1.3.0 looks to be the newest version.

4) You have 
    Requires: rubygem(stomp)
   and 
    Requires: rubygem-stomp
   in some places. 
   See ruby guidelines:
   
5) The ruby(abi) requires should be defined.
   http://fedoraproject.org/wiki/Packaging:Ruby

6) Notice how ruby_sitelib is defined here.
   http://fedoraproject.org/wiki/Packaging:Ruby

Comment 3 Steve Traylen 2011-11-12 16:13:57 UTC
ping

Comment 4 Jeffrey C. Ollie 2011-11-21 13:58:22 UTC
Spec URL: http://jcollie.fedorapeople.org/mcollective.spec
SRPM URL: http://jcollie.fedorapeople.org/mcollective-1.3.1-2.fc16.src.rpm

Yes, I will be maintaining branches for EPEL in addition to Fedora.

Comment 5 Steve Traylen 2011-11-30 23:38:29 UTC
All points above look to be fixed, thanks.

The SPEC file just does a unpack and copy things in to the correct place
so the review is pretty simple.

A build and rpmlint shows

./mcollective.spec:179: E: files-attr-not-set

if you really plan to target EPEL5 this should be added back.

More significant:

mcollective.noarch: E: non-readable /etc/mcollective/server.cfg 0440L
mcollective.noarch: W: service-default-enabled /etc/rc.d/init.d/mcollective
mcollective.noarch: W: service-default-enabled /etc/rc.d/init.d/mcollective
mcollective-client.noarch: W: conffile-without-noreplace-flag /etc/mcollective/rpc-help.erb

Checking quickly all of these they can and should be fixed.

This is other wise looking fairly good, licensing all looks fine.

Comment 6 Jeffrey C. Ollie 2011-12-01 14:42:03 UTC
(In reply to comment #5)
>
> ./mcollective.spec:179: E: files-attr-not-set
> 
> if you really plan to target EPEL5 this should be added back.

Fixed.

> More significant:
> 
> mcollective.noarch: E: non-readable /etc/mcollective/server.cfg 0440L

This is not world-readable deliberately as it may contain sensitive passwords.  The client.cfg should be that way as well.  If the admin wants to open up access later it's probably best to use an ACL to give access to specific users.

> mcollective.noarch: W: service-default-enabled /etc/rc.d/init.d/mcollective
> mcollective.noarch: W: service-default-enabled /etc/rc.d/init.d/mcollective
> mcollective-client.noarch: W: conffile-without-noreplace-flag
> /etc/mcollective/rpc-help.erb

These should all be fixed.

Spec URL: http://jcollie.fedorapeople.org/mcollective.spec
SRPM URL: http://jcollie.fedorapeople.org/mcollective-1.3.1-3.fc16.src.rpm

Comment 7 Steve Traylen 2011-12-05 19:16:12 UTC
 +:ok, =:needs attention, -:needs fixing

MUST Items:
[=] MUST: rpmlint must be run on every package.

$ rpmlint ./mcollective.spec 
./mcollective.spec:164: E: files-attr-not-set
./mcollective.spec:173: E: files-attr-not-set
./mcollective.spec:181: E: files-attr-not-set
0 packages and 1 specfiles checked; 3 errors, 0 warnings.

Can you swap the lines:
%doc COPYING
%defattr(-,root,root,-)

to get rid of this.

$ rpmlint ./*.rpm

This is still present:
mcollective.noarch: W: service-default-enabled /etc/rc.d/init.d/mcollective

See

$ rpmlint  -I service-default-enabled

[+] MUST: The package must be named according to the Package Naming Guidelines.
[+] MUST: The spec file name must match the base package %{name}
[+] MUST: The package must be licensed with a Fedora approved license and meet the Licensing Guidelines.
ASL 2.0
[+] MUST: The License field in the package spec file must match the actual license.
Licensing is very clear.
[+] MUST: If (and only if) the source package includes the text of the license(s) in its own file, then that file, containing the text of the license(s) for the package must be included in %doc.
COPYING
[+] MUST: The spec file must be written in American English.
[+] MUST: The spec file for the package MUST be legible.
[+] MUST: The sources used to build the package must match the upstream source, as provided in the spec URL.
$ md5sum mcollective-1.3.1.tgz ../SOURCES/mcollective-1.3.1.tgz 
c63901134ea66973702720861f020d3a  mcollective-1.3.1.tgz
c63901134ea66973702720861f020d3a  ../SOURCES/mcollective-1.3.1.tgz


[+] MUST: The package must successfully compile and build into binary rpms on at least one supported architecture..
[+] MUST: A package must own all directories that it creates. If it does not create a directory that it uses, then it should require a package which does create that directory.
[+] MUST: A package must not contain any duplicate files in the %files listing.
[+] MUST: Permissions on files must be set properly. Executables should be set with executable permissions, for example. Every %files section must include a %defattr(...) line.
[+] MUST: Each package must have a %clean section, which contains rm -rf %{buildroot} (or $RPM_BUILD_ROOT).
[+] MUST: Each package must consistently use macros, as described in the macros section of Packaging Guidelines.
[] MUST: The package must contain code, or permissible content. This is described in detail in the code vs. content section of Packaging Guidelines.
[+] MUST: Large documentation files should go in a doc subpackage.
[+] MUST: In the vast majority of cases, devel packages must require the base package using a fully versioned dependency: Requires: %{name} = %{version}-%{release} 
[+] MUST: Packages must not own files or directories already owned by other packages.
[+] MUST: At the beginning of %install, each package MUST run rm -rf %{buildroot} (or $RPM_BUILD_ROOT).
[+] MUST: All filenames in rpm packages must be valid UTF-8.

SHOULD Items:
[+] SHOULD: The reviewer should test that the package builds in mock.

[to be checked.] SHOULD: The reviewer should test that the package functions as described.
[+] SHOULD: If scriptlets are used, those scriptlets must be sane.
[=] SHOULD: Packages should try to preserve timestamps of original installed files.

You missed a few '-p' on the last installs:

%{__install} -m0640 etc/server.cfg.dist %{buildroot}/etc/mcollective/server.cfg
%{__install} -m0640 etc/client.cfg.dist %{buildroot}/etc/mcollective/client.cfg
%{__install} -m0644 etc/facts.yaml.dist %{buildroot}/etc/mcollective/facts.yaml
%{__install} -m0644 etc/rpc-help.erb %{buildroot}/etc/mcollective/rpc-help.erb


Looking good generally, let me have a go at installing and running it.

Steve.

Comment 8 Steve Traylen 2011-12-05 20:10:36 UTC
Just installed rabbitmq and ran mcollective okay.

# mc-find-hosts 
herb.home

The deamon starts  up fine and creates the logfile

/var/log/mcollective.log

is there something that rotates and deletes this log file eventually?


I see the package contains

http://github.com/ahoward/systemu

I think this should be packaged separately?

Steve.

Comment 9 Jeffrey C. Ollie 2011-12-05 21:17:20 UTC
Spec URL: http://jcollie.fedorapeople.org/mcollective.spec
SRPM URL: http://jcollie.fedorapeople.org/mcollective-1.3.1-4.fc16.src.rpm

OK, this should take care of the remaining rpmlint problems.  mcollective manages it's logfile itself.  See the documentation for the keeplogs and max_log_size parameters:

http://docs.puppetlabs.com/mcollective/reference/basic/configuration.html

I'll work on getting a package review of systemu up ASAP.

Comment 10 Jeffrey C. Ollie 2011-12-05 21:50:46 UTC
systemu review request here:

https://bugzilla.redhat.com/show_bug.cgi?id=760357

Comment 11 Vít Ondruch 2011-12-06 13:42:52 UTC
Could you please replace the Perl call with either sed, awk or some nice ruby oneliner [1]? I understand that Perl is just tool to do the job and this may sound ridiculous, but:

1) This is Ruby package
2) There are attempts to remove Perl from minimal build root, so you'll save yourself from troubles in future.


[1] http://benoithamelin.tumblr.com/ruby1line

Comment 12 Jeffrey C. Ollie 2011-12-06 21:35:45 UTC
(In reply to comment #11)
> Could you please replace the Perl call with either sed, awk or some nice ruby
> oneliner [1]? I understand that Perl is just tool to do the job and this may
> sound ridiculous, but:

Actually, I've switched to using a pure patch to do the work...

Comment 13 Jeffrey C. Ollie 2011-12-06 21:38:09 UTC
Spec URL: http://jcollie.fedorapeople.org/mcollective.spec
SRPM URL: http://jcollie.fedorapeople.org/mcollective-1.3.1-5.fc16.src.rpm

Switches to use a patch to remove vendor libraries and to disable service by default.  Removes internal systemu library and requires systemu rubygem.

Comment 14 Vít Ondruch 2011-12-07 12:29:16 UTC
Nice try, thank you. Unfortunately I can see another two perl lines still present in %build section.

Comment 15 Jeffrey C. Ollie 2011-12-07 17:20:22 UTC
(In reply to comment #14)
> Nice try, thank you. Unfortunately I can see another two perl lines still
> present in %build section.

Doh! Completely forgot about those.

Spec URL: http://jcollie.fedorapeople.org/mcollective.spec
SRPM URL: http://jcollie.fedorapeople.org/mcollective-1.3.1-6.fc16.src.rpm

Comment 16 Steve Traylen 2011-12-09 13:57:24 UTC
All comments were addressed, 

I also cringed at the %perl statements in there but could not give a reason for them to
go. I'm glad they have :-)

APPROVED

Comment 17 Jeffrey C. Ollie 2011-12-09 14:57:18 UTC
(In reply to comment #16)
> 
> I also cringed at the %perl statements in there but could not give a reason for
> them to
> go. I'm glad they have :-)

Yeah using Perl is like trying to pound a nail with a sledgehammer but I'm usually better at writing regexes with Perl than sed, especially if they are complex.  These were trivial so it was easy to replace them.

Comment 18 Jeffrey C. Ollie 2011-12-09 15:20:35 UTC
New Package SCM Request
=======================
Package Name: mcollective
Short Description: A framework to build server orchestration or parallel job execution systems
Owners: jcollie
Branches: f16 f15 el6 el5
InitialCC:

Comment 19 Gwyn Ciesla 2011-12-09 15:37:19 UTC
Git done (by process-git-requests).

Comment 20 Fedora Update System 2011-12-09 20:55:34 UTC
mcollective-1.3.1-6.el6 has been submitted as an update for Fedora EPEL 6.
https://admin.fedoraproject.org/updates/mcollective-1.3.1-6.el6

Comment 21 Fedora Update System 2011-12-09 20:57:04 UTC
mcollective-1.3.1-6.el5 has been submitted as an update for Fedora EPEL 5.
https://admin.fedoraproject.org/updates/mcollective-1.3.1-6.el5

Comment 22 Steve Traylen 2011-12-09 23:42:43 UTC
Hi Jeffrey,
 
  It's your call but does it make sense to release with 1.3.2 at this point
  since just released.

Steve.

Comment 23 Fedora Update System 2011-12-10 21:23:36 UTC
Package mcollective-1.3.1-6.el6:
* should fix your issue,
* was pushed to the Fedora EPEL 6 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=epel-testing mcollective-1.3.1-6.el6'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-EPEL-2011-5182/mcollective-1.3.1-6.el6
then log in and leave karma (feedback).

Comment 24 Fedora Update System 2011-12-14 20:18:22 UTC
rubygem-systemu-2.4.1-2.fc15,mcollective-1.3.1-6.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/rubygem-systemu-2.4.1-2.fc15,mcollective-1.3.1-6.fc15

Comment 25 Fedora Update System 2011-12-16 16:04:43 UTC
mcollective-1.3.1-6.fc16,rubygem-systemu-2.4.1-2.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/mcollective-1.3.1-6.fc16,rubygem-systemu-2.4.1-2.fc16

Comment 26 Steve Traylen 2012-04-16 14:33:52 UTC
Hi,
As this is not released yet can we update this to the current 1.3.3 or are you waiting
for mcollective 1.3 to be declared stable?

https://groups.google.com/forum/?fromgroups#!topic/mcollective-users/h2M07IhL-Ug

Comment 27 Jeffrey C. Ollie 2012-04-17 20:11:50 UTC
I pushed 1.3.1 to stable for F15 & F16 and I built 1.3.3 for rawhide.  Unfortunately in the time that has passed since I last worked on this I've discovered that Puppet and related technologies like mcollective aren't what I want to use to manage my servers any more so I should probably orphan mcollective and find a new maintainer.

Comment 28 Steve Traylen 2012-04-18 07:03:27 UTC
Hi Jeffrey,

 I am happy to take over, let me know when you orphan.

Steve.

Comment 29 Jeffrey C. Ollie 2012-04-18 12:48:52 UTC
OK, I've released ownership.

Comment 30 Fedora Update System 2012-04-19 05:20:01 UTC
rubygem-systemu-2.4.1-2.fc15, mcollective-1.3.1-6.fc15 has been pushed to the Fedora 15 stable repository.

Comment 31 Fedora Update System 2012-04-19 05:24:41 UTC
mcollective-1.3.1-6.fc16, rubygem-systemu-2.4.1-2.fc16 has been pushed to the Fedora 16 stable repository.

Comment 32 Fedora Update System 2012-05-18 21:37:11 UTC
mcollective-2.0.0-1.el6 has been pushed to the Fedora EPEL 6 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 33 Fedora Update System 2012-05-18 21:38:15 UTC
mcollective-2.0.0-1.el5 has been pushed to the Fedora EPEL 5 stable repository.


Note You need to log in before you can comment on or make changes to this bug.