Bug 829528

Summary: Review Request: kyua-cli - Automated testing framework (Command line interface)
Product: [Fedora] Fedora Reporter: Julio Merino <julio>
Component: Package ReviewAssignee: Michel Lind <michel>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: medium    
Version: rawhideCC: michel, notting, package-review
Target Milestone: ---Flags: michel: fedora-review+
gwync: fedora-cvs+
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-06-17 22:26:00 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Julio Merino 2012-06-07 01:08:24 UTC
Spec URL: ftp://ftp.netbsd.org/pub/NetBSD/misc/jmmv/kyua-cli/kyua-cli.spec
SRPM URL: ftp://ftp.netbsd.org/pub/NetBSD/misc/jmmv/kyua-cli/kyua-cli-0.4-1.fc17.src.rpm

Description:

Kyua (pronounced Q.A.) is a testing framework for both developers and
users.  Kyua is different from most other testing frameworks in that it
puts the end user experience before anything else.  There are multiple
reasons for users to run the tests themselves, and Kyua ensures that
they can do so in the most convenient way.

This module, kyua-cli, provides the command-line interface to the Kyua
runtime system.  The major purpose of this tool is to run test cases and
generate unified reports for their results.

Fedora Account System Username: jmmv

Comment 1 Michel Lind 2012-06-07 15:18:19 UTC
Taking this review

Comment 2 Michel Lind 2012-06-08 01:52:51 UTC
Almost ready for approval; there's just a couple of things:
- source URL doesn't match the one downloaded from the listed URL (see checksum below). Please rebuild the srpm and update the link

- could you use 'install -p' instead of install to preserve modification times?

- any reason ATF support is disabled, now that ATF is packaged

Thanks -- look forward to using ATF/Kyua myself in the future.

Package Review
==============

Key:
- = N/A
x = Pass
! = Fail
? = Not evaluated



==== C/C++ ====
[x]: MUST Package does not contain any libtool archives (.la)
[x]: MUST Package does not contain kernel modules.
[x]: MUST Package contains no static executables.
[x]: MUST Rpath absent or only used for internal libs.
[x]: MUST Package is not relocatable.


==== Generic ====
[x]: MUST Package is licensed with an open-source compatible license and meets
     other legal requirements as defined in the legal section of Packaging
     Guidelines.
[x]: MUST Package successfully compiles and builds into binary rpms on at
     least one supported primary architecture.
[x]: MUST %build honors applicable compiler flags or justifies otherwise.
[x]: MUST All build dependencies are listed in BuildRequires, except for any
     that are listed in the exceptions section of Packaging Guidelines.
[x]: MUST Buildroot is not present
     Note: Unless packager wants to package for EPEL5 this is fine
[x]: MUST Package contains no bundled libraries.
[x]: MUST Changelog in prescribed format.
[x]: MUST Package has no %clean section with rm -rf %{buildroot} (or
     $RPM_BUILD_ROOT)
     Note: Clean would be needed if support for EPEL is required
[x]: MUST Sources contain only permissible code or content.
[x]: MUST Each %files section contains %defattr if rpm < 4.4
     Note: Note: defattr macros not found. They would be needed for EPEL5
[x]: MUST Macros in Summary, %description expandable at SRPM build time.
[x]: MUST Package requires other packages for directories it uses.
[x]: MUST Package uses nothing in %doc for runtime.
[x]: MUST Package is not known to require ExcludeArch.
[x]: MUST Permissions on files are set properly.
[x]: MUST Package does not contain duplicates in %files.
[x]: MUST Spec file lacks Packager, Vendor, PreReq tags.
[x]: MUST Package does not run rm -rf %{buildroot} (or $RPM_BUILD_ROOT) at the
     beginning of %install.
     Note: rm -rf would be needed if support for EPEL5 is required
[-]: MUST Large documentation files are in a -doc subpackage, if required.
[x]: MUST If (and only if) the source package includes the text of the
     license(s) in its own file, then that file, containing the text of the
     license(s) for the package is included in %doc.
[x]: MUST License field in the package spec file matches the actual license.
     Note: Licenses found: "BSD (3 clause) GENERATED FILE", "BSD (3 clause) "
     For detailed output of licensecheck see file:
     /home/michel/sources/fedora/projects/FedoraReview/src/829528/licensecheck.txt
[x]: MUST Package consistently uses macros (instead of hard-coded directory
     names).
[x]: MUST Package is named according to the Package Naming Guidelines.
[x]: MUST Package does not generate any conflict.
[x]: MUST Package obeys FHS, except libexecdir and /usr/target.
[x]: MUST Package must own all directories that it creates.
[x]: MUST Package does not own files or directories owned by other packages.
[x]: MUST Package installs properly.
[x]: MUST Requires correct, justified where necessary.
[x]: MUST Rpmlint output is silent.
[!]: MUST Sources used to build the package match the upstream source, as
     provided in the spec URL.
/home/michel/sources/fedora/projects/FedoraReview/src/829528/kyua-cli-0.4.tar.gz :
  MD5SUM this package     : 09a1b938649e2d5e2901e1131d05313d
  MD5SUM upstream package : 3af7490a7b99d5b65b16be8a3abe9c02

[x]: MUST Spec file is legible and written in American English.
[x]: MUST Spec file name must match the spec package %{name}, in the format
     %{name}.spec.
[-]: MUST Package contains a SysV-style init script if in need of one.
[x]: MUST File names are valid UTF-8.
[x]: MUST Useful -debuginfo package or justification otherwise.
[x]: SHOULD Reviewer should test that the package builds in mock.
[-]: SHOULD If the source package does not include license text(s) as a
     separate file from upstream, the packager SHOULD query upstream to
     include it.
[x]: SHOULD Dist tag is present.
[x]: SHOULD No file requires outside of /etc, /bin, /sbin, /usr/bin,
     /usr/sbin.
[x]: SHOULD Final provides and requires are sane (rpm -q --provides and rpm -q
     --requires).
[?]: SHOULD Package functions as described.
[x]: SHOULD Latest version is packaged.
[x]: SHOULD Package does not include license text files separate from
     upstream.
[x]: SHOULD Scriptlets must be sane, if used.
[x]: SHOULD SourceX is a working URL.
[-]: SHOULD Description and summary sections in the package spec file contains
     translations for supported Non-English languages, if available.
[?]: SHOULD Package should compile and build into binary rpms on all supported
     architectures.
[x]: SHOULD %check is present and all tests pass.
[!]: SHOULD Packages should try to preserve timestamps of original installed
     files.
[x]: SHOULD Spec use %global instead of %define.

Issues:
[!]: MUST Sources used to build the package match the upstream source, as
     provided in the spec URL.
/home/michel/sources/fedora/projects/FedoraReview/src/829528/kyua-cli-0.4.tar.gz :
  MD5SUM this package     : 09a1b938649e2d5e2901e1131d05313d
  MD5SUM upstream package : 3af7490a7b99d5b65b16be8a3abe9c02

See: http://fedoraproject.org/wiki/Packaging/SourceURL

[!]: SHOULD Packages should try to preserve timestamps of original installed
     files.
     Please add the -p flag on Linux to preserve modification times
     (not sure if this works on BSD as well, but in any case, you can
     override it either when configuring or as a make option

Generated by fedora-review 0.2.0git
External plugins:

Comment 3 Julio Merino 2012-06-08 02:14:33 UTC
* Distfile checksum mismatch: Agh, my bad.  I had created the SRPM before uploading the final 0.4 release file to ensure things would work... and then forgot to regenerate the SRPM with the final tarball.  Fixed.

* Not using ATF: If I enable ATF support, this will build and _install_ a bunch of test programs.  The reason for not installing these is the same as in the ATF and Lutok packages: the tests are expected to go into ${prefix}/tests at the moment, but that is most likely unacceptable as mentioned earlier.  I'd like to discuss this topic separately, make a decision, and then enable the tests in all of atf, lutok and kyua-cli at once in the same manner.

* install -p: I'm not sure I understand your comment about this.  The spec file is not using install directly; it only uses install-info(1).  All the install(1) calls come from automake, and these should be the same for any other package; correct?  (I didn't do anything special for atf nor lutok, for example.)

Thanks for the quick turnaround!

Comment 4 Michel Lind 2012-06-08 02:45:50 UTC
(In reply to comment #3)
> * Distfile checksum mismatch: Agh, my bad.  I had created the SRPM before
> uploading the final 0.4 release file to ensure things would work... and then
> forgot to regenerate the SRPM with the final tarball.  Fixed.
>
Ah, good
 
> * Not using ATF: If I enable ATF support, this will build and _install_ a
> bunch of test programs.  The reason for not installing these is the same as
> in the ATF and Lutok packages: the tests are expected to go into
> ${prefix}/tests at the moment, but that is most likely unacceptable as
> mentioned earlier.  I'd like to discuss this topic separately, make a
> decision, and then enable the tests in all of atf, lutok and kyua-cli at
> once in the same manner.
> 
Ah, of course. That would not have been a review blocker anyway, I just didn't remember the reasoning -- thanks for the refresher!

> * install -p: I'm not sure I understand your comment about this.  The spec
> file is not using install directly; it only uses install-info(1).  All the
> install(1) calls come from automake, and these should be the same for any
> other package; correct?  (I didn't do anything special for atf nor lutok,
> for example.)
> 
This used to be more of an issue in the past, when installing 32-bit and 64-bit versions of the same package would cause file overlaps in %{_datadir} (/usr/share) -- and unless the timestamps were identical RPM would refuse to do the install. I think now the checksums are compared so this is not that serious a problem anymore (plus since this package does not provide libraries, it's not that big a deal anyway).

I didn't catch this problem when reviewing atf and lutok -- was not using the same review template I'm using now and it simply wasn't something I checked (my bad.. though again, it's a minor suggestion). It seems that what you want to do is override, in configure.ac, AC_PROG_INSTALL -- and then the right invocation would end up written into the Makefiles.

See http://seul.org/docs/autotut/

But you can make that change in the next release, it's not urgent. It's just nice if the user can see when each of the files on their computer were actually last modified, instead of when the package was built (except for files that were created during the build process, where those two times would be almost identical)

APPROVED

Comment 5 Julio Merino 2012-06-08 02:54:07 UTC
New Package SCM Request
=======================
Package Name: kyua-cli
Short Description: Automated testing framework (Command line interface)
Owners: jmmv
Branches: f17
InitialCC:

Comment 6 Julio Merino 2012-06-08 02:56:18 UTC
Replying separately from the SCM request, as I don't know if that template and my comment would work fine together.

Michel: I'll give a try to the install -p thingy and then apply it to the other packages.  It's just a bit surprising to me (when compared to the pkgsrc world) that this has to be done manually, particularly when there are tons of autotools-based packages and they would also benefit from the same tweaks!

Thanks again :-)

Comment 7 Michel Lind 2012-06-08 09:09:52 UTC
(In reply to comment #6)
> Replying separately from the SCM request, as I don't know if that template
> and my comment would work fine together.

I normally put my SCM request at the bottom of the comment and that works. I don't remember if it works if there's a subsequent comment -- I'm assuming it would, but let's wait a few hours and see; if nothing happens, then I guess the request has to be at the bottom (I doubt it's that brittle though)

> Michel: I'll give a try to the install -p thingy and then apply it to the
> other packages.  It's just a bit surprising to me (when compared to the
> pkgsrc world) that this has to be done manually, particularly when there are
> tons of autotools-based packages and they would also benefit from the same
> tweaks!
> 
It surprised me too, to be honest. I suspect that e.g. for GNOME packages, gnome-common provide a saner default -- but as to why autotools doesn't just default to install -p, who knows?

> Thanks again :-)
No problem!

Comment 8 Gwyn Ciesla 2012-06-08 12:46:48 UTC
Git done (by process-git-requests).

Yes, it does. :)

Comment 9 Fedora Update System 2012-06-08 18:38:27 UTC
kyua-cli-0.4-1.fc17 has been submitted as an update for Fedora 17.
https://admin.fedoraproject.org/updates/kyua-cli-0.4-1.fc17

Comment 10 Fedora Update System 2012-06-10 01:26:11 UTC
kyua-cli-0.4-1.fc17 has been pushed to the Fedora 17 testing repository.

Comment 11 Fedora Update System 2012-06-17 22:26:00 UTC
kyua-cli-0.4-1.fc17 has been pushed to the Fedora 17 stable repository.