Bug 1234649 - Review Request: testcloud - a small tool for running cloud images locally
Summary: Review Request: testcloud - a small tool for running cloud images locally
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: Package Review
Version: rawhide
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Adam Miller
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-06-22 23:06 UTC by Mike Ruckman
Modified: 2016-01-18 10:58 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-01-14 08:55:26 UTC
Type: ---
Embargoed:
admiller: fedora-review+


Attachments (Terms of Use)

Description Mike Ruckman 2015-06-22 23:06:26 UTC
Spec URL: https://roshi.fedorapeople.org/packages/testcloud.spec
SRPM URL: https://roshi.fedorapeople.org/packages/testcloud-0.1.0-1.fc21.src.rpm
Description: testcloud is a relatively simple system which is capable of booting images designed for cloud systems on a local system with minimal configuration. testcloud is designed to be (and remain) somewhat simple, trading fancy cloud system features for ease of use and sanity in development.
Fedora Account System Username: roshi

Comment 1 Mike Ruckman 2015-06-22 23:22:48 UTC
This is the first package I've submitted, so I'll need a sponsor. I'm also the upstream maintainer of this package. I don't have much experience with packaging, but I'm active on the QA side of things and would like to get more involved.

Link to a successful copr build: https://copr.fedoraproject.org/coprs/roshi/testCloud/builds/

Comment 2 gil cattaneo 2015-06-22 23:46:27 UTC
wellcome
there are two issues in the spec file 
1. in install section
rm -rf $RPM_BUILD_ROOT should be removed
2. 
%doc README.rst LICENSE
you should use:
%doc README.rst
%license LICENSE

thanks for your work

Comment 3 Mike Ruckman 2015-06-23 00:09:27 UTC
Thanks! Updated :)

Comment 4 Michael Schwendt 2015-06-23 09:34:36 UTC
> # sitelib for noarch packages, sitearch for others (remove the unneeded one)

> BuildArch:      noarch

Notice the hint in brackets.


> Requires:       libvirt
> Requires:       libguestfs
> Requires:       libguestfs-tools
> Requires:       python-requests

Explicit Requires benefit from a comment that explains the dependency. Without such comments, packagers sometimes remove dependencies accidentally.


> %dir %{_sysconfdir}/testcloud
> %dir %attr(777, root, root) %{_sharedstatedir}/testcloud/cache
> %dir %attr(777, root, root) %{_sharedstatedir}/testcloud/instances
> 
> %{_sysconfdir}/testcloud/settings.p*

Directory %{_sharedstatedir}/testcloud/ is not included yet:
https://fedoraproject.org/wiki/Packaging:Guidelines#File_and_Directory_Ownership

What is the reason to make the directories *world-writable*? The README doesn't give a rationale either, but just says "any permitted user" which is anyone for a world-writable dir.

https://fedoraproject.org/wiki/Packaging:Guidelines#File_Permissions


> %{_sysconfdir}/testcloud/settings.p*

Its default contents refer to /var/lib/testCloud/ with a different and misleading uppercase 'C' in the spelling.


> /usr/bin/testcloud
> %{python_sitelib}/testcloud/init.py

Both define version 0.0.1 while the package claims it is 0.1.0.

Comment 5 Mike Ruckman 2015-06-24 05:53:37 UTC
(In reply to Michael Schwendt (Fedora Packager Sponsors Group) from comment #4)
> > # sitelib for noarch packages, sitearch for others (remove the unneeded one)
> 
> > BuildArch:      noarch
> 
> Notice the hint in brackets.
> 

Fixed.

> 
> > Requires:       libvirt
> > Requires:       libguestfs
> > Requires:       libguestfs-tools
> > Requires:       python-requests
> 
> Explicit Requires benefit from a comment that explains the dependency.
> Without such comments, packagers sometimes remove dependencies accidentally.
> 

Added comments for each of the requires.

> 
> > %dir %{_sysconfdir}/testcloud
> > %dir %attr(777, root, root) %{_sharedstatedir}/testcloud/cache
> > %dir %attr(777, root, root) %{_sharedstatedir}/testcloud/instances
> > 
> > %{_sysconfdir}/testcloud/settings.p*
> 
> Directory %{_sharedstatedir}/testcloud/ is not included yet:
> https://fedoraproject.org/wiki/Packaging:
> Guidelines#File_and_Directory_Ownership
> 

Thanks for catching this. I didn't know I needed to be that explicit, I thought claiming the other dirs was enough.

> What is the reason to make the directories *world-writable*? The README
> doesn't give a rationale either, but just says "any permitted user" which is
> anyone for a world-writable dir.
> 
> https://fedoraproject.org/wiki/Packaging:Guidelines#File_Permissions
> 

This one I'm not sure how to figure out. Getting permissions ironed out for this has been a pain so far - so I welcome any ideas :) As for the world-writeable permissions I was using, it allowed anyone to run the tool and download images to those directories. I've tested a couple other permissions sets and haven't found something that works.

What's the best way to solve this? The user fires off a command that downloads the image (as that user), then virsh and friends take over, which need to be able to manipulate that directory. Would giving permissions to the qemu group be a better fit?

> 
> > %{_sysconfdir}/testcloud/settings.p*
> 
> Its default contents refer to /var/lib/testCloud/ with a different and
> misleading uppercase 'C' in the spelling.
> 

Thanks, thought I'd caught all the capitals (the old name was testCloud - but decided to change it before release).

> 
> > /usr/bin/testcloud
> > %{python_sitelib}/testcloud/init.py
> 
> Both define version 0.0.1 while the package claims it is 0.1.0.

Again, thought I'd caught everything that needed updating in the code. It's fixed now, thanks!

Once I get the permissions ironed out I'll update the srpm and spec file online for you to take a look at.

Thanks again!

Comment 6 Michael Schwendt 2015-06-24 10:15:02 UTC
> Getting permissions ironed out for this has been a pain so far - 
> so I welcome any ideas :) 

> What's the best way to solve this? 

More elegant, more ordinary and more expected would be to set up a local tree in user's $HOME and to work within that space. What else have you tried that hasn't worked? It seems simplification has gone too far, if users operate in a world-writable directory below /var/lib.

It could be an idea to open a thread on devel@ list, mention any specific requirements there are, and ask for comments.

Comment 7 Kamil Páral 2015-06-25 08:06:25 UTC
Out of curiosity, why is /var/lib/testcloud/cache used for cache dir instead of /var/cache/testcloud ? If I'm running out of space and I want to delete some caches, the latter location is expected and probably the first place where the user will look for such files.

Comment 8 Mike Ruckman 2015-06-25 15:28:52 UTC
(In reply to Michael Schwendt (Fedora Packager Sponsors Group) from comment #6)
> > Getting permissions ironed out for this has been a pain so far - 
> > so I welcome any ideas :) 
> 
> > What's the best way to solve this? 
> 
> More elegant, more ordinary and more expected would be to set up a local
> tree in user's $HOME and to work within that space. What else have you tried
> that hasn't worked? It seems simplification has gone too far, if users
> operate in a world-writable directory below /var/lib.

We've tried to put this directory tree in $HOME, but libvirt doesn't play well with data in a home directory, since it has to own all the images it works with. Libvirt already keeps it's images in /var/lib - so it seemed a logical place to put them in this case. In the case of the permissions, the user running the script needs to have write access to that directory in order to download images for libvirt to use.

We're currently revisiting this to see if there's a viable approach for our use case.

(In reply to Kamil Páral from comment #7)
> Out of curiosity, why is /var/lib/testcloud/cache used for cache dir instead
> of /var/cache/testcloud ? If I'm running out of space and I want to delete
> some caches, the latter location is expected and probably the first place
> where the user will look for such files.

I suppose the name "cache" is a bit misleading in a system context. Originally, testcloud downloaded the image for your instance each time you launched an instance. This made everything take a lot longer to run, wasted bandwidth and disk space. So I started to cache the images, and use them as backing stores for instances; saving disk space and bandwidth.

In your case, if you were to delete images from what is now /var/lib/testcloud/cache it would break any instances using those as a backing store. I've opened a ticket in phab [0] to make the name change on that directory to better communicate what it contains.

[0] https://phab.qadevel.cloud.fedoraproject.org/T521

Comment 9 Mike Ruckman 2015-07-14 18:41:21 UTC
I've updated the package to get rid of the world writeable directory [0]. The spec file can be seen here [1]. In order to get permissions to work, I had to create a testcloud group (the user has to add themselves to this group), and add a polkit rule so that users can manage instances over ssh.

[0] https://copr.fedoraproject.org/coprs/roshi/testCloud/build/104053/
[1] https://roshi.fedorapeople.org/packages/testcloud.spec

Comment 10 Tim Flink 2015-11-19 20:19:41 UTC
testcloud has been updated to a new version that doesn't subprocess virt-install calls to do installation.

I'm not Mike but I am a co-maintainer of testcloud

https://copr.fedoraproject.org/coprs/roshi/testCloud/build/140592/
https://tflink.fedorapeople.org/packages/testcloud/testcloud.spec

Comment 11 Adam Miller 2015-11-24 21:53:55 UTC
Package Review
==============

Legend:
[x] = Pass, [!] = Fail, [-] = Not applicable, [?] = Not evaluated
[ ] = Manual review needed


Issues:
=======
- Package installs properly.
  Note: Installation errors (see attachment)
  See: https://fedoraproject.org/wiki/Packaging:Guidelines
- Permissions on files are set properly.
  Note: See rpmlint output
  See: http://fedoraproject.org/wiki/Packaging/Guidelines#FilePermissions


===== MUST items =====

Generic:
[x]: Package is licensed with an open-source compatible license and meets
     other legal requirements as defined in the legal section of Packaging
     Guidelines.
[x]: License field in the package spec file matches the actual license.
     Note: Checking patched sources after %prep for licenses. Licenses
     found: "Unknown or generated". 28 files have unknown license. Detailed
     output of licensecheck in
     /home/admiller/reviews/testcloud/licensecheck.txt
[x]: Package contains no bundled libraries without FPC exception.
[x]: Changelog in prescribed format.
[x]: Sources contain only permissible code or content.
[!]: Package contains desktop file if it is a GUI application.
[x]: Development files must be in a -devel package
[x]: Package uses nothing in %doc for runtime.
[x]: Package consistently uses macros (instead of hard-coded directory
     names).
[x]: Package is named according to the Package Naming Guidelines.
[x]: Package does not generate any conflict.
[x]: Package obeys FHS, except libexecdir and /usr/target.
[x]: If the package is a rename of another package, proper Obsoletes and
     Provides are present.
[x]: Requires correct, justified where necessary.
[x]: Spec file is legible and written in American English.
[-]: Package contains systemd file(s) if in need.
[x]: Package is not known to require an ExcludeArch tag.
[-]: Large documentation must go in a -doc subpackage. Large could be size
     (~1MB) or number of files.
     Note: Documentation size is 10240 bytes in 1 files.
[x]: Package complies to the Packaging Guidelines
[x]: Package successfully compiles and builds into binary rpms on at least
     one supported primary architecture.
[x]: Rpmlint is run on all rpms the build produces.
     Note: There are rpmlint messages (see attachment).
[x]: If (and only if) the source package includes the text of the
     license(s) in its own file, then that file, containing the text of the
     license(s) for the package is included in %license.
[x]: Package requires other packages for directories it uses.
[x]: Package must own all directories that it creates.
[x]: Package does not own files or directories owned by other packages.
[x]: All build dependencies are listed in BuildRequires, except for any
     that are listed in the exceptions section of Packaging Guidelines.
[x]: Package uses either %{buildroot} or $RPM_BUILD_ROOT
[x]: Package does not run rm -rf %{buildroot} (or $RPM_BUILD_ROOT) at the
     beginning of %install.
[x]: %config files are marked noreplace or the reason is justified.
[x]: Macros in Summary, %description expandable at SRPM build time.
[x]: Dist tag is present.
[x]: Package does not contain duplicates in %files.
[x]: Package use %makeinstall only when make install DESTDIR=... doesn't
     work.
[x]: Package is named using only allowed ASCII characters.
[x]: No %config files under /usr.
[x]: Package does not use a name that already exists.
[x]: Package is not relocatable.
[x]: Sources used to build the package match the upstream source, as
     provided in the spec URL.
[x]: Spec file name must match the spec package %{name}, in the format
     %{name}.spec.
[x]: File names are valid UTF-8.
[x]: Packages must not store files under /srv, /opt or /usr/local

Python:
[x]: Python eggs must not download any dependencies during the build
     process.
[x]: A package which is used by another package via an egg interface should
     provide egg info.
[!]: Package meets the Packaging Guidelines::Python
[x]: Package contains BR: python2-devel or python3-devel
[x]: Binary eggs must be removed in %prep

===== SHOULD items =====

Generic:
[x]: If the source package does not include license text(s) as a separate
     file from upstream, the packager SHOULD query upstream to include it.
[x]: Final provides and requires are sane (see attachments).
[x]: Package functions as described.
[x]: Latest version is packaged.
[x]: Package does not include license text files separate from upstream.
[x]: Description and summary sections in the package spec file contains
     translations for supported Non-English languages, if available.
[x]: Package should compile and build into binary rpms on all supported
     architectures.
[x]: %check is present and all tests pass.
[x]: Packages should try to preserve timestamps of original installed
     files.
[x]: Packager, Vendor, PreReq, Copyright tags should not be in spec file
[x]: Sources can be downloaded from URI in Source: tag
[x]: Reviewer should test that the package builds in mock.
[x]: Buildroot is not present
[x]: Package has no %clean section with rm -rf %{buildroot} (or
     $RPM_BUILD_ROOT)
[x]: No file requires outside of /etc, /bin, /sbin, /usr/bin, /usr/sbin.
[x]: SourceX is a working URL.
[x]: Spec use %global instead of %define unless justified.

===== EXTRA items =====

Generic:
[!]: Rpmlint is run on all installed packages.
     Note: Mock build failed
     See: http://fedoraproject.org/wiki/Packaging/Guidelines#rpmlint


Installation errors
-------------------
INFO: mock.py version 1.2.13 starting (python version = 3.4.2)...
Start: init plugins
INFO: selinux enabled
Finish: init plugins
Start: run
Start: chroot init
INFO: calling preinit hooks
INFO: enabled root cache
INFO: enabled dnf cache
Start: cleaning dnf metadata
Finish: cleaning dnf metadata
INFO: enabled ccache
Mock Version: 1.2.13
INFO: Mock Version: 1.2.13
Finish: chroot init
INFO: installing package(s): /home/admiller/reviews/testcloud/results/testcloud-0.1.5-4.fc24.noarch.rpm
ERROR: Command failed. See logs for output.
 # /usr/bin/dnf --installroot /var/lib/mock/fedora-rawhide-x86_64/root/ --releasever 24 install /home/admiller/reviews/testcloud/results/testcloud-0.1.5-4.fc24.noarch.rpm --setopt=tsflags=nocontexts


Rpmlint
-------
Checking: testcloud-0.1.5-4.fc24.noarch.rpm
          testcloud-0.1.5-4.fc24.src.rpm
testcloud.noarch: E: explicit-lib-dependency libguestfs
testcloud.noarch: E: explicit-lib-dependency libguestfs-tools
testcloud.noarch: E: description-line-too-long C testcloud is a relatively simple system which is capable of booting images designed for cloud systems on a local system with minimal configuration. testcloud is designed to be (and remain) somewhat simple, trading fancy cloud system features for ease of use and sanity in development.
testcloud.noarch: W: non-standard-gid /var/lib/testcloud testcloud
testcloud.noarch: E: non-standard-dir-perm /var/lib/testcloud 775
testcloud.noarch: W: non-standard-gid /var/lib/testcloud/instances testcloud
testcloud.noarch: E: non-standard-dir-perm /var/lib/testcloud/instances 775
testcloud.noarch: E: non-executable-script /usr/lib/python2.7/site-packages/testcloud/exceptions.py 644 /usr/bin/env
testcloud.noarch: W: non-conffile-in-etc /etc/polkit-1/rules.d/99-testcloud-nonroot-libvirt-access.rules
testcloud.noarch: E: executable-marked-as-config-file /etc/testcloud/settings.py
testcloud.noarch: E: script-without-shebang /etc/testcloud/settings.py
testcloud.noarch: E: non-executable-script /usr/lib/python2.7/site-packages/testcloud/image.py 644 /usr/bin/env
testcloud.noarch: W: non-standard-gid /var/lib/testcloud/domain-template.jinja testcloud
testcloud.noarch: E: non-standard-executable-perm /var/lib/testcloud/domain-template.jinja 775
testcloud.noarch: E: script-without-shebang /var/lib/testcloud/domain-template.jinja
testcloud.noarch: E: non-executable-script /usr/lib/python2.7/site-packages/testcloud/util.py 644 /usr/bin/env
testcloud.noarch: W: non-standard-gid /var/lib/testcloud/cache testcloud
testcloud.noarch: E: non-standard-dir-perm /var/lib/testcloud/cache 775
testcloud.noarch: E: non-executable-script /usr/lib/python2.7/site-packages/testcloud/cli.py 644 /usr/bin/env
testcloud.noarch: E: non-executable-script /usr/lib/python2.7/site-packages/testcloud/instance.py 644 /usr/bin/env
testcloud.noarch: W: no-manual-page-for-binary testcloud
testcloud.src: E: description-line-too-long C testcloud is a relatively simple system which is capable of booting images designed for cloud systems on a local system with minimal configuration. testcloud is designed to be (and remain) somewhat simple, trading fancy cloud system features for ease of use and sanity in development.
2 packages and 0 specfiles checked; 16 errors, 6 warnings.




Requires
--------
testcloud (rpmlib, GLIBC filtered):
    /bin/sh
    /usr/bin/python
    config(testcloud)
    libguestfs
    libguestfs-tools
    libvirt
    libvirt-python
    net-tools
    polkit
    python(abi)
    python-jinja2
    python-requests
    shadow-utils



Provides
--------
testcloud:
    config(testcloud)
    testcloud



Source checksums
----------------
https://github.com/Rorosha/testcloud/archive/0.1.5.tar.gz :
  CHECKSUM(SHA256) this package     : 675bad6a9fea5defbf4c045dc12fd4ad3c9a92b3543542aa2aa1b51c705dfa05
  CHECKSUM(SHA256) upstream package : 675bad6a9fea5defbf4c045dc12fd4ad3c9a92b3543542aa2aa1b51c705dfa05


Generated by fedora-review 0.6.0 (3c5c9d7) last change: 2015-05-20
Command line :/usr/bin/fedora-review --rpm-spec -n /tmp/testcloud-0.1.5-4.fc24.src.rpm -m fedora-rawhide-x86_64
Buildroot used: fedora-rawhide-x86_64
Active plugins: Python, Generic, Shell-api
Disabled plugins: Java, C/C++, fonts, SugarActivity, Ocaml, Perl, Haskell, R, PHP, Ruby
Disabled flags: EXARCH, DISTTAG, EPEL5, BATCH, EPEL6

Notes
-----

- Update %{__python} to %{__python2} so to not hit a buildroot with python3 as default and to comply with Python Packaging Guidelines
- Fix install errors
- Fix failed items in the template above

Comment 12 Adam Miller 2015-11-24 21:54:58 UTC
If you still need a Fedora Packager Sponsor, I would be happy to sponsor you.

Comment 13 Tim Flink 2015-12-01 19:16:17 UTC
For the template errors, testcloud isn't a GUI application and thus, doesn't need a desktop file unless I'm misunderstanding something.

Can you give a few more details about the installation error? I can't reproduce the issue locally and I'm not really sure what's wrong.

Comment 14 Adam Miller 2015-12-03 13:58:01 UTC
Apologies on the desktop file mark, that was an oversight on my part.

The install failure was from attempting to install in a rawhide mock chroot, which I can no longer reproduce either (the machine I did the original build/review on got re-installed over the past weekend from Fedora 22 to Fedora 23).

Comment 15 Tim Flink 2015-12-03 14:46:06 UTC
I've updated the specfile and code per review comments.

https://tflink.fedorapeople.org/packages/testcloud/testcloud.spec
https://copr.fedoraproject.org/coprs/roshi/testCloud/build/144587/

Comment 16 Adam Miller 2015-12-08 00:00:17 UTC
APPROVED.

Comment 17 Gwyn Ciesla 2015-12-08 13:22:15 UTC
Package request has been approved: https://admin.fedoraproject.org/pkgdb/package/testcloud

Comment 18 Fedora Update System 2015-12-08 20:01:13 UTC
testcloud-0.1.7-1.fc23 has been submitted as an update to Fedora 23. https://bodhi.fedoraproject.org/updates/FEDORA-2015-3b22b49d3a

Comment 19 Fedora Update System 2015-12-08 20:03:05 UTC
testcloud-0.1.7-1.fc22 has been submitted as an update to Fedora 22. https://bodhi.fedoraproject.org/updates/FEDORA-2015-f632781616

Comment 20 Fedora Update System 2015-12-10 04:56:22 UTC
testcloud-0.1.7-1.fc23 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report.
If you want to test the update, you can install it with
$ su -c 'dnf --enablerepo=updates-testing update testcloud'
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-3b22b49d3a

Comment 21 Fedora Update System 2016-01-14 08:55:24 UTC
testcloud-0.1.7-1.fc23 has been pushed to the Fedora 23 stable repository. If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.