Bug 1211517 - Review Request: docker-swarm - Docker-native clustering system
Summary: Review Request: docker-swarm - Docker-native clustering system
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Fedora
Classification: Fedora
Component: Package Review
Version: rawhide
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Jan Chaloupka
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On: 1208616 1212626 1212653
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-04-14 08:33 UTC by Lokesh Mandvekar
Modified: 2017-05-06 06:26 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-05-06 06:26:29 UTC
Type: ---
Embargoed:
jchaloup: fedora-review+
gwync: fedora-cvs+


Attachments (Terms of Use)
spec file modification (1.50 KB, patch)
2015-04-14 10:39 UTC, Jan Chaloupka
no flags Details | Diff

Description Lokesh Mandvekar 2015-04-14 08:33:48 UTC
Spec URL: https://lsm5.fedorapeople.org/swarm/swarm.spec
SRPM URL: https://lsm5.fedorapeople.org/swarm/SRPMS/swarm-0-0.1.git369c856.fc23.src.rpm

Description:
Docker Swarm is native clustering for Docker. It turns a pool of Docker hosts
into a single, virtual host.

Swarm serves the standard Docker API, so any tool which already communicates
with a Docker daemon can use Swarm to transparently scale to multiple hosts:
Dokku, Compose, Krane, Flynn, Deis, DockerUI, Shipyard, Drone, Jenkins...
and, of course, the Docker client itself.

Fedora Account System Username: lsm5

koji: http://koji.fedoraproject.org/koji/taskinfo?taskID=9475133

Comment 1 Lokesh Mandvekar 2015-04-14 08:51:26 UTC
This is probably better off being called 'docker-swarm' given docker-compose and docker-machine. (Not certain though)

Comment 2 Jan Chaloupka 2015-04-14 10:36:54 UTC
OK
- license OK
- provides OK
- binary is build from bundled source codes (lets keep all its deps up2date at least and package new packages)

not OK
- missing debug info

Missing packages:
$ gofed ggi -c -s -d
Class: github.com/docker/swarm (golang-github-docker-swarm) PkgDB=False
Class: github.com/hashicorp/consul (golang-github-hashicorp-consul) PkgDB=False
Class: github.com/samalba/dockerclient (golang-github-samalba-dockerclient) PkgDB=False
Class: github.com/samuel/go-zookeeper (golang-github-samuel-go-zookeeper) PkgDB=False

As some golang packages still break API building tools from dependencies is possible but can lead to missing or modified symbols during building. Building from bundled source codes handle this situation. However we should keep all its dependencies up2date and debundle them anyway. As one day dependencies's API get stable we can then easily switch to packaged dependencies.

Comment 3 Jan Chaloupka 2015-04-14 10:39:05 UTC
Created attachment 1014234 [details]
spec file modification

Comment 4 Jan Chaloupka 2015-04-14 10:40:50 UTC
CONTRIBUTING.md, ROADMAP.md and LICENSE are missing in %doc

Comment 5 Jan Chaloupka 2015-04-14 11:39:03 UTC
As swarm is "Swarm is the app for keeping up and meeting up with friends" found by google, definitely docker-swarm :).

Comment 6 Lokesh Mandvekar 2015-04-14 18:46:25 UTC
will make other changes soon-ish. Thanks for the comments

Comment 7 Michael S. 2015-05-08 03:05:41 UTC
LICENSE should go in %license, not %doc, to be precise :)

Comment 8 Jan Chaloupka 2015-05-09 09:28:18 UTC
Michael thanks for reviews. Looking at [1] it states:

These prefixes are not valid in Fedora: %license and %readme.

Looks like Fedora does not use any %license tag.

[1] https://fedoraproject.org/wiki/How_to_create_an_RPM_package

Comment 9 Michael S. 2015-05-09 16:18:28 UTC
That's what the guidelines say :
https://fedoraproject.org/wiki/Packaging:LicensingGuidelines#License_Text

But that's a quite recent addition :
http://www.rpm.org/ticket/116
https://fedorahosted.org/fpc/ticket/411

It matter for the cloud product, as it is used without %doc, but we still need the license somewhere.

Comment 10 Jan Chaloupka 2015-05-12 13:58:01 UTC
Yeah, valid from rpm >= 4.11. F20-rawhide are fine, epel6 is not.

Comment 11 Michael S. 2015-05-13 04:49:45 UTC
The fpc ticket show a workaround. More ugly than what I believed however, but this can be defered for the EPEL6 package maybe ?

Comment 12 Jan Chaloupka 2015-05-13 08:03:49 UTC
%if 0%{?fedora} || 0%{?rhel} >= 7
%license LICENSE
%doc README.md
%else
%doc README.md LICENSE
%endif

will do the things. It is just it has to be changed in every packaged golang project :). There is about 120 packages => ~480 spec files + those on review (36). So about 500 spec files to modify :)

Comment 13 M. Scherer 2015-05-14 04:44:06 UTC
It can be scripted, so that shouldn't be too annoying. 

Something like :
perl -i -p -e 's/^(\%doc .*)(LICENSE|COPYING)(.*)$/\%if 0\%{?fedora} || 0\%{?rhel} >= 7\n\%license \2\n\1 \3\n%else\n\1\2\3\n\%endif\n/g' foo.spec 

+ the commit/checkout machinery around ( I do not think this warrant a rebuild )

Comment 14 Robert P. J. Day 2015-08-23 09:34:48 UTC
Is there any ETA for docker-swarm in updates-testing?

Comment 15 Jan Chaloupka 2015-09-01 13:14:33 UTC
This macro if defined in the top of the spec file can be used instead of %license.

%define copying() \
%if 0%{?fedora} >= 21 || 0%{?rhel} >= 7 \
%license %{*} \
%else \
%doc %{*} \
%endif

I am ready to continue with the review. Lokesh, can you upload updated spec file?

Comment 16 Lokesh Mandvekar 2015-09-30 15:10:49 UTC
Spec URL: https://lsm5.fedorapeople.org/docker-swarm/docker-swarm.spec
SRPM URL: https://lsm5.fedorapeople.org/docker-swarm/SRPMS/docker-swarm-0.5.0-0.1.gitd17264f.fc24.src.rpm

- debuginfo not generated coz 'No Build ID' error.
- package doesn't build with rpm-ed deps. So built using bundled deps (there's no difference in using the former v/s the latter anyway when they are simply build-deps and not runtime deps).
- the spec wasn't generated using gofed, but I could add gofed macros later. Hope that's not a blocker :)

Comment 17 Jan Chaloupka 2015-09-30 18:23:58 UTC
Spec file is ok, no need to generate it via gofed. As it depends on docker, it has to be built from tarball. Would be great to generate entire spec file and just set with_bundled 1 and with_devel 0. This was it can be switch to Fedora deps at any time without additional work.

Approving.

Comment 18 Lokesh Mandvekar 2015-09-30 21:59:42 UTC
New Package SCM Request
=======================
Package Name: docker-swarm 
Short Description: Docker-native clustering system
Upstream URL: https://github.com/docker/swarm
Owners: lsm5 jchaloup fpokorny
Branches: f23 f22 f21
InitialCC: golang-sig

Comment 19 M. Scherer 2015-09-30 23:01:17 UTC
If you want to bundle source, shouldn't you ask first to the FPC ?

This is golang, everything is build statically, so yes, using the bundled copy vs the non bundled one might result in a difference.

And I just looked at the policy, it didn't seems to have changed, afaik:
https://fedoraproject.org/wiki/Packaging:No_Bundled_Libraries

Comment 20 Jan Chaloupka 2015-10-01 07:16:29 UTC
> If you want to bundle source, shouldn't you ask first to the FPC ?

https://fedorahosted.org/fpc/ticket/382

The fpc ticket for golang packaging is still unresolved due to API breakages. Yes, I should ask fpc first.

> This is golang, everything is build statically, so yes, using the bundled copy
> vs the non bundled one might result in a difference.

> And I just looked at the policy, it didn't seems to have changed, afaik:
> https://fedoraproject.org/wiki/Packaging:No_Bundled_Libraries

Comment 21 Gwyn Ciesla 2015-10-01 12:51:19 UTC
Git done (by process-git-requests).

Comment 22 Jan Chaloupka 2015-10-01 13:05:05 UTC
> If you want to bundle source, shouldn't you ask first to the FPC ?

This is a long standing problem. I am trying to minimize it with better tools and analysis of dependencies. However, at some point, there is no other way than use bundled deps. The aim here is to build docker-swarm from debundled deps as much as possible and fill the non-working deps with those in the tarball.

As I mentioned earlier, this is not the only golang project that is built from bundled deps. Until API breakage problem is resolved (at least to some extent), I don't want to bother fpc with new exception for each golang project we have in Fedora whenever there is incompatible update. The incompatibility make disappear meantime or show repeatedly.

Instead of spending time on figuring out how to justify building from tarball, let's concentrate on elimination and finding alternatives how to get better picture of the current problems and how we should deal with the problems. There is already ongoing discussion on the fpc ticket. So be welcome to continue this discussion there.

Comment 23 M. Scherer 2015-10-01 13:08:08 UTC
If this is unresolved, then that shouldn't be done. 

We have processes decided by community, and just acting as if they do not matter when we are using our @redhat.com email reflect very badly on the community, cuase it send the message "we have rules for others, not for us". 

I think this is to the FPC to decide if they are bothered or not, not to us.

Comment 24 Jan Chaloupka 2015-10-01 13:32:51 UTC
> We have processes decided by community, and just acting as if they do not
> matter when we are using our @redhat.com email reflect very badly on the
> community, cuase it send the message "we have rules for others, not for us". 

They do matter for me. And there is no excuse for breaking those rules. As you can see in many reviews of golang project, me and my colleagues are doing what we can to make golang packaging better. We have a lack of resource and there is still a lot of tasks to do to make automatic discovering of issues and updates of golang spec files. The effort to deal with dependencies is on. I could dick more into the deps and find what function or variable is broken, fix it and bult docker-swarm just from Fedora's deps. However, doing this for each update of docker is very frustrating. Once the tooling is done, it will be faster and easier to deal with it.

Would be better to have an exception for every golang project until API breakage is resolved globally. It means functioning communication with upstream maintainers, tooling to automatically discover issues, automatic updates of spec files, etc.

Comment 25 Haïkel Guémar 2015-10-01 14:15:10 UTC
As an individual packager, I understand the situation and would like that we improve bundled libraries policies.

Before anything, at the very least, declare bundled libraries as requested per guidelines:
https://fedoraproject.org/wiki/Packaging:No_Bundled_Libraries#Requirement_if_you_bundle 

As a fesco member, I kindly ask you to stop bypassing FPC, and if you feel that the discussion is blocked, please notify fesco about it.
FPC has been appointed by Fesco to deal with these topics, so we are entitled to enforce their decisions unless *we* decide to override them (and this is not a popular option among Fesco members).

---

And I agree with Michael that this is not helping to improve Red Hat image within Fedora community. We have to respect the rules decided by the Fedora community, even if we don't like them.

Comment 26 Yajo 2016-02-19 08:16:31 UTC
Any news on this?

Comment 27 Jan Chaloupka 2016-02-23 12:04:52 UTC
Lokesh' links to spec seems to be lost. Lokesh, do you have a backup?

Comment 28 Lokesh Mandvekar 2016-02-25 16:16:03 UTC
Awesome, I seem to have conveniently lost the spec file and the backup, let's do it right this time and make everyone happy :) .

Jan, could you post an auto-generated spec file on to gofed reviews and I'll keep making PRs to it.

Comment 30 Yajo 2016-10-14 08:13:20 UTC
It seems to me that this is not necessary anymore after Docker 1.12 with its built-in swarm mode.


Note You need to log in before you can comment on or make changes to this bug.