Bug 1169966 - Review Request: rkt - CLI for running app containers
Summary: Review Request: rkt - CLI for running app containers
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: Package Review
Version: rawhide
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Jan Chaloupka
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On: 1298270 1174030 1298317 1298554
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-12-02 21:13 UTC by Lokesh Mandvekar
Modified: 2017-05-01 23:05 UTC (History)
30 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-02-27 20:42:22 UTC
Type: ---
jchaloup: fedora-review+


Attachments (Terms of Use)

Description Lokesh Mandvekar 2014-12-02 21:13:18 UTC
Spec URL: https://lsm5.fedorapeople.org/rocket/rocket.spec
SRPM URL: https://lsm5.fedorapeople.org/rocket/SRPMS/rocket-0.1.0-1.git553023e.fc22.src.rpm

Description: 
CLI for running app containers

Fedora Account System Username: lsm5

Extra files used (included in srpm): http://stable.release.core-os.net/amd64-usr/current/coreos_production_pxe_image.cpio.gz

koji: http://koji.fedoraproject.org/koji/taskinfo?taskID=8280081

 $ rpmlint rocket.spec SRPMS/rocket-0.1.0-1.git553023e.fc22.src.rpm RPMS/x86_64/rocket-0.1.0-1.git553023e.fc22.x86_64.rpm 
rocket.x86_64: W: unstripped-binary-or-object /usr/bin/actool
rocket.x86_64: E: statically-linked-binary /usr/bin/actool
rocket.x86_64: W: unstripped-binary-or-object /usr/bin/ace-validator
rocket.x86_64: E: statically-linked-binary /usr/bin/ace-validator
rocket.x86_64: W: unstripped-binary-or-object /usr/bin/metadatasvc
rocket.x86_64: E: statically-linked-binary /usr/bin/metadatasvc
rocket.x86_64: W: unstripped-binary-or-object /usr/bin/rkt
rocket.x86_64: E: statically-linked-binary /usr/bin/rkt
rocket.x86_64: W: unstripped-binary-or-object /usr/bin/init
rocket.x86_64: E: statically-linked-binary /usr/bin/init
rocket.x86_64: W: no-manual-page-for-binary actool
rocket.x86_64: W: no-manual-page-for-binary ace-validator
rocket.x86_64: W: no-manual-page-for-binary init
rocket.x86_64: W: no-manual-page-for-binary metadatasvc
rocket.x86_64: W: no-manual-page-for-binary rkt
2 packages and 1 specfiles checked; 5 errors, 10 warnings.


I'm kinda worried about the 'init' binary name, also the pxe image is _huge_

Comment 1 kushaldas@gmail.com 2014-12-02 21:38:13 UTC
I opened https://github.com/coreos/rocket/issues/173 for the 'init' binary name.

Comment 2 Lokesh Mandvekar 2014-12-02 23:10:24 UTC
init is installed in %{_libexecdir}/rocket/stage1 as per issue URL in Comment 1.

Spec and SRPM updated, links are same as in Comment 0.

Comment 3 Jan Chaloupka 2014-12-03 12:15:53 UTC
1) Is it build only on x86_64?

2) As for Fedora, there cannot be bundled third party sources. For rocket, there is Godeps folder, which must be broken into at least two dependencies on golang.org/x/text and golang.org/x/crypto/.

$ go2fed ggi -c
	golang.org/x/crypto/ripemd160
	golang.org/x/text/encoding
	golang.org/x/text/encoding/charmap
	golang.org/x/text/encoding/japanese
	golang.org/x/text/encoding/korean
	golang.org/x/text/encoding/simplifiedchinese
	golang.org/x/text/encoding/traditionalchinese
	golang.org/x/text/encoding/unicode
	golang.org/x/text/transform

3) What about devel subpackage?

Comment 4 Dan Horák 2014-12-03 13:22:38 UTC
I think http://fedoraproject.org/wiki/Packaging:Guidelines#No_inclusion_of_pre-built_binaries_or_libraries applies to the PXE image in Source1

Comment 5 Matthew Miller 2014-12-03 15:18:18 UTC
(In reply to Dan Horák from comment #4)
> I think
> http://fedoraproject.org/wiki/Packaging:Guidelines#No_inclusion_of_pre-
> built_binaries_or_libraries applies to the PXE image in Source1

Err, yes. That's an entire whole 'nuther Linux distribution there.

Comment 6 Tom "spot" Callaway 2014-12-03 19:02:14 UTC
Do I want to know why this package needs an entire prebuilt PXE image of another Linux distribution?

Comment 7 Lokesh Mandvekar 2014-12-03 21:39:42 UTC
(In reply to Tom "spot" Callaway from comment #6)
> Do I want to know why this package needs an entire prebuilt PXE image of
> another Linux distribution?

From what I've gathered as yet, the rkt (stage 0 binary) depends on stage1_rootfs which is gotten from the pxe image. Commenting out the part which fetches the pxe image from build script, I see this:

$ ./build 
Building actool...
Building ACE validator...
Building init (stage1)...
Packaging init (stage1)...
Building rkt (stage0)...
gopath/src/github.com/coreos/rocket/stage0/run.go:35:2: cannot find package "github.com/coreos/rocket/stage0/stage1_rootfs" in any of:
	/usr/lib/golang/src/github.com/coreos/rocket/stage0/stage1_rootfs (from $GOROOT)
	/usr/share/gocode/src/github.com/coreos/rocket/stage0/stage1_rootfs (from $GOPATH)
	/home/lsm5/gocode/src/github.com/coreos/rocket/stage0/stage1_rootfs
	/home/lsm5/repositories/github.com/coreos/rocket/gopath/src/github.com/coreos/rocket/stage0/stage1_rootfs

Maybe we could try a fedora rootfs instead, but no idea how well that'd work, also gotta check with upstream about that (if we're to pursue that option)

Correct me if I'm wrong, people

Comment 8 Lokesh Mandvekar 2014-12-03 21:41:19 UTC
(In reply to Jan Chaloupka from comment #3)
> 1) Is it build only on x86_64?

Though the build process doesn't complain for x86 and arm, upstream's README mentions support only for x86_64.

> 
> 2) As for Fedora, there cannot be bundled third party sources. For rocket,
> there is Godeps folder, which must be broken into at least two dependencies
> on golang.org/x/text and golang.org/x/crypto/.
> 
> $ go2fed ggi -c
> 	golang.org/x/crypto/ripemd160
> 	golang.org/x/text/encoding
> 	golang.org/x/text/encoding/charmap
> 	golang.org/x/text/encoding/japanese
> 	golang.org/x/text/encoding/korean
> 	golang.org/x/text/encoding/simplifiedchinese
> 	golang.org/x/text/encoding/traditionalchinese
> 	golang.org/x/text/encoding/unicode
> 	golang.org/x/text/transform

Ohh yes... thanks for this, will update spec.

> 
> 3) What about devel subpackage?

Will update this too

Comment 9 Tomas Tomecek 2014-12-04 09:43:04 UTC
Wanted to give rocket a shot, so I made your spec a source of information; couple points:

 * latest build of go-bindata is 0.0.6, while your spec requires 3.0.7-1 (on the other hand, I can see that you have built the new version in koji, looks like that repos weren't regenerated yet)

 * my VM missed gnupg, might be a good idea to add it to buildrequires

 * (same point for tar but since it's in minimum build env set, it doesn't make sense to make it a BR I guess)

Comment 10 Lokesh Mandvekar 2014-12-04 22:54:14 UTC
(In reply to Tomas Tomecek from comment #9)
> Wanted to give rocket a shot, so I made your spec a source of information;
> couple points:
> 
>  * latest build of go-bindata is 0.0.6, while your spec requires 3.0.7-1 (on
> the other hand, I can see that you have built the new version in koji, looks
> like that repos weren't regenerated yet)

yup only built for rawhide (mostly to get this going), I'll push builds for others soon, co-maintainers welcome :)

> 
>  * my VM missed gnupg, might be a good idea to add it to buildrequires

hmm, koji didn't seem to complain, but I will double-check

> 
>  * (same point for tar but since it's in minimum build env set, it doesn't
> make sense to make it a BR I guess)

ack

Comment 11 Matthew Miller 2014-12-04 23:05:54 UTC
(In reply to Lokesh Mandvekar from comment #7)
> Maybe we could try a fedora rootfs instead, but no idea how well that'd
> work, also gotta check with upstream about that (if we're to pursue that
> option)
> 
> Correct me if I'm wrong, people

If the intention is to require CoreOS as the stage1 rootfs it's kind of a nonstarter for every other distribution....

Comment 12 Timothy St. Clair 2014-12-05 03:20:37 UTC
As I had mentioned: 

An important point to clarify is that two things were announced:

1. a spec (App Container) https://github.com/coreos/rocket/blob/master/app-container/SPEC.md
2. and an implementation (Rocket).
 
Meaning, there may well be several implementations to the spec.

Comment 13 Lokesh Mandvekar 2014-12-05 06:27:42 UTC
(In reply to Timothy St. Clair from comment #12)
> As I had mentioned: 
> 
> An important point to clarify is that two things were announced:
> 
> 1. a spec (App Container)
> https://github.com/coreos/rocket/blob/master/app-container/SPEC.md
> 2. and an implementation (Rocket).
>  
> Meaning, there may well be several implementations to the spec.

Yup, comment from upstream author:
-----
The coreos rootfs is by no means a hard requirement of rocket - we intend for stage1 and stage0 to be decoupled and modular.

However, there's a fairly close relationship between the stage1 init binary and the stage1 rootfs, so if you want to change the latter, you have two options:

 - Replace it with something that's still compatible with the existing stage1 init binary. This will probably be fairly plausible with something Fedora-based, as long as the systemd is relatively recent (this is basically the main thing we use, as you can see in the mkrootfs.sh)
 - Roll your own init binary as well as rootfs. In this case, you are free to use whatever you like in both, as long as you can continue to work with the output that the stage0 generates, and then comply with the ACE spec
-----

Origin: https://github.com/coreos/rocket/issues/173#issuecomment-65739099

Comment 14 Lokesh Mandvekar 2014-12-15 03:16:53 UTC
%changelog
* Sun Dec 14 2014 Lokesh Mandvekar <lsm5@fedoraproject.org> - 0.1.1-1
- use upstream v0.1.1
- mkrootfs uses fedora docker base image from koji
via Václav Pavlin <vpavlin@redhat.com>

Spec URL: https://lsm5.fedorapeople.org/rocket/rocket.spec
SRPM URL: https://lsm5.fedorapeople.org/rocket/SRPMS/rocket-0.1.1-1.fc22.src.rpm

mkrootfs.sh: https://lsm5.fedorapeople.org/rocket/mkrootfs.sh

Spot, this should hopefully take care of FE-Legal.

Jan, I'm yet to address -devel subpackage (and probably other deps in Godeps.json), will do so asap. Btw, new build deps for this have been put up for review.

Not sure yet if koji will find the koji base image via the URL, else it'll have to be included in the SRPM itself.

Comment 15 kushaldas@gmail.com 2014-12-15 05:14:26 UTC
(In reply to Lokesh Mandvekar from comment #14)

> 
> Not sure yet if koji will find the koji base image via the URL, else it'll
> have to be included in the SRPM itself.

You will have to include it in the SRPM as generally koji builders do not get network access.

Comment 16 Lokesh Mandvekar 2014-12-15 05:43:49 UTC
(In reply to Kushal Das from comment #15)
> (In reply to Lokesh Mandvekar from comment #14)
> 
> > 
> > Not sure yet if koji will find the koji base image via the URL, else it'll
> > have to be included in the SRPM itself.
> 
> You will have to include it in the SRPM as generally koji builders do not
> get network access.

Hmm, does this also apply to images generated via (and present in) koji itself? If yes, could we make an exception for this? Just that the image addition will ridiculously bloat up the srpm :\

Comment 17 kushaldas@gmail.com 2014-12-15 05:48:39 UTC
I think you will have to open a FESCo ticket CCing rel-eng for the same. I guess they block these accesses due to security reasons.

Comment 18 Ralf Corsepius 2014-12-15 06:04:57 UTC
(In reply to Kushal Das from comment #15)
> (In reply to Lokesh Mandvekar from comment #14)
> 
> > 
> > Not sure yet if koji will find the koji base image via the URL, else it'll
> > have to be included in the SRPM itself.
> 
> You will have to include it in the SRPM as generally koji builders do not
> get network access.

That's not the point. SRPMS are supposed to be self-contained/usable stand-alone and to be built from constant sources.

In short: Network access to fetch sources is strictly prohibited in Fedora.

Comment 19 kushaldas@gmail.com 2014-12-15 08:57:48 UTC
# rkt fetch https://github.com/coreos/etcd/releases/download/v0.5.0-alpha.4/etcd-v0.5.0-alpha.4-linux-amd64.aci
Downloading aci: [===================                          ] 3.01 MB/6.78 MB
sha256-6635e9cbe18c6f51e8c70c143948df111b5626db39198182fbeb9277beb606db

# find /var/lib/rkt/cas/blob/
/var/lib/rkt/cas/blob/
/var/lib/rkt/cas/blob/sha256
/var/lib/rkt/cas/blob/sha256/66
/var/lib/rkt/cas/blob/sha256/66/sha256-6635e9cbe18c6f51e8c70c143948df111b5626db39198182fbeb9277beb606db

# rkt run sha256-6635e9cbe18c6f51e8c70c143948df111b5626db39198182fbeb9277beb606db
Failed to execute nspawn: no such file or directory

Comment 20 Tom "spot" Callaway 2014-12-15 14:46:15 UTC
I don't think that mkrootfs.sh is going to work.

A) There is no networking in the koji runtime instances, so it can't download anything, even from koji.
B) Including that binary image into the SRPM is ... troublesome. I'm not confident that distributing the binary RPM along with the SRPM guarantees that we remain in compliance with all of the licenses in that binary docker image.

Is there no way we can distribute this tool without a binary distribution image? If the answer is "the tool doesn't do anything without one", the options are:

1) Add a README.fedora that points to the latest Fedora image and let the user figure it out AND/OR Include a script to download the latest Fedora image (but don't run it in a %post scriptlet please).
2) Somehow get the latest Fedora image into an RPM where the corresponding SRPM has the BuildRequires to properly build it from the RPMs in the Fedora tree, then Requires on that image RPM here. (This sounds convoluted, and it is, but it ensures that we are in license compliance).
3) Figure out a way to document and track the source revisions of EVERY component inside the binary Fedora image, then work with rel-eng to ensure that as long as we distribute the binary Fedora image inside of this package, the other SRPMs remain available (and accurate). This is going to be painful, so I strongly advise against it.

I'm open to other suggestions, but this is all that comes to mind at the moment.

Comment 21 Lokesh Mandvekar 2014-12-17 13:48:24 UTC
(In reply to Tom "spot" Callaway from comment #20)

> 1) Add a README.fedora that points to the latest Fedora image and let the
> user figure it out AND/OR Include a script to download the latest Fedora
> image (but don't run it in a %post scriptlet please).

If I'm reading this right, this option still _does not_ allow me to include the fedora image in the srpm but leave it to the user to fetch it, correct? If that is the case, this probably won't be doable.

> 2) Somehow get the latest Fedora image into an RPM where the corresponding
> SRPM has the BuildRequires to properly build it from the RPMs in the Fedora
> tree, then Requires on that image RPM here. (This sounds convoluted, and it
> is, but it ensures that we are in license compliance).

Does this involve building the fedora docker base image from scratch in the other rpm (something like http://koji.fedoraproject.org/koji/taskinfo?taskID=8289768 ) or can I simply add the image tarball in the other rpm?

Comment 22 Tom "spot" Callaway 2014-12-17 15:41:56 UTC
(In reply to Lokesh Mandvekar from comment #21)

> If I'm reading this right, this option still _does not_ allow me to include
> the fedora image in the srpm but leave it to the user to fetch it, correct?
> If that is the case, this probably won't be doable.

Yes. This is correct.

> > 2) Somehow get the latest Fedora image into an RPM where the corresponding
> > SRPM has the BuildRequires to properly build it from the RPMs in the Fedora
> > tree, then Requires on that image RPM here. (This sounds convoluted, and it
> > is, but it ensures that we are in license compliance).
> 
> Does this involve building the fedora docker base image from scratch in the
> other rpm (something like
> http://koji.fedoraproject.org/koji/taskinfo?taskID=8289768 ) or can I simply
> add the image tarball in the other rpm?

Yes, it would involve building the fedora docker base image from scratch. Now, the process which currently generates that image (hopefully by running some sort of tool or script to generate the image from pre-built Fedora RPMS) could also generate an RPM with:
* a list of the Name-Version-Release data of the binary Fedora RPMS, and their corresponding SRPMS, and instructions on where to find these items in Koji.
* a copy of the Fedora docker base image

The idea being that anyone who gets that Fedora docker base image via RPM will also get a list of the source and binary Fedora RPMS used to compose it, and can go get them from Koji or the repositories in order to comply with the source licenses. (There is a minimal risk of us garbage collecting these builds depending on when this is done...)

If we were trying to be very clever, we'd wrap all of the image creation logic inside a SRPM with the proper BuildRequires, so that someone could simply rpmbuild --rebuild that SRPM to generate a custom (updated/tweaked) Fedora docker base image in an RPM. However, this aspect of it is not a legal requirement, just a nice-to-have.

Comment 23 Colin Walters 2014-12-17 18:06:09 UTC
So, have you guys seen how libguestfs does it?

See: http://libguestfs.org/supermin.1.html

Basically at runtime it pulls in a subset of the host binaries, verifying at startup they haven't changed.

This means the image automatically updates any time host content changes, and it doesn't need another delivery vehicle, or a separate build process for that image.

Comment 24 Daniel Walsh 2014-12-17 19:19:26 UTC
Sounds like a great solution. can we break supermin out of kibguestfs, or just make Rocket require it?

Comment 25 Colin Walters 2014-12-17 19:30:56 UTC
It's already a separate tool.  Just need to patch rocket to use it on startup.

Comment 26 Daniel Walsh 2014-12-18 19:14:39 UTC
Awesome.

Lokesh just have Rocket require supermin, and then run supermin in post install or in its init file.

Comment 27 Colin Walters 2014-12-18 19:43:57 UTC
You want to do it on start, not in %post, because otherwise a new systemd or glibc RPM would require "yum reinstall rocket" or the like, which would be awkward (and wouldn't work on Atomic).

Comment 28 Daniel Walsh 2014-12-23 16:19:55 UTC
Ok, we should do this i nthe start of the daemon.

Comment 29 Iago López 2015-01-09 13:36:36 UTC
I just wanna mention that, as discussed in https://github.com/coreos/rocket/issues/43, fetching and running images should not be restricted to the superuser. To allow that, a "rocket" group should be created and permissions in /var/lib/rkt/ should be set accordingly. That is:

drwxrwxr-x  4 root rocket 4.0K Jan  9 10:37 cas/
drwxrwx---  4 root rocket 4.0K Jan  9 10:56 containers/
drwxrwxr-x  2 root rocket 4.0K Jan  9 10:56 tmp/

Then, any user belonging to "rocket" can fetch images and read them (running is not possible since systemd-nspawn requires root).

I think distribution packages are a good place for setting these.

Comment 30 Lokesh Mandvekar 2015-03-28 18:21:13 UTC
Just want to redirect people to https://github.com/coreos/rocket/issues/686#issuecomment-87199448 which has the latest updates of the wip rpm with links.

I guess it would be great if people could leave their comments/concerns over there such that we can sort things out directly with upstream in a single place.

I probably haven't addressed all comments in the bugzilla yet, but guess that can be done once we have the major build process agreeable with everyone.

Comment 31 Lokesh Mandvekar 2015-08-12 15:34:35 UTC
No longer blocks FE-Legal as we're not using a CoreOS image anymore.

Comment 32 Tom "spot" Callaway 2015-08-24 17:15:17 UTC
To be clear, please confirm that there is no longer any OS image bundled into this package (whether CoreOS or something else).

Comment 33 Upstream Release Monitoring 2016-01-10 23:10:43 UTC
lsm5's scratch build of rkt-0.15.0-2.git7575500.fc24.src.rpm for rawhide completed http://koji.fedoraproject.org/koji/taskinfo?taskID=12493432

Comment 34 Lokesh Mandvekar 2016-01-11 02:09:47 UTC
(In reply to Tom "spot" Callaway from comment #32)
> To be clear, please confirm that there is no longer any OS image bundled
> into this package (whether CoreOS or something else).

Spot, there's a stage1-host.aci built during a koji build and installed at rpm install time, and this is generated completely within a koji env, without using any pre-built OS image. See scratch build in Comment 33 above. Is this still gonna be a problem?

Spec URL: https://github.com/fedora-cloud/rkt-rpm/blob/master/rkt.spec

Comment 35 Tom "spot" Callaway 2016-01-11 15:54:51 UTC
Okay. I've looked through that, and outside of a lot of bundled go code, there are no OS images that I could see. Thanks for following through on this.

Comment 36 Lokesh Mandvekar 2016-01-11 19:21:40 UTC
(In reply to Tom "spot" Callaway from comment #35)
> Okay. I've looked through that, and outside of a lot of bundled go code,
> there are no OS images that I could see. Thanks for following through on
> this.

Rocking...thanks spot!

Comment 37 Jan Chaloupka 2016-01-11 19:32:30 UTC
There is definitely a lot of work done on this package, guys.

Lokesh, I can check the golang part of the spec when it is ready. Just let me know.

Comment 38 Lokesh Mandvekar 2016-01-11 19:43:46 UTC
(In reply to Jan Chaloupka from comment #37)
> There is definitely a lot of work done on this package, guys.
> 
> Lokesh, I can check the golang part of the spec when it is ready. Just let
> me know.

I hope the golang part isn't a blocker either, given that bundling golang was kinda okayed, coz unbundling would probably be a perpetual task which imho we could handle after package approval.

Comment 39 Jan Chaloupka 2016-01-11 19:52:22 UTC
As rkt does not depend on docker or any other problematic package (e.g. kubernetes), I believe rkt will have to be debundled before review approval. As by the new policy, you can bundled deps if the upstream does not give you any other choice and the essence of the golang packages does not allow you to reasonably debundle (e.g. conflicting dependencies, collision with RHEL as in epel7).

Comment 40 Lokesh Mandvekar 2016-01-11 21:12:32 UTC
$ gofed check-deps -v
import path github.com/appc/cni/pkg/invoke not found
import path github.com/appc/cni/pkg/ip not found
import path github.com/appc/cni/pkg/ipam not found
import path github.com/appc/cni/pkg/ns not found
import path github.com/appc/cni/pkg/skel not found
import path github.com/appc/cni/pkg/types not found
import path github.com/appc/cni/plugins/ipam/dhcp not found
import path github.com/appc/cni/plugins/ipam/host-local not found
import path github.com/appc/cni/plugins/main/bridge not found
import path github.com/appc/cni/plugins/main/ipvlan not found
import path github.com/appc/cni/plugins/main/macvlan not found
import path github.com/appc/cni/plugins/main/ptp not found
import path github.com/appc/cni/plugins/meta/flannel not found
import path github.com/appc/docker2aci/lib not found
import path github.com/appc/docker2aci/tarball not found
import path github.com/appc/goaci/proj2aci not found
package golang-github-appc-spec not found in golang.repos
package golang-github-appc-spec not found in golang.repos
package golang-github-appc-spec not found in golang.repos
package golang-github-appc-spec not found in golang.repos
import path github.com/appc/spec/pkg/acirenderer not found
import path github.com/appc/spec/pkg/device not found
package golang-github-appc-spec not found in golang.repos
package golang-github-appc-spec not found in golang.repos
import path github.com/bradfitz/http2 not found
import path github.com/camlistore/camlistore/pkg/errorutil not found
import path github.com/camlistore/lock not found
import path github.com/coreos/gexpect not found
import path github.com/coreos/go-iptables/iptables not found
Traceback (most recent call last):
  File "/usr/share/gofed/checkDeps.py", line 94, in <module>
    ups_commits = modules.Repos.getRepoCommits(path, upstream, pull=options.pull)
  File "/usr/share/gofed/modules/Repos.py", line 173, in getRepoCommits
    os.chdir('/'.join(path.split('/')[:-1]))
OSError: [Errno 2] No such file or directory: '/var/lib/gofed/packages/golang-github-coreos-go-semver/upstream/'


the traceback aside, i'm not sure why it says golang-github-appc-spec not found, or does it mean something else? that package does exist on fedora

Comment 41 Jan Chaloupka 2016-01-12 07:34:54 UTC
golang.repos is outdated. If you add "golang-github-appc-spec         spec    https://github.com/appc/spec.git" line at the end of /var/lib/data/golang.repos, it will find it.

rkt-7575500011a56584755be5e9978c8cfef4aff433]# gofed check-deps Godeps/Godeps.json
import path github.com/appc/cni/pkg/invoke not found
import path github.com/appc/cni/pkg/ip not found
import path github.com/appc/cni/pkg/ipam not found
import path github.com/appc/cni/pkg/ns not found
import path github.com/appc/cni/pkg/skel not found
import path github.com/appc/cni/pkg/types not found
import path github.com/appc/cni/plugins/ipam/dhcp not found
import path github.com/appc/cni/plugins/ipam/host-local not found
import path github.com/appc/cni/plugins/main/bridge not found
import path github.com/appc/cni/plugins/main/ipvlan not found
import path github.com/appc/cni/plugins/main/macvlan not found
import path github.com/appc/cni/plugins/main/ptp not found
import path github.com/appc/cni/plugins/meta/flannel not found
import path github.com/appc/docker2aci/lib not found
import path github.com/appc/docker2aci/tarball not found
import path github.com/appc/goaci/proj2aci not found
package golang-github-appc-spec outdated
import path github.com/appc/spec/pkg/device not found
import path github.com/bradfitz/http2 not found
import path github.com/camlistore/camlistore/pkg/errorutil not found
import path github.com/camlistore/lock not found
import path github.com/coreos/gexpect not found
import path github.com/coreos/go-iptables/iptables not found
package golang-github-coreos-go-systemd outdated
import path github.com/coreos/go-tspi/attestation not found
import path github.com/coreos/go-tspi/tpmclient not found
import path github.com/coreos/go-tspi/tspi not found
import path github.com/coreos/ioprogress not found
import path github.com/cpuguy83/go-md2man/md2man not found
import path github.com/cznic/b not found
import path github.com/cznic/bufs not found
import path github.com/cznic/exp/lldb not found
import path github.com/cznic/fileutil not found
import path github.com/cznic/mathutil not found
import path github.com/cznic/ql not found
import path github.com/cznic/sortutil not found
import path github.com/cznic/strutil not found
import path github.com/cznic/zappy not found
import path github.com/d2g/dhcp4 not found
import path github.com/d2g/dhcp4client not found
import path github.com/dustin/go-humanize not found
package golang-googlecode-goprotobuf outdated
import path github.com/hydrogen18/stoppableListener not found
import path github.com/inconshreveable/mousetrap not found
import path github.com/kballard/go-shellquote not found
package golang-github-kr-pty outdated
import path github.com/naegelejd/go-acl not found
import path github.com/pborman/uuid not found
import path github.com/petar/GoLLRB/llrb not found
import path github.com/peterbourgon/diskv not found
package golang-github-russross-blackfriday outdated
package golang-github-shurcooL-sanitized_anchor_name outdated
package golang-github-spf13-cobra outdated
package golang-github-spf13-pflag outdated
package golang-github-vishvananda-netlink outdated
package golang-googlecode-go-crypto not found in golang.repos
package golang-googlecode-go-crypto not found in golang.repos
package golang-googlecode-go-crypto not found in golang.repos
package golang-googlecode-net outdated
import path golang.org/x/net/internal/timeseries not found
import path golang.org/x/sys/unix not found
package golang-googlecode-tools not found in golang.repos
import path google.golang.org/grpc not found
kubernetes: package commit bf56e235826baded1772fb340266b8419c3e8f30 not found
golang-googlecode-go-decimal-inf: package commit  not found

A lot of new package I see for the first time.

Comment 42 Jan Chaloupka 2016-01-14 13:11:40 UTC
Ahh, rkt depends on k8s.io/kubernetes/pkg/api/resource which is one of those packages that are changing all the time and can not provide stable dependency. From that reason, rkt can not be built from de-bundled dependencies.

As per point 3 in [1]:

"All packages whose upstreams have no mechanism to build against system libraries may opt to carry bundled libraries, but if they do, they must include Provides: bundled(<libname>) = <version> in their RPM spec file."

rkt package could be partially built from bundled and partially from debundled. However, at the end some dependencies has to be bundled. The closest kubernetes commit with the required k8s.io/kubernetes/pkg/api/resource is 
bbb447916674cc4bba3ad1e98414af4de356f9d6 from 2015-02-02. Since that time there have been changes in the resource package and we can predict there will be others changing its functionality. As the required commit does not belong to versioned release and even if at the moment it would work to build rkt with kubernetes as a dependency, in future there is a high chance of bundling it anyway. At the same time the currently packaged commit of kubernetes can introduce improper and unexpected functionality. Thus it is impossible to built rkt from de-bundled dependencies and at the same time keep rkt stable.

We can keep the current dependencies open until we have sufficient tooling which would tells us how much de-bundling is needed.

[1] https://fedorahosted.org/fesco/ticket/1483#comment:17

Comment 43 Lokesh Mandvekar 2016-01-14 16:04:59 UTC
w00t!! saved me a lot of time, I'll work on those golang reviews anyway, but glad it won't block rkt.

See updates...

Spec URL: https://github.com/fedora-cloud/rkt-rpm/blob/master/rkt.spec

Scratch build should be auto-posted once finished.

Comment 44 Upstream Release Monitoring 2016-01-14 16:06:56 UTC
lsm5's scratch build of rkt-0.15.0-3.git5988b72.fc24.src.rpm for f23 completed http://koji.fedoraproject.org/koji/taskinfo?taskID=12548044

Comment 45 Lokesh Mandvekar 2016-01-14 16:08:41 UTC
and rawhide scratch build: https://koji.fedoraproject.org/koji/taskinfo?taskID=12548032

Comment 46 Lokesh Mandvekar 2016-01-14 16:30:25 UTC
forwarding a question from Alban Crequy, do we need to install licenses of those bundled dependencies if we're using those to build rkt (and likewise for docker et.al.), or is it ok to skip those?

Comment 47 Tom "spot" Callaway 2016-01-14 16:34:49 UTC
If the go code is shipped in the rkt binary or src package, then the license file needs to be in that binary or src package.

This means that if you may have code which is only used at build-time but not packaged into the rkt binary rpm, then you only need to make sure the license text is in the src.rpm. However, this is rare, and most people simply include the license texts in the binary rpm to be safe (as %license).

Comment 48 Upstream Release Monitoring 2016-01-14 17:45:36 UTC
lsm5's scratch build of rkt-0.15.0-4.git5988b72.fc24.src.rpm for rawhide completed http://koji.fedoraproject.org/koji/taskinfo?taskID=12548934

Comment 49 Lokesh Mandvekar 2016-01-14 19:00:48 UTC
(In reply to Tom "spot" Callaway from comment #47)
> If the go code is shipped in the rkt binary or src package, then the license
> file needs to be in that binary or src package.
> 
> This means that if you may have code which is only used at build-time but
> not packaged into the rkt binary rpm, then you only need to make sure the
> license text is in the src.rpm. However, this is rare, and most people
> simply include the license texts in the binary rpm to be safe (as %license).

Thanks spot, we're not shipping the sources for now, so I think we're fine. I'll include licenses once we ship -devel.

Comment 50 Jan Chaloupka 2016-01-14 19:38:03 UTC
There is nothing wrong about golang packaging. As the rpm does not carry any libraries, there is no need to used Provides: bundled(...). And you don't build devel subpackage so no Provides: bundled(...) for that either.

The spec file conforms to the current golang guidelines draft.

Comment 51 Lokesh Mandvekar 2016-01-14 19:50:23 UTC
Thanks Jan for the approval. There are still some issues RE: selinux and rkt unitfiles, but those aren't packaging issues. Guess it'd be preferable to wait until those issues are fixed before adding koji builds.

unitfiles: https://github.com/coreos/rkt/issues/1978

selinux: https://github.com/coreos/rkt/issues/1882 and https://github.com/coreos/rkt/issues/1727#issuecomment-163608916

Comment 52 Gwyn Ciesla 2016-01-15 14:26:31 UTC
Package request has been approved: https://admin.fedoraproject.org/pkgdb/package/rkt

Comment 53 Haïkel Guémar 2016-02-27 20:42:22 UTC
Built and imported in F24 and rawhide for over a week now.

Comment 54 Joe Brockmeier 2016-02-29 13:51:48 UTC
(In reply to Haïkel Guémar from comment #53)
> Built and imported in F24 and rawhide for over a week now.

Fantastic! This is great news.

Comment 55 Lokesh Mandvekar 2016-02-29 14:50:32 UTC
(In reply to Joe Brockmeier from comment #54)
> (In reply to Haïkel Guémar from comment #53)
> > Built and imported in F24 and rawhide for over a week now.
> 
> Fantastic! This is great news.

Umm, don't celebrate just yet, this only works with setenforce 0 for now. Also, I hope this doesn't get pushed into f24 stable/release until that's sorted. This got into f24 when it was cut from rawhide.

Comment 56 Daniel Walsh 2016-02-29 17:26:04 UTC
Working with coreos on the SELinux issues, a couple can be fixed simply, the others might involve code changes in RKT.


Note You need to log in before you can comment on or make changes to this bug.