Bug 1641187 - Please make pm_request available (and working) in copr
Summary: Please make pm_request available (and working) in copr
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Copr
Classification: Community
Component: backend
Version: unspecified
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Miroslav Suchý
QA Contact:
URL:
Whiteboard:
Depends On: 1642840 1641252 1641253 1642844
Blocks: 1629371
TreeView+ depends on / blocked
 
Reported: 2018-10-19 20:09 UTC by Nicolas Mailhot
Modified: 2019-04-19 21:53 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-04-19 21:53:23 UTC


Attachments (Terms of Use)

Description Nicolas Mailhot 2018-10-19 20:09:28 UTC
So the Go (golang) ecosystem is a morass of fast-changing software, with massive code reuse, and components that get created/forked/renamed/deprecated at a fast pace. This is similar to other "modern" language ecosystems such as javascript.

This has led to mass-generation of Go spec files in Fedora from code analysis tools (and has prevented any form of official Fedora Go packaging guidelines).

However, generating specs outside rpm leads to specs that rot at a fast pace. No one really understands or audits the generated code, and as soon as you need to adapt it due to some upstream quirk you lose the ability to regenerate it cleanly.

And you can not ignore golang or javascript, the first is used by pretty much any container-oriented software, the other by pretty much anything that needs to present a web ui.

Therefore I've been trying for a year to put back the generation logic within rpm macros, so it's centralized, audited and and controlled by Fedora, and the generation logic is cleanly separated from human adaptations in the corresponding spec files.

For build requires, that means computing the code needs in %prep and getting mock to install the corresponding packages. The approach agreed on with FPC members and upstream mock was to use the pm_request mock plug-in and the corresponding logic written by the Java sig in javapackages.
https://github.com/rpm-software-management/mock/issues/160

Unfortunately the Java sig code was too imbricated with Java specific things to be reusable so I ended up writing a separate mock pm request client
https://github.com/nim-nim/mock-install
https://copr.fedorainfracloud.org/coprs/nim/mock-install/
https://bugzilla.redhat.com/show_bug.cgi?id=1629371

And now I find out pm_request is not available in copr (don't know it it was before and has been removed since, or if it was never enabled because the java sig built its stuff elsewhere)

Anyway:
1. please enable pm_request in copr
2. please make sure that it works
https://github.com/rpm-software-management/mock/issues/218

How to test:
1. take a mock install binary from
https://copr.fedorainfracloud.org/coprs/nim/mock-install/
2. use any spec you like that calls mock-install <package-name> from %prep

It's ridiculous that I can get copr to download random software from the internet, but I can't get it to install Fedora packages from Fedora repositories using Fedora infra like mock

Comment 1 clime 2018-10-19 21:43:19 UTC
Note that maybe you want to give a try to SCM rpkg method, which is able to pregenerate certain parts of spec file during srpm generation phase including BuildRequires.

I am not sure how exactly BuildRequires for Go packages are generated but it would be very cool to include the macros for doing that in rpkg standard macro library.

I don't really mind a different method here and but maybe it would be good to join efforts here.

See https://docs.pagure.org/rpkg-util/spec_templates_from_scratch.html how rpkg spec templates work. But you will see there only the basic Git versioning macros.
I would be happy if we could make the library wider.

Note that I am planning to get support for this feature (spec templates) into Koji too (although you might be faster) so it will not stay limited to Copr Build System only.

So I am just mentioning this before we can take a proper look at this issue.

Comment 2 Nicolas Mailhot 2018-10-19 22:11:07 UTC
Hi,

At the point of the automation work you just have a 

%goprep

call in %prep that does the heavy lifting (source unpacking, setup of the GOPATH tree, call to the code analysis engine, relaying of the result to mock)

Though of course real-world upstream code is imperfect, in some cases full of garbage, so you may need some patching and removing before the result can be fed to code analysis and produce clean buildrequires.

I'm still working on defining the simpler and most effective cleanup patterns of upstream code. The basic tech works, I know how to fix code manually, I just need to formalize the usual way you clean up upstream mistakes.

So I believe there won't be any problem to include the result in rpkg. Just let me finish the human packaging template examples first :)

Comment 3 clime 2018-10-19 22:54:14 UTC
Ok, sure, just note that rpkg computes the resulting spec file at srpm build phase (and not rpm build phase) and it doesn't need mock or anything because it just statically renders computed values of BuildRequires into the resulting spec file e.g. like this:

BuildRequires:x
BuildRequires:y

The resulting rpm spec is then packed into srpm (together with sources) and only after that, the srpm with already computed BR values is passed to mock for a build.

So in case, we would like to include this into rpkg, we would need something that does all the code analysis and other needed logic and produces block of BuildRequires in the end:

BuildRequires: <pkg1>
BuildRequires: <pkg2>

to stdout.

There is no need to communicate with mock or anything. So actually packaging this will be slightly different. The rpkg macro will be e.g. {{{ goprep }}} and will be placed where BuildRequires are normally placed in a spec file.

I am not saying you should jump to do something else than you are currently doing but I think it might help to understand how rpkg actually may support this case with your help.

Comment 4 Nicolas Mailhot 2018-10-20 09:07:27 UTC
You can see in line 2500 of
https://paste.fedoraproject.org/paste/hglkYSWJAHd2YuGBX2XnsA

That deps are put in an env var before being passed to mock.

It needs to be in prep because you have all the unpack logic before, and that logic is tricky if you have a package that includes multiple sources, some of which are patched or fixed in some way in prep. Doing it in rpkg would require creating a simili-prep rpkg section in spec files.

Thus, I would be more interested if rpkg allowed me to do BuildRequires-like declarations in prep, and rpmbuild just picked up those at the end of prep, and it was transparent to the spec writer. %prep is more complex than the header section where BuildRequires are manually declarated right now, so simplifying this header section, at the cost of %prep complexity, is a huge no-no.

Also, I hope you realise the prepy things the macro does have their own requirements, so it's not just "rpkg will pregenerate at rpmbuild stage" it's also "rpkg will require all the stuff needed by prep to be in the default build root", unless you add a separate Requires level just for rpkg. Right now we have a nice system where one only needs a single macro-only package in the buildroot, and it adds transparently prep requirements for packages that need them only with the BR stage :

https://copr-be.cloud.fedoraproject.org/results/nim/go-macros-3/fedora-rawhide-x86_64/00812443-golang-github-shopspring-decimal/builder-live.log

→ nothing Go specific in the default buildroot except go-srpm-macros, that triggers a BR on go-rpm-macros for Go packages, and go-rpm-macro pulls in all the Go env (including the one used in %prep)

https://copr-be.cloud.fedoraproject.org/results/nim/fontpackages-2/fedora-rawhide-x86_64/00811647-sil-mondulkiri-extra-fonts/build.log.gz

→ nothing fonts specific in the default buildroot except fonts-srpm-macros, that triggers a BR on fonts-rpm-macros for fonts packages, and fonts-rpm-macro pulls in all the fonts env (including the one used in %prep)

(both macro sets use the same zero-default-buildroot-footprint except for macro files approcah)

So, I'm not saying it can't be done by rpkg, the concepts are close enough it would probably work with little adaptation, but there is clearly quite a lot of engineering to do to move rpkg from "interesting concept" to "ready for everyday mass use" stage. And I'm far from convinced it would end in a net simplification of fedora specs and infra.

Besides:

1. at this point builds work in plain rpmbuild and mock with pm_request, so we can do all the private builds we want, including for el7

2. the only question is how to push the result to Fedora, and that requires only activating a mock option in copr and koji, and mock is one of Fedora's standard infra build tools (even though this is probably slightly more work because the nswpawn guys block the socket pm_request uses, but that's just fixing nspawn and mock which is a win-win for Fedora)

3. the current syntax is familiar to every packager and reviewer, it's just a fooprep call in prep. There's no separate *setup call anymore

4. I have (and I'm not the only Go packager concerned in Fedora) 771 existing specs that need this

So, I'm not going to rush to rewrite 771 checked and QA spec files to use rpkg, and make them rpkg-specific, when rpkg is not part of Fedora standard build tools (so that would force every single Go packager in the distro to install, deploy and configure rpkg on their systems), the resulting spec syntax is non standard (so that would require rewriting many Fedora packaging guidelines, and forming reviewers on the new rpkg guidelines), and so on. Lots of work to rewrite specs, lots of work to QA the result, high chance rpkg is so alien to Fedora packaging customs the result will be rejected. Also, high chance the result will be more verbose specs, since the current solution is a one-line macro call in prep, and my primary objective is to make Go spec terser and easier to maintain for Fedora packagers.

So, the cheapest and easiest way for Fedora and me right now is to just use pm_request. That the Java SIG could probably use too as they wrote the thing in the first place.

And then in second stage you can arrive and show rpkg can make all those specs better, and get Fedora buy-in.

If you want to play with it I can sent you my Go spec stash as it is now. I'd have pointed you to a copr, which is how such engineering tests are supposed to be done, but I won't load copr with 700+ builds that I know will fail in the absence of pm-request.

The Go macros themselves are in
https://copr.fedorainfracloud.org/coprs/nim/go-macros-3/

You can ignore the golang-* packages, I stopped the build as soon as I saw pm request was absent so they're not complete.

It all works because @rh people sneaked bundled code in go-compilers, so the golist command the macros need is available, even though its built is completely non conformant with Fedora best practices (that's the king of thing the new macros are supposed to help eradicating).

Comment 5 clime 2018-10-20 13:17:10 UTC
Well, I think most of your points about adoption and usability of rpkg can be answered.

As for this RFE, pm-request enable/disable checkbox (similar to network-enabled) might be desired according to https://github.com/rpm-software-management/mock/wiki/Plugin-PMRequest docs. 

We should wait for https://github.com/rpm-software-management/mock/issues/218 to get resolved first though.

Comment 6 Nicolas Mailhot 2018-10-20 13:44:45 UTC
Sure, I don't think they're blocking from a technical POW, they're just a huge pile of work to do before rpkg can get viable as solution.

As for https://github.com/rpm-software-management/mock/issues/218 that's probably just a bad security rule in selinux or systemd, but no one systemd or selinux side is going to look at it if infra does not ask it. So if that's the only blocker please ask dwalsh or Lennart to look at it, I can do it too, but things will go faster if infra asks.

Longer term I've described a cleaner more integrated solution in
https://github.com/rpm-software-management/rpm/issues/104
and
https://github.com/rpm-software-management/mock/issues/160

but it needs changes at least in mock, and also in rpm if you want rpm querying to support the result (as some people do). Short to mid term pm request is the only existing deployable working solution.

It's totally orthogonal to rpkg use or not, I sure hope rpkg will be able to template-out the result, it just does not need rpkg to work at the most basic level.

Comment 7 Pavel Raiskup 2018-10-25 08:27:04 UTC
(In reply to Nicolas Mailhot from comment #6)
> As for https://github.com/rpm-software-management/mock/issues/218 that's
> probably just a bad security rule in selinux or systemd, but no one systemd
> or selinux side is going to look at it if infra does not ask it. So if
> that's the only blocker please ask dwalsh or Lennart to look at it, I can do
> it too, but things will go faster if infra asks.

Could you help with debugging that issue?  Does that mean that you tried
to switch SELinux to Permissive and things were working fine?

> It's totally orthogonal to rpkg use or not, I sure hope rpkg will be able to
> template-out the result, it just does not need rpkg to work at the most
> basic level.

Agreed, dynamic BuildRequires have different scope than tools that generate
sources for SRPM (tito, rpkg, ..).

Comment 8 Nicolas Mailhot 2018-10-25 09:33:46 UTC
I'd be happy to run any test you want, to be honest I'm lost in all the layers of security here, so I don't quite know why to test.

I see mock creating the pm_request socket with or without nspawn

When nspawn is enabled, the pm request client can not talk to this socket anymore.

What tests do you want me to run?

Comment 9 Nicolas Mailhot 2018-10-25 09:35:38 UTC
Just permissive mode on the mock host? Something else that changes selinux settings in the buildroot?

Comment 10 Pavel Raiskup 2018-10-25 10:50:01 UTC
Dunno, capabilities (of the process), permissions, ownership, ...?

Comment 11 Pavel Raiskup 2018-10-25 10:53:37 UTC
I'd say that you should be able to start interactive /bin/bash process (from
%prep phase for example) and experiment with the socket.

Comment 12 Nicolas Mailhot 2018-10-25 11:56:50 UTC
I will retry but IIRC things running from spec the do not see at all the socket in nswpan mode. It's hidden by the security settings (selinux or container side). And one can see from the outside of the container, that mock did create the pm request socket in its usual place.

Comment 13 Pavel Raiskup 2019-04-19 21:53:23 UTC
Closing in favor of Dynamic Build Dependencies, which just landed in Mock - and
should be soon in RPM.


Note You need to log in before you can comment on or make changes to this bug.