Bug 2260538

We currently don't have a sponsor. This is our first package.

Taking this review.

(In reply to Matthew Yee from comment #0)
> Spec URL:
> https://github.com/sustainable-computing-io/kepler/blob/main/packaging/rpm/
> kepler.spec
> SRPM URL: N/A
> Description: Kepler (Kubernetes-based Efficient Power Level Exporter) uses
> eBPF to probe performance counters and other system stats, use ML models to
> estimate workload energy consumption based on these stats, and exports them
> as Prometheus metrics
> Fedora Account System Username: myee111

Please provide a spec file and an SRPM from URLs that can be curled so that fedora-review can fetch them.

Hey Neal

This URL should work:
https://raw.githubusercontent.com/sustainable-computing-io/kepler/main/packaging/rpm/kepler.spec

The RPM is available here https://github.com/sustainable-computing-io/kepler/releases/tag/v0.7.3

@Neal, anything you need please let us know asap, thanks

Spec review:

> %undefine _disable_source_fetch

This is not allowed. The Fedora build system does not provide internet access.

> Version:        %{getenv:_VERSION_}
> Release:        %{getenv:_RELEASE_}

This is not allowed. These values need to be static and managed from the spec file.

> BuildArch:      %{getenv:_ARCH_}

Ignoring the fact that this form is not allowed, this makes no sense and should be dropped. The build system will build for all architectures.

> License:        Apache License 2.0

This is not valid. This needs to be an SPDX license identifier tag.

Cf. https://docs.fedoraproject.org/en-US/packaging-guidelines/LicensingGuidelines/#_valid_license_short_names

> BuildRequires: systemd

I don't see a reason to require systemd when you only need the RPM macros. Instead, require "systemd-rpm-macros".

> BuildRequires: clang llvm llvm-devel zlib-devel make libbpf

Please format these such that each build dependency on its own line.

> Requires:       elfutils-libelf-devel

Are you sure you need the devel package at runtime?

> %ifarch x86_64
> GOARCH=amd64
> %endif

You're going to need aliases for all RPM arches in Fedora to map to Go arches.

> make genlibbpf _build_local GOOS=${GOOS} GOARCH=${GOARCH} ATTACHER_TAG=libbpf

The Go compilation build flags need to be passed in.

> install -d %{buildroot}/var/lib/kepler/data
> install -d %{buildroot}/var/lib/kepler/bpfassets
> install -d %{buildroot}/etc/kepler/kepler.config

"/var/lib" should be "%{_sharedstatedir}" and "/etc" should be "%{_sysconfdir}".

> install -p -m644 ./data/cpus.yaml %{buildroot}/var/lib/kepler/data/cpus.yaml
> install -p -m644 ./data/model_weight/acpi_AbsPowerModel.json %{buildroot}/var/lib/kepler/data/acpi_AbsPowerModel.json
> install -p -m644 ./data/model_weight/acpi_DynPowerModel.json %{buildroot}/var/lib/kepler/data/acpi_DynPowerModel.json
> install -p -m644 ./data/model_weight/intel_rapl_AbsPowerModel.json %{buildroot}/var/lib/kepler/data/intel_rapl_AbsPowerModel.json
> install -p -m644 ./data/model_weight/intel_rapl_DynPowerModel.json %{buildroot}/var/lib/kepler/data/intel_rapl_DynPowerModel.json

Are you sure this should be in "%{_sharedstatedir}"? It seems like these are static files and should be installed into "%{_datadir}".

> %post
>
> %systemd_post kepler.service

This is incomplete. 

Cf. https://docs.fedoraproject.org/en-US/packaging-guidelines/Scriptlets/#_systemd

> /var/lib/kepler/bpfassets/amd64_kepler.bpf.o

This should probably be installed into "%{_libexecdir}/kepler/bpfassets" instead.

Also one more thing...

> Source0:        https://github.com/sustainable-computing-io/kepler/archive/refs/tags/%{version}.tar.gz

This should be "%{url}/archive/%{version}/%{name}-%{version}.tar.gz".

> install -p -m644 ./bpfassets/libbpf/bpf.o/amd64_kepler.bpf.o %{buildroot}/var/lib/kepler/bpfassets/amd64_kepler.bpf.o

Is this amd64_kepler.bpf.o binary generated during the build or are we using the one from the git repo/tarball here?

For official Fedora builds, it will need to be built as part of the rpmbuild process (binary blobs are not allowed) and  that will result in a binary of the build system architecture (which is not necessarily x86_64).

| The build system will build for all architectures.

What I suspect is actually needed here is the intersection of 'all support golang architectures' and 'all supported bpf architectures' (which is not all architectures, it turns out).  There's a similar situation in the Grafana build, where we need golang and nodejs architectures only, and this spec file snippet is used there:

%global grafana_arches %{lua: go_arches = {}
  for arch in rpm.expand("%{go_arches}"):gmatch("%S+") do
    go_arches[arch] = 1
  end
  for arch in rpm.expand("%{nodejs_arches}"):gmatch("%S+") do
    if go_arches[arch] then
      print(arch .. " ")
  end
end}

However, I'm not aware of a bpf_arches macro - for PCP we are using a local macro defined as: 'x86_64 %{power64} aarch64 s390x'.

Thanks Neal! Will post a new spec for the Fedora requirements.

(In reply to Nathan Scott from comment #9)
> > install -p -m644 ./bpfassets/libbpf/bpf.o/amd64_kepler.bpf.o %{buildroot}/var/lib/kepler/bpfassets/amd64_kepler.bpf.o
> 
> Is this amd64_kepler.bpf.o binary generated during the build or are we using
> the one from the git repo/tarball here?
> 
> For official Fedora builds, it will need to be built as part of the rpmbuild
> process (binary blobs are not allowed) and  that will result in a binary of
> the build system architecture (which is not necessarily x86_64).

eBPF binaries, regardless of arch, apparently look like x86_64 ELF blobs, which means the only way to distinguish them is by name...

From the kepler git repo https://github.com/sustainable-computing-io/kepler.git ...

$ file ./bpfassets/libbpf/bpf.o/amd64_kepler.bpf.o 
./bpfassets/libbpf/bpf.o/amd64_kepler.bpf.o: ELF 64-bit LSB relocatable, eBPF, version 1 (SYSV), with debug_info, not stripped

My question remains - is this built during the rpmbuild process or is it pulled from the tarball?  I believe it needs to be built locally as part of the Fedora build each time - 
https://docs.fedoraproject.org/en-US/packaging-guidelines/what-can-be-packaged/#prebuilt-binaries-or-libraries

New question :) - if this is platform-independent eBPF bytecode, why prefix the name with amd64?  For people on other arches this is going to be a source of confusion - looking at my aarch64 laptop and my confusion RN for example ;)

Relevant excerpt from the packaging guidelines (link above):

| When you encounter prebuilt binaries in a package you MUST:
| 
|    Remove all pre-built program binaries and program libraries in %prep prior to the building of the package. Examples include, but are not limited to, *.class, *.dll, *.DS_Store, *.exe, *.jar, *.o, *.pyc, *.pyo, *.egg, *.so, *.swf files.
| 
|    Ask upstream to remove the binaries in their next release.

so kepler.spec needs a %prep section that removes ./bpfassets/libbpf/bpf.o/amd64_kepler.bpf.o prior to the build commencing.

From specfile:

> BuildRequires: [...] libbpf

How does Kepler load the BPF program? I assume it uses libbpf so libbpf should be a runtime dependency (i.e. in "Requires"), unless you're linking against it statically.

On the contrary, I'd expect libbpf-devel in "BuildRequires".

> New question :) - if this is platform-independent eBPF bytecode, why prefix the name with amd64?  For people on other arches this is going to be a source of confusion - looking at my aarch64 laptop and my confusion RN for example ;)

Agreed with Nathan here. BPF binaries are arch-independent (they are ELFs with a special "BPF" architecture) so the prefix shouldn't be there.

>  I believe it needs to be built locally as part of the Fedora build each time

Despite being arch-independent, I agree that the binary should be rebuilt as a part of the specfile, as per Fedora guidelines.
In fact, IIUC, that's exactly what

> make genlibbpf _build_local GOOS=${GOOS} GOARCH=${GOARCH} ATTACHER_TAG=libbpf

in the specfile does. So you just need to remove ./bpfassets/libbpf/bpf.o/amd64_kepler.bpf.o in the %prep phase, as mentioned above.

(In reply to Neal Gompa from comment #7)
> > /var/lib/kepler/bpfassets/amd64_kepler.bpf.o
> 
> This should probably be installed into "%{_libexecdir}/kepler/bpfassets"
> instead.

Actually, we've been trying to establish %{_libdir}/bpf/ as the place to put BPF object files, cf: https://src.fedoraproject.org/rpms/xdp-tools/blob/rawhide/f/xdp-tools.spec#_105

However, it looks like the loader code hard-codes the location of the object file: https://github.com/sustainable-computing-io/kepler/blob/main/pkg/bpfassets/attacher/libbpf_attacher.go#L41

This should probably be changed so the Makefile can override it.

(In reply to Viktor Malik from comment #15)
> From specfile:
> 
> > BuildRequires: [...] libbpf
> 
> How does Kepler load the BPF program? I assume it uses libbpf so libbpf
> should be a runtime dependency (i.e. in "Requires"), unless you're linking
> against it statically.
> 
> On the contrary, I'd expect libbpf-devel in "BuildRequires".

Looking at the top-level Makefile, it does appear that Kepler links statically against libbpf: https://github.com/sustainable-computing-io/kepler/blob/main/Makefile#L51

Not sure if we have a policy for this in Fedora, but at least in RHEL this won't be allowed. So at some point this needs to be changed to link dynamically against the system libbpf.

> > New question :) - if this is platform-independent eBPF bytecode, why prefix the name with amd64?  For people on other arches this is going to be a source of confusion - looking at my aarch64 laptop and my confusion RN for example ;)
> 
> Agreed with Nathan here. BPF binaries are arch-independent (they are ELFs
> with a special "BPF" architecture) so the prefix shouldn't be there.

AFAICT from the Kepler sources, it's arch-specific because the build includes different versions of vmlinux.h depending on the defined architecture: https://github.com/sustainable-computing-io/kepler/blob/main/bpfassets/libbpf/src/kepler.bpf.h

This should not be necessary for most accesses, CO-RE can be used instead. I am not sure if there are some arch-specific structs that Kepler needs to access, though, and whether CO-RE can handle those.

(In reply to Toke Høiland-Jørgensen from comment #16)
> 
> Looking at the top-level Makefile, it does appear that Kepler links
> statically against libbpf:
> https://github.com/sustainable-computing-io/kepler/blob/main/Makefile#L51
> 
> Not sure if we have a policy for this in Fedora, but at least in RHEL this
> won't be allowed. So at some point this needs to be changed to link
> dynamically against the system libbpf.

In Fedora, it is very much discouraged:

https://docs.fedoraproject.org/en-US/packaging-guidelines/#bundling

Won't do.