Can you check whether the transitive dependencies of the -client are a problem in that they'd enlarge the @core?  Mainly: libcurl.  If that's already there, I suggest leaving Recommends as it is and if necessary, adding this 47kB subrpm to the @core comps.

The current setup is the best compromise we've come up with so far.

Frank, is having elfutils-debuginfod-client is essential on every minimal host installation, so that it should be added to @core group?

> Frank, is having elfutils-debuginfod-client is essential on every minimal
> host installation, so that it should be added to @core group?

Not sure what sort of answer / rationale would be convincing here.  There exist a few intermittent debuginfo consumer tools in "@core" (systemd), if I'm not mistaken.  So maybe by that alone, the 50 kB (installed size) of this RPM is sufficiently justified.

It is also an indirect but real important functional dependency of a variety of larger debugger-type tools, (not sure whether any in "@core" or only in "@appstream"), but they number in the dozens.  We'd have to add a Recommends: or something to all of them, even though their own code base doesn't even mention the debuginfod-client library directly.

Suggests: is too weak (ignored by dnf by default).

Thanks for the analysis, Frank, that helps.

The problem is not so much the actual size, it's the general architecture of the dependencies in the distribution and of its groups and their purposes, plus a risk that some bigger dependency set will get pulled in by accident. So I look at it much more from the point of view of various functional goals than pure space saving. And of course, one of the goals can be minimized, hardened setup.

You mention that systemd is a debuginfo consumer tool that benefits from elfutils-debuginfod-client. Shouldn't systemd Require it then?

Alternatively, listing the package in @core group would work as well, if ease of debugging is a goal.

Josh, would you have a guidance here?

Ease of debugging is not a goal of a minimal installation.

Will we have a debuginfod server running for RHEL content by default with RHEL 9?  If not, I'm not sure why any of the other tools mentioned could have functional dependencies on a client.  If there's no server offering debuginfod content, what exactly are they depending on?

Sorry, I missed this earlier. elfutils-libs package Recommends elfutils-debuginfod-client because in normal operation having elfutils-debuginfod-client installed provides functionality that users would expect from elfutils-libs (easy finding of debuginfo). We used Recommends because is isn't essential and so doesn't have to be installed in a minimal configuaration. Suggests didn't work because then elfutils-debuginfod-client isn't normally installed.

It is not clear what we should do in this case. The cases you list all seem not applicable to this situation:

> If that package is needed by elfutils-libs for correct operation, Requires should be used.

It isn't needed for correct operation, but the functionality is reduced if it isn't installed. So Requires seems overkill.

> If elfutils-debuginfod-client is essential in minimal host installations, it should be listed in the @core group in the comps file, not pulled in as a weak side-effect of having elfutils-libs in @core dependencies.

It isn't essential in a minimal host installation, so it shouldn't be needed to be listed in the @core group in the comps file. It isn't clear to me why it would be pulled in as a weak side-effect.

> If it is listed primarily for convenience, Suggests might be better option. Or just drop the weak dependency completely.

It isn't primarily for convenience, without it some core functionality is missing, which is why Recommends is used, it should normally be installed. Suggests is too weak because then it will never be installed normally.

Please see comment #10. Can we close this as a non-issue?

(In reply to Mark Wielaard from comment #10)
> 
> > If elfutils-debuginfod-client is essential in minimal host installations, it should be listed in the @core group in the comps file, not pulled in as a weak side-effect of having elfutils-libs in @core dependencies.
> 
> It isn't essential in a minimal host installation, so it shouldn't be needed
> to be listed in the @core group in the comps file. It isn't clear to me why
> it would be pulled in as a weak side-effect.

I'm not sure I follow here -- do you mean a weak side-effect of Requires or of something else?

Looking at the existing groups at

   https://gitlab.com/redhat/centos-stream/release-engineering/comps/-/blob/main/comps-centos-stream-9.xml.in

are there perhaps other groups beyond @core that might be more fitting to cover the typical debugging use-case? It seems @debugging or @development might be more fitting.

> > If it is listed primarily for convenience, Suggests might be better option. Or just drop the weak dependency completely.
> 
> It isn't primarily for convenience, without it some core functionality is
> missing, which is why Recommends is used, it should normally be installed.
> Suggests is too weak because then it will never be installed normally.

The problem with using Recommends to allow uninstallation of the package does not work with current dnf because the package will get installed back in during its requiree's upgrade. So it does not really allow admins to create long-term setup that it promises to create.

Hi Jan,

(In reply to Jan Pazdziora from comment #12)
> (In reply to Mark Wielaard from comment #10)
> > 
> > > If elfutils-debuginfod-client is essential in minimal host installations, it should be listed in the @core group in the comps file, not pulled in as a weak side-effect of having elfutils-libs in @core dependencies.
> > 
> > It isn't essential in a minimal host installation, so it shouldn't be needed
> > to be listed in the @core group in the comps file. It isn't clear to me why
> > it would be pulled in as a weak side-effect.
> 
> I'm not sure I follow here -- do you mean a weak side-effect of Requires or
> of something else?

I think we are indeed talking past each other somehow.
So you are concerned with the @core group. And I'll admit I don't fully understand what that is, or why we care about that.
I am talking about (weak) dependencies and why we are using a Recommends on elfutils-debuginfod-client for elfutils-libs.
We use that to make sure that elfutils-debuginfod-client is pulled in whenever elfutils-libs is pulled in, unless it is a minimal install.
I assume the @core group is such a minimal install, so I am not clear on why it would pull in the Recommends, it is not supposed to if it is concerned with just a minimal install.

> Looking at the existing groups at
> 
>   
> https://gitlab.com/redhat/centos-stream/release-engineering/comps/-/blob/
> main/comps-centos-stream-9.xml.in
> 
> are there perhaps other groups beyond @core that might be more fitting to
> cover the typical debugging use-case? It seems @debugging or @development
> might be more fitting.

I am not completely sure what these groups are, but sure those groups sound fine for adding elfutils-debuginfod-client.
As long as the Recommends is honered when someone installs elfutils-libs in a non-minimal environment.

> > > If it is listed primarily for convenience, Suggests might be better option. Or just drop the weak dependency completely.
> > 
> > It isn't primarily for convenience, without it some core functionality is
> > missing, which is why Recommends is used, it should normally be installed.
> > Suggests is too weak because then it will never be installed normally.
> 
> The problem with using Recommends to allow uninstallation of the package
> does not work with current dnf because the package will get installed back
> in during its requiree's upgrade. So it does not really allow admins to
> create long-term setup that it promises to create.

What exactly is the admin scenario you are thinking about? Is this when an admin creates a none-minimal install, then explicitly removes some packages to create a more minimal install? But since the packages might still be Recommended from installed packages, they get sucked back in when they upgrade? That seems a design flaw/bug in dnf.

(In reply to Mark Wielaard from comment #13)
> 
> I think we are indeed talking past each other somehow.
> So you are concerned with the @core group.

Yes, and on a more general note, I'm concerned about predictability of the installations and their upgrades.

> And I'll admit I don't fully
> understand what that is, or why we care about that.

The @core group is a set of packages that are always installed on a system unless the %packages directive in a kickstart is used with  --nocore option.

> We use that to make sure that elfutils-debuginfod-client is pulled in
> whenever elfutils-libs is pulled in, unless it is a minimal install.
> I assume the @core group is such a minimal install, so I am not clear on why
> it would pull in the Recommends, it is not supposed to if it is concerned
> with just a minimal install.

And that's exactly the point. Even if you do not specify *any* package in the kickstart and don't select any extra group, so you are only installing @core, the elfutils-debuginfod-client will get installed, even in this minimal host installation.

> I am not completely sure what these groups are, but sure those groups sound
> fine for adding elfutils-debuginfod-client.
> As long as the Recommends is honered when someone installs elfutils-libs in
> a non-minimal environment.

Well, I meant add them there and remove the Recommends. Because that Recommends applies in both the non-minimal and minimal installations.

> What exactly is the admin scenario you are thinking about? Is this when an
> admin creates a none-minimal install, then explicitly removes some packages
> to create a more minimal install? But since the packages might still be
> Recommended from installed packages, they get sucked back in when they
> upgrade? That seems a design flaw/bug in dnf.

Yes, either install @core and remove, or use dnf -x or kickstart's %packages' - lines to exclude those weak dependencies in the first place. And yes, I wish dnf behaved differently and requests for changes have been filed but it does not look like we can rely on the change happening for RHEL 9.

Hi Jan,

(In reply to Jan Pazdziora from comment #14)
> (In reply to Mark Wielaard from comment #13)
> > 
> > I think we are indeed talking past each other somehow.
> > So you are concerned with the @core group.
> 
> Yes, and on a more general note, I'm concerned about predictability of the
> installations and their upgrades.
> 
> > And I'll admit I don't fully
> > understand what that is, or why we care about that.
> 
> The @core group is a set of packages that are always installed on a system
> unless the %packages directive in a kickstart is used with  --nocore option.

Thanks for the explanation.
 
> > We use that to make sure that elfutils-debuginfod-client is pulled in
> > whenever elfutils-libs is pulled in, unless it is a minimal install.
> > I assume the @core group is such a minimal install, so I am not clear on why
> > it would pull in the Recommends, it is not supposed to if it is concerned
> > with just a minimal install.
> 
> And that's exactly the point. Even if you do not specify *any* package in
> the kickstart and don't select any extra group, so you are only installing
> @core, the elfutils-debuginfod-client will get installed, even in this
> minimal host installation.

OK, but that seems a bug in the kickstart install, it should not pull in any Recommends from outside the group.
 
> > What exactly is the admin scenario you are thinking about? Is this when an
> > admin creates a none-minimal install, then explicitly removes some packages
> > to create a more minimal install? But since the packages might still be
> > Recommended from installed packages, they get sucked back in when they
> > upgrade? That seems a design flaw/bug in dnf.
> 
> Yes, either install @core and remove, or use dnf -x or kickstart's
> %packages' - lines to exclude those weak dependencies in the first place.
> And yes, I wish dnf behaved differently and requests for changes have been
> filed but it does not look like we can rely on the change happening for RHEL
> 9.

I would suggest filing a bug against kickstart/dnf.
We cannot really drop the Recommends if that means on a normal install elfutils-debuginfod-client isn't installed.

(In reply to Mark Wielaard from comment #16)
>
> OK, but that seems a bug in the kickstart install, it should not pull in any
> Recommends from outside the group.

Interesting point. Did you find it documented as a formal expectation somewhere?

(In reply to Jan Pazdziora from comment #17)
> (In reply to Mark Wielaard from comment #16)
> >
> > OK, but that seems a bug in the kickstart install, it should not pull in any
> > Recommends from outside the group.
> 
> Interesting point. Did you find it documented as a formal expectation
> somewhere?

No, but from what you expect to happen, this seems like a requirement.

Can we close this bug or move it to another component? The Recommends are correct as far as the elfutils package is concerned.

I'd rather see it turned to Requires, or elfutils-debuginfod-client listed in @core group explicitly and the Recommends removed.

But I'll let Josh make the call.

Given that there hasn't been an update in more than a week on this issue I am going to close it as not a bug in elfutils based on the existing discussion. Please feel free to reopen (maybe against a different package) if you disagree.

My sincere apologies for the lack of response.  This got buried in my bugmail during the massive flood from the 8.4 GA and Jan kindly emailed me separately.

This bug is frustrating because it appears we have a set of disconnected efforts conflicting with each other at the macro level.  Let me lay out a summary and see if everyone agrees.

1) elfutils-debuginfod-client is an optional library used to gather debuginfo from a server running somewhere.

2) RHEL does not have a debuginfod server running but may in the future.  Some other debuginfod server offers UBI debuginfo data.

3) elfutils-libs uses Recommends to avoid pulling this into a minimal installation.  (It only does this for 8+, presumably because RHEL 7 doesn't work with weak deps but I also don't think this even exists on RHEL 7 so I don't understand the conditional in the spec file.  That's orthogonal I guess.)

4) The @core comps group is literally the minimal installation comps group.  Anaconda defaults to using this group if none are specified otherwise.  This is intentional.

5) We use @core for our Common Criteria certifications, so the more we install the more we have to certify.

6) We don't install gdb, systemtap, or anything else that strikes me as obviously related to debugging systems with @core.

7) yum/dnf defaults to including weak dependencies, which pulls -debuginfo-client into a minimal installation because elfutils-libs is critical to systemd (as a direct dependency).

Is that an accurate summary to everyone?

Is there some way to use elfutils-debuginfod-client directly in a manner we'd recommend customers use, or would we prefer them to use gdb/systemtap, etc instead?  I understand at the technical level that putting the Recommends in elfutils-libs is a convenient catchall for the higher level tools, but at the product level we have inconsistency that could be resolved by putting the Recommends in those tools.

Josh, that factual summary looks correct enough.  For 2) note that customers can run debuginfod servers for themselves, and we expect to teach them specifically how to do so in conjunction with Satellite.  So this won't be an unheard-of sort of configuration.

As an alternative to asking Every component that might indirectly link to to this library, what do you think about enumerating reverse dependencies in this elfutils-debuginfod-client package, namely:

    Supplements: gdb binutils systemtap perf [etc etc etc etc] 

?

(In reply to Frank Ch. Eigler from comment #23)
> Josh, that factual summary looks correct enough.  For 2) note that customers
> can run debuginfod servers for themselves, and we expect to teach them
> specifically how to do so in conjunction with Satellite.  So this won't be
> an unheard-of sort of configuration.
> 
> As an alternative to asking Every component that might indirectly link to to
> this library, what do you think about enumerating reverse dependencies in
> this elfutils-debuginfod-client package, namely:
> 
>     Supplements: gdb binutils systemtap perf [etc etc etc etc] 
> 
> ?

That actually might be the ideal solution if dnf does indeed honor Supplements statements and installs elfutils-debuginfod-client when those packages are installed.  Can someone try that out?

I can find one example of Supplements in fedora/rhel: glibc-langpack-FOO subrpms.  I think we should just try it with the real c9s build system as an update under this BZ.  Not sure how else to test it, as one'd need a createrepo database built the same way rhel9's is.

Tentative c9s merge request:
https://gitlab.com/redhat/centos-stream/rpms/elfutils/-/merge_requests/3

The Supplements approach seems much more targeted to the intended use-case.

I wonder still, though, if for those Satellite / debuginfod servers setups, having documentation say "Install elfutils-debuginfod-client package" might actually be the most explicit and most teaching approach. Note that if customers disable weak dependencies in their dnf configurations completely, they will not get the package anyway. We might try to enhance the user experience but the variety of resulting installations might have an opposite effect, leaving users wondering why things don't work as expected.

Sometimes less "automation" logic and more explicit documentation can be a good thing.

(In reply to Frank Ch. Eigler from comment #26)
> Tentative c9s merge request:
> https://gitlab.com/redhat/centos-stream/rpms/elfutils/-/merge_requests/3

That seems plausible. How did you generate the Supplements package list?
Could it be automated? How do we make sure it covers CRB and EPEL (or do we simply state those packages should use Recommends themselves?)

I don't like how it makes the spec different from what we do in Fedora, and I have to think how this interacts with DTS/GTS.
But It seems the most clean solution to solve the different requirements people have.
Shall I just merge and build this into c9s to see whether it works as intended?

(In reply to Mark Wielaard from comment #28)
> (In reply to Frank Ch. Eigler from comment #26)
> > Tentative c9s merge request:
> > https://gitlab.com/redhat/centos-stream/rpms/elfutils/-/merge_requests/3
> 
> That seems plausible. How did you generate the Supplements package list?

Just guessing - I'm not privy to the whole transitive dependency graph. :-)

> Could it be automated? How do we make sure it covers CRB and EPEL (or do we
> simply state those packages should use Recommends themselves?)

Yeah.  Chances are some basic developer tools would already be installed on such a user's workstation, so these bits would be dragged in that way already.

> I don't like how it makes the spec different from what we do in Fedora

We could do this in Fedora too actually.

> and I have to think how this interacts with DTS/GTS.

DTS/GTS tools should in theory link with DTS/GTS copies of elfutils and/or make SOLIB links to libdebuginfod.so so this may not affect them at all.

> But It seems the most clean solution to solve the different requirements
> people have.  Shall I just merge and build this into c9s to see whether
> it works as intended?

Yeah, how about let's give it a try.

Not sure this is fixed.  I attempted to kickstart install a VM using following KS:

$ cat core.ks  | grep -v ^#
keyboard 'us'
lang en_US
firewall --disabled
selinux --disabled
reboot
rootpw redhat
user --name=mcermak --homedir=/home/mcermak --password=redhat
timezone Europe/Prague
bootloader --location=mbr
zerombr
timezone --utc Europe/Prague
url --url=http://download.eng.rdu2.redhat.com/rhel-9/nightly/RHEL-9-Beta/latest-RHEL-9.0.0/compose/BaseOS/x86_64/os/
%packages
@core
%end
$

Where latest now resolves to RHEL-9.0.0-20210615.4.  I ended up with the debuginfod-client-0.185-3 installed.  Not sure this is right.


Jan, wdyt?

# rpm -q elfutils-libs
elfutils-libs-0.185-3.el9.x86_64
# rpm -q --recommends elfutils-libs
elfutils-debuginfod-client(x86-64) = 0.185-3.el9

The patch https://gitlab.com/redhat/centos-stream/rpms/elfutils/-/merge_requests/3 seems to have left the

%if 0%{?rhel} >= 8 || 0%{?fedora} >= 20
Recommends: elfutils-debuginfod-client%{depsuffix} = %{version}-%{release}
%else
Requires: elfutils-debuginfod-client%{depsuffix} = %{version}-%{release}
%endif

part in %package libs unchanged.

Sorry for the oversight, new build just arrived, with clean Recommends: in -libs.

Allrightie, looks better now with -4.  Verified.  Thanks!