Bug 1689645 - Out of date manual for dnf download
Summary: Out of date manual for dnf download
Keywords:
Status: CLOSED NEXTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: dnf-plugins-core
Version: 29
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Pavla Kratochvilova
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-03-17 08:48 UTC by Dridi Boukelmoune
Modified: 2019-11-12 11:06 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-11-12 11:06:37 UTC


Attachments (Terms of Use)

Description Dridi Boukelmoune 2019-03-17 08:48:32 UTC
I recently needed to download a set of el7 RPM from a private repository on my Fedora 29 system.

I found that `dnf download` could take a --repofrompath option, but it's not documented in the dnf.plugin.download(8) manual.

In particular, I was trying to use it as such:

    dnf download --repofrompath reponame,/path/to/name.repo pkg

It took me a while to figure that the "path" could actually be either the path to a local repository on the filesystem, or a URL to a remote repository.

In my case it ended up as:

    dnf download --repofrompath reponame,https://... pkg

I guess it would also be nice if it could take a .repo file, and use the name part to select one of the repositories it may contain. Because specifying the baseurl works fine but then I lose the opportunity to get the proper PGP checks, or to configure the ad-hoc repo not to "sslverify" when it is self-signed etc.

Current versions:

dnf-4.1.0-1.fc29.noarch
dnf-plugins-core-4.0.4-1.fc29.noarch

Comment 2 Michal Domonkos 2019-03-18 13:11:54 UTC
(In reply to Dridi Boukelmoune from comment #0)
> I guess it would also be nice if it could take a .repo file, and use the
> name part to select one of the repositories it may contain. Because
> specifying the baseurl works fine but then I lose the opportunity to get the
> proper PGP checks, or to configure the ad-hoc repo not to "sslverify" when
> it is self-signed etc.

There's no need to have a CLI option that takes a .repo file; if you need a more elaborate repo configuration, just drop the .repo file into the /etc/yum.repos.d/ directory.  Alternatively, you can use the reposdir config option to extend the directory list where DNF should look for .repo files.

--repofrompath is only meant for the very basic use case involving a baseurl.

Comment 3 Dridi Boukelmoune 2019-03-18 13:24:32 UTC
The main problem in this bug report is the incomplete manual.

Regarding the bonus suggestion about --repofrompath that could take a .repo file, you missed a point in the assumption that I don't need this: for my use case I was working on a Fedora system and tried to download el7 packages from a private repository. So definitely not a repository I would want to drop in /etc/yum.repos.d/ since this is not something I'm trying to install. It worked fine with just the baseurl, eventually I got it working without help from the documentation seeing how dnf tried to fetch "/path/to/name.repo/repodata/repomd.xml", but then it downloaded the RPMs without checking their signature.

But again, I don't know whether `dnf download` would check the signature in the first place without the --repofrompath option.

$ dnf download --help | grep -A2 repofrompath
[redacted]
  --repofrompath [repo,path]
                        label and path to additional repository, can be
                        specified multiple times.

I guess the help message could also be improved:

> label and path to additional repository, the path
> must point to a local or remote repository by its
> baseurl, can be specified multiple times.

HTH :)

Comment 4 Michal Domonkos 2019-03-18 14:05:08 UTC
(In reply to Dridi Boukelmoune from comment #3)
> The main problem in this bug report is the incomplete manual.

Yes, we are going to improve that (hence the Triaged keyword addition).  I was only replying to the bonus suggestion ;)

> Regarding the bonus suggestion about --repofrompath that could take a .repo
> file, you missed a point in the assumption that I don't need this: for my
> use case I was working on a Fedora system and tried to download el7 packages
> from a private repository. So definitely not a repository I would want to
> drop in /etc/yum.repos.d/ since this is not something I'm trying to install.
> It worked fine with just the baseurl, eventually I got it working without
> help from the documentation seeing how dnf tried to fetch
> "/path/to/name.repo/repodata/repomd.xml", but then it downloaded the RPMs
> without checking their signature.

Well, it's still a repository you want to configure.  Whether you use is to install anything or not makes no difference.  Repo config files are supposed to go into /etc/yum.repos.d/ and that's about it.  You can always make it enabled=0 and only enable it when needed.  If it really is just a throwaway repo, then just remove the file afterwards.  It's a bit more inconvenient than being able to specify the .repo file directly on the CLI, but it's not a big deal.

That said, if you don't want to copy the .repo file around, you can always use --setopt together with --repofrompath, to configure individual repo options such as sslverify.

> But again, I don't know whether `dnf download` would check the signature in
> the first place without the --repofrompath option.

I believe it's not checked, at least that's my impression when looking at the code.

> $ dnf download --help | grep -A2 repofrompath
> [redacted]
>   --repofrompath [repo,path]
>                         label and path to additional repository, can be
>                         specified multiple times.
> 
> I guess the help message could also be improved:
> 
> > label and path to additional repository, the path
> > must point to a local or remote repository by its
> > baseurl, can be specified multiple times.
> 
> HTH :)

Thanks for the suggestions!

Comment 5 Michal Domonkos 2019-03-18 14:08:21 UTC
(In reply to Michal Domonkos from comment #4)
> Well, it's still a repository you want to configure.  Whether you use is to
> install anything or not makes no difference.  Repo config files are supposed
> to go into /etc/yum.repos.d/ and that's about it.  You can always make it
> enabled=0 and only enable it when needed.  If it really is just a throwaway
> repo, then just remove the file afterwards.  It's a bit more inconvenient
> than being able to specify the .repo file directly on the CLI, but it's not
> a big deal.

Thinking about it a bit more, it really seems that "reposdir" fits your use case exactly.  It's basically what you need; being able to pass DNF a .repo file.  The only difference is that you don't pass it one file but a directory where .repo files are to be found.

Comment 6 Michal Domonkos 2019-03-18 14:18:58 UTC
(In reply to Michal Domonkos from comment #4)
> I believe it's not checked, at least that's my impression when looking at
> the code.

Correction: It seems we _do_ the verification; it's handled by librepo:
https://github.com/rpm-software-management/libdnf/blob/8f1fedf8551b72d6bc24018f5980714b3a103aeb/libdnf/repo/Repo.cpp#L967

Comment 7 Dridi Boukelmoune 2019-03-20 09:27:16 UTC
Well, if supporting an actual .repo file in addition to a baseurl could make the verification happen, please reconsider :)

One more thing I noticed, the docs could mention that variables such as $releasever or $basearch are expanded when present.

Comment 8 Michal Domonkos 2019-03-20 12:17:19 UTC
(In reply to Dridi Boukelmoune from comment #7)
> Well, if supporting an actual .repo file in addition to a baseurl could make
> the verification happen, please reconsider :)

It does happen already, even with --repofrompath (see Comment 6).

Comment 9 Dridi Boukelmoune 2019-03-20 12:22:31 UTC
I don't remember being prompted to check and accept the repo or package keys when I used the baseurl.

I think only a .repo file would let dnf know where to find them in the first place.

Comment 10 Michal Domonkos 2019-03-20 12:27:26 UTC
(In reply to Dridi Boukelmoune from comment #9)
> I don't remember being prompted to check and accept the repo or package keys
> when I used the baseurl.

It was probably imported already in your rpmdb.  You can try removing it manually with "rpm -e ..." and then using --repofrompath to see if a prompt appears.  My expectation is that it will (but I haven't tried).

Comment 11 Dridi Boukelmoune 2019-03-20 12:30:03 UTC
I would be surprised to have the keys for an el7 private repository already imported on my Fedora 29 system.

In this case, I merely downloaded packages to inspect them, nothing was installed or even install-able.

Comment 12 Michal Domonkos 2019-03-20 14:58:34 UTC
My apologies, you were right.

I just tried using "dnf download" to download a package from a repo with gpgcheck=1 and no key installed, and it finished successfully, without complaining about the missing key or even asking to import it.

That means, contrary to my original belief, "dnf download" does *not* verify signatures of the packages it downloads.  Note that this is true regardless of whether or not --repofrompath is used to configure the repository in question.

Which also means I was wrong in Comment 6 (sorry about the confusion there).  It turns out the LRO_GPGCHECK setting in librepo only applies to metadata files but not packages.  Package verification is handled by DNF & RPM as part of the transaction, but that code path is not triggered by "dnf download".  The only way to verify the signatures after a "dnf download" is to run "rpm -K" manually on the downloaded RPMs.

That said, if GPG checks specifically are what you're missing in the "dnf download" plugin, then please open another bug to track that as an RFE.  This bug will only be used to deliver the documentation fixes we discussed above.

Thanks again!

Comment 13 Dridi Boukelmoune 2019-03-20 15:15:47 UTC
If specifying a .repo file wouldn't bring more benefit than a baseurl, I can moreover live without it.

Again, the main topic here is the manual lagging behind.

Thanks for the thorough research on your end!

Comment 14 Michal Domonkos 2019-03-20 15:38:55 UTC
(In reply to Dridi Boukelmoune from comment #13)
> If specifying a .repo file wouldn't bring more benefit than a baseurl, I can
> moreover live without it.

Right, it wouldn't help in this regard :)  The --repofrompath option really is just yet another source of repository configuration from which DNF constructs its repository objects at runtime.  If you need a more elaborate setup for such a repo, you can use --setopt=repo.option=value to configure as many options as you need at runtime.  Or, better yet, if you already have a .repo file, just point DNF to its parent directory with "reposdir".  With the latter, I think there is little point in being able to specify a .repo file directly.

> Thanks for the thorough research on your end!

No problem!

Comment 15 Pavla Kratochvilova 2019-09-11 12:50:56 UTC
Hi, I made a patch that adds a note to all plugins that all general DNF options are also available: https://github.com/rpm-software-management/dnf-plugins-core/pull/360
And another that improves the help message for the --repofrompath option: https://github.com/rpm-software-management/dnf/pull/1479
As for the documentation of $releasever and $basearch variables, it's actually there: https://dnf.readthedocs.io/en/latest/conf_ref.html#repo-variables

Comment 16 Dridi Boukelmoune 2019-09-16 07:43:21 UTC
Hi,

Thanks for the update. Regarding variables, I wasn't suggesting that they should be documented here in the manual, but simply that --repofrompath supports those variables. Thank you for improving the manuals.

Dridi

Comment 17 Pavla Kratochvilova 2019-10-09 07:26:29 UTC
The following PR adds a note about the repo variables to the --repofrompath option: https://github.com/rpm-software-management/dnf/pull/1499

Comment 18 Ben Cotton 2019-10-31 18:47:08 UTC
This message is a reminder that Fedora 29 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 29 on 2019-11-26.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '29'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 29 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 19 amatej 2019-11-12 11:06:37 UTC
Given that this is a documentation fix I am closing it as NEXTRELEASE. 
The update is in dnf 4.2.15 and dnf-plugins-core 4.0.11 (>= Feodra 30)


Note You need to log in before you can comment on or make changes to this bug.