Bug 1203491

Summary: dnf cannot download a package group (like yum groupinstall --downloadonly)
Product: [Fedora] Fedora Reporter: Nandan Bhat <nandanlbhat>
Component: dnfAssignee: rpm-software-management
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: low    
Version: 23CC: aliakc, atu, bugzilla, jsilhan, jzeleny, mluscon, mmraka, packaging-team-maint, pgervase, pnemade, redhat, spetreolle, tim.lauridsen, travneff
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-11-25 08:35:10 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1048433, 1185741    
Bug Blocks:    

Description Nandan Bhat 2015-03-19 01:25:40 UTC
Description of problem:
dnf does not have an option to download a package group

Version-Release number of selected component (if applicable):
dnf-0.6.4-1.fc21.noarch
dnf-plugins-core-0.1.5-1.fc21.noarch

How reproducible:
always

Steps to Reproduce:
1. try manual for dnf, dnf.plugin.download
2. there is nothing related to downloading group
3.

Actual results:
For any action related to package download, it would help if there is an option similar to yum --downloadonly --downloaddir=/some/path. That option allows install, groupinstall, update etc. to work only up to download stage and then quit. The same packages may be installed on all machines using rpm -Uvh *.rpm. This feature saves internet bandwidth usage.

Expected results:
some functionality that allows download-only option in dnf

Additional info:

Comment 1 Ali Akcaagac 2015-04-07 23:29:14 UTC
Let me back you up. I require this as well.

https://bugzilla.redhat.com/show_bug.cgi?id=1209638

Comment 2 Radek Holy 2015-04-08 06:39:40 UTC
Hello, thank you for the report. We'll take a look.

BTW, IIUIC, a shared network cache of the packages would help you in this case and even it may simplify your workflow, right?

Comment 3 Ali Akcaagac 2015-04-08 07:29:58 UTC
No, that's not what I was asking for in my bug report above. I exactly expect

--downloadonly and --downloaddir=<somedir>

to exist, since DNF now becomes the "drop in replacement" for yum (marking yum deprecated). I do expect it to offer the same command set doing the same tasks as yum did - and no "we can offer an alternative that does it differently". We depend on these features because the infrastructure around it does exactly expect these commands to exist.

Please understand and do expect, that there are external infrastructure build around yum that is well documented (not at fedora) and works for a few years now. There is nothing against dnf. Please only make sure that a full yum equivalent command set exists on Fedora 22 delivery.

Comment 4 Radek Holy 2015-04-08 07:43:13 UTC
AFAIK, DNF was never intended to be a drop in replacement. Did I miss the document that says the opposite?

And the "BTW" in the sentence meant that I'm just curious whether I understand your use case. There is nothing that indicates that I'm thinking about "offering an alternative that does it differently".

Comment 5 Honza Silhan 2015-04-08 12:25:17 UTC
*** Bug 1209638 has been marked as a duplicate of this bug. ***

Comment 6 Nandan Bhat 2015-07-05 08:40:10 UTC
I'm afraid, dnf is trying to work exactly like a drop-in replacement.

Here is a problem in my setup. I have two identical installs of Fedora 22. Now that I would like to update the systems, I would like to run 

 # dnf update --downloadonly --downloaddir=/some/path

Worse, I try yum, since I remember it used to do something like this. Surprise! The warning says 

"# yum update
Yum command has been deprecated, redirecting to '/usr/bin/dnf update'.
See 'man dnf' and 'man yum2dnf' for more information.
To transfer transaction metadata from yum to DNF, run:
'dnf install python-dnf-plugins-extras-migrate && dnf-2 migrate'"

The package list totals 850 MB of download. I would like to do that download only once, if possible. This is not related to group install, though.

Comment 7 Chris Schanzle 2015-07-09 04:25:55 UTC
I would like to describe another use case: updating 150+ workstations at roughly the same time with all update packages predownloaded.

I schedule with the users an hour window in the evening where I update and perhaps reboot systems.  I keep local copies of nearly all repos on the local LAN.

Even on a local LAN (with a few dozen over a WAN), an update transaction could be hundreds of MB per system, it's not practical to update all these systems simultaneously and expect downloads to succeed, much less within the timeframe required.

I currently accomplish this by sequentially running 'yum -y update --downloadonly' a few hours in advance to pre-download all the packages to the local system.  Then when the update window starts, I essentially just do a 'yum -y update'.  With all the packages pre-downloaded, there is only moderate amounts of network traffic:  repo metadata is often all that is downloaded, and even that can peg the http server > 100MB/s for a few minutes.

While I'm new to dnf, I haven't found a way to download locally the packages pending updates (including dependencies), then update from that cache.  Any suggestions would be appreciated!

Comment 8 Ali Akcaagac 2015-08-24 10:02:16 UTC
I share the described use case explained by Chris Schanzle and like to extend it a bit more. We have only one script left using old YUM because we haven't found a proper solution for doing the same with DNF as of today.

# yum clean all --installroot=/some/path
# yum install --downloadonly --downloaddir=/some/path --installroot=/some/path

1) clean all metadata
2) check all packages for installation by checking for already downloaded packages in downloaddir. If newer packages exists online (than within the downloaddir) then download only the subset into the downloaddir. Using yum gives an output saying "downloading 7mb" and a list of already downloaded (bold marked) and to be downloaded (not bold) list of packages. Then it asks you whether you like to proceed or not (you can add --assumeno --assumeyes). The installroot here is just used to dump the downloaded meta-data which will be erased afterwards.

We've been trying to use

# dnf download <listofpacks> --downloaddir=/some/path

(I hope downloaddir was right, just writing from my mind)

Sadly it instantly starts downloading without giving some information about how much it wants to download and whether we want to proceed or not (--assumeyes --assumeno). Good thing is, it skips downloaded packages and has the --resolve feature. Bad thing is, it still lacks downloading groups.

Comment 9 Fedora End Of Life 2015-11-04 10:18:44 UTC
This message is a reminder that Fedora 21 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 21. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '21'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 21 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 10 Ali Akcaagac 2015-11-04 10:29:46 UTC
Someone please re-assign to Fedora 22, because the issue still remains and hasn't been addressed until now.

Comment 11 Nandan Bhat 2015-11-05 07:39:44 UTC
Actually, looks like dnf-1.1.3-1.fc23.noarch (Fedora 23) has the ability to "download only". However, it will do so only in the /var/cache/dnf directory structure. If you need to copy packages to an alternate directory and create a repository, my guess is you may need the python-dnf-plugins-extras-local.noarch package.

This is quite acceptable to me, but it still cannot be termed feature-complete to replace yum --downloadonly --downloaddir=/some/path .

Comment 12 Ali Akcaagac 2015-11-05 08:43:05 UTC
I haven't looked at extras-local myself because I do not do (or keep) a plugins collection here. Something like this should be part of the main software (package). Although it's right that dnf has a *newly?* downloadonly feature but it still can not deal with groups.

Recently we changed our very own infrastructure (some scripts we've written to automate some installation processes) to use "strace" infront of dnf.

strace -f -qq -e stat64 dnf someoption ...

And grep the output for the rpm files. I recommend using one of the -e options to trigger only the output you really need. That way you can reduce output.

A possible scenario might look like this ?

strace -f -qq -e stat64 -o log.txt dnf groupinstall "xfce-desktop-environment" --downloadonly

After the download completes grep the ouput for stat64 and use some sed magic to get the files.

Disadvantage (and the reason why I don't do this right now):
- I have my downloaddir somewhere else.
- I would like to have already downloaded rpm's (with same versioning) being skipped from being downloaded again
- DNF can't deal with groups in therms of downloading only

Yum addresses this use case far better atm.

Comment 13 Fedora Admin XMLRPC Client 2016-07-08 09:38:36 UTC
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Comment 14 Fedora End Of Life 2016-11-24 11:34:46 UTC
This message is a reminder that Fedora 23 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 23. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '23'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 23 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 15 Michael Mráka 2016-11-25 08:35:10 UTC
dnf group install --downloadonly <group>

works in dnf-2.0.0-0.rc1.1.git.0.e6df665

Comment 16 Ali Akcaagac 2016-11-25 13:43:12 UTC
(In reply to Michael Mráka from comment #15)
> dnf group install --downloadonly <group>
> 
> works in dnf-2.0.0-0.rc1.1.git.0.e6df665

Regarding the first comment (and initial poster) there is also a requirement for

--downloaddir

Right now all packages are still downloaded into /var/.../dnf and

dnf download --downdir=xxxx nvra.rpm

still can't handle groups ...