Bug 1046244

Summary: dnf doesn't keep the downloaded packages in the cache
Product: [Fedora] Fedora Reporter: Ahmad Samir <ahmadsamir3891>
Component: dnfAssignee: Packaging Maintenance Team <packaging-team-maint>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: low Docs Contact:
Priority: low    
Version: 20CC: akozumpl, cra, cristian.ciupitu, frankly3d, giallu, h.reindl, jzeleny, kevin, m.a.young, michal, packaging-team-maint, pmatilai, pnemade, rholy, stefan.home, tmraz, travneff, wilf
Target Milestone: ---Keywords: Patch
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: dnf-0.4.13-2.fc20 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-02-09 03:54:29 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Ahmad Samir 2013-12-24 08:11:17 UTC
Reading http://akozumpl.github.io/dnf/cli_vs_yum.html#id22 , I see that dnf dropped the option to keep the downloaded packages in the cache. This is a bit problematic for users with slow internet connections (like me), for example it's much faster to use the already downloaded packages in the cache to update other systems, or to update the packages in local chroots or VMs, rather than having to download those packages multiple times.

The rationale for dropping it was that keepcache has been disabled in Fedora for a while. It's been only disabled but not removed altogether, users who need it could still enable it.

Comment 1 Gianluca Sforna 2013-12-24 20:24:55 UTC
I canceled a transaction after downloading updated RPMs and was surprised to wait again for the download to take place. Is this really intended?

Comment 2 Ales Kozumplik 2014-01-02 07:47:43 UTC
Ahmad, we only dropped it because we believed there was no real users. Your input on this obviously changes that decision and we'll see about adding the option again.

Comment 3 Harald Reindl 2014-01-03 05:53:06 UTC
thanks - there are *a lot* more usecases like the script below and this is true for nearly all yum-options

so please consider it the wrong way start from scratch with remove previous functionality you consider useless instead think about how the overall software-design should look like with knowing them all instead let them grow organic what happened in the past - this will end sooner or later right where you stared: with a not well designed codebase and functions implemented as a hack (besides the bad user expierience)
__________________________________

that is what people are doing with the yum-cache to feed 10, 20, 30, 40, 100 machines with only one time download packages and the way yum-presto works currently you get even the full packages this way in the cache originally downloaded only as delta

that is what you need in production environments because you can't risk that after the testmachines are done the production servers get newer packages as you had tested - do not look only at the ordinary user with no interest in how things are working while you design software!

#!/usr/bin/bash
basearch=`uname -i`
releasever=`rpm -q --qf "%{version}\n" fedora-release`
for g in `ls -1b /var/cache/yum`
do
 if [ -d /var/cache/yum/$g/packages ]
 then
  echo "/var/cache/yum/$g/packages/ > /repo/cache/fc$releasever/"
  sudo mv --verbose /var/cache/yum/$g/packages/*.rpm /repo/cache/fc$releasever/ 2> /dev/null
 fi
done
/buildserver/repo-create.sh

Comment 4 Panu Matilainen 2014-01-03 07:05:56 UTC
Just FWIW, I too use the keep_cache option quite a bit, for example to:
1) pre-download updates to be (quickly) applied at a (more) convenient time
2) create local install benchmark/test-cases of various package-sets

While its not necessarily something you'd want to enable by default on a distro, it has numerous other uses.

Comment 5 Frank Murphy 2014-01-03 08:57:20 UTC
I keep a nfs shared cache, which all boxes update from with -c option.
Only one box actually downloads for all

Comment 6 Charles R. Anderson 2014-01-04 16:20:44 UTC
Another usecase is having the ability to downgrade back to a previous version that is still kept in the cache in case something goes wrong with the new version.

Comment 7 Kevin Kofler 2014-01-05 00:25:36 UTC
Right, for 2 reasons:
1. Fedora mirrors don't keep old packages around (as I explained in https://lists.fedoraproject.org/pipermail/devel/2014-January/193374.html ).
2. One cannot get packages from the mirrors if networking is broken (see https://lists.fedoraproject.org/pipermail/devel/2014-January/193379.html for the detailed use case).

Comment 8 Cristian Ciupitu 2014-01-06 10:42:23 UTC
I've been using keepcache since at least 2009. It's for keeping packages
around in case I remove and reinstall them and to share them between
virtual machines.

Comment 9 Kevin Kofler 2014-01-12 18:06:49 UTC
Ping?

Comment 10 Kevin Kofler 2014-01-22 20:47:07 UTC
What's the status of this? I hope we provided enough evidence that the use case(s) for keepcache exist(s)!

Comment 11 Ales Kozumplik 2014-01-23 08:59:37 UTC
The status of this bug is: "NEW".

Comment 12 Kevin Kofler 2014-01-23 16:26:51 UTC
That much, I can see myself. But could you please:
1. remove the claim on http://akozumpl.github.io/dnf/cli_vs_yum.html#keepcache-configuration-option-dropped that there are "no real use cases" for that feature, and
2. consider prioritizing it somehow? I haven't looked at the relevant code yet, but, judging from my experience, simply NOT doing something (deleting the cache) if a boolean flag is set should be relatively easy to implement.

Comment 13 Jan Zeleny 2014-01-24 07:38:23 UTC
Kevin, in that case I encourage to send a patch for it. The DNF team is swamped lately so any help like that will be much appreciated.

Comment 14 Ahmad Samir 2014-01-24 09:06:02 UTC
(In reply to Jan Zeleny from comment #13)
> Kevin, in that case I encourage to send a patch for it. The DNF team is
> swamped lately so any help like that will be much appreciated.

The code is already there: https://github.com/akozumpl/dnf/commit/c51d06f97a2e76ff3f3a00056cff4dcee41ab103

The issue here is the developer(s) need to make a decision to restore the feature/option - or not restore it, it's as simple as that.

Given one can comment out the self.clean_used_packages() line in  dnf/base.py it just would be tedious doing that after every update of dnf itself.

Comment 15 Ales Kozumplik 2014-01-24 09:13:14 UTC
I confirm we are going to provide keepcache again, off by default. If you can supply the proper pull request for this we'll be glad. Make sure to go through 

https://github.com/akozumpl/dnf/wiki/Hacking

Thanks!

Comment 16 Kevin Kofler 2014-01-25 00:47:46 UTC
Now that was quite a silly exercise given that I only had to revert your commit and fix a trivial conflict in doc/cli_vs_yum.rst, but there you go:
https://github.com/akozumpl/dnf/pull/76

Comment 17 Ahmad Samir 2014-02-01 06:45:25 UTC
Change done upstream https://github.com/akozumpl/dnf/commit/c368aef05fc245410b6cfa16bf5de0acddf12407 .

Thanks.

Comment 18 Fedora Update System 2014-02-07 06:43:57 UTC
dnf-0.4.13-2.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/dnf-0.4.13-2.fc20

Comment 19 Fedora Update System 2014-02-08 05:07:46 UTC
Package dnf-0.4.13-2.fc20:
* should fix your issue,
* was pushed to the Fedora 20 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing dnf-0.4.13-2.fc20'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2014-2172/dnf-0.4.13-2.fc20
then log in and leave karma (feedback).

Comment 20 Fedora Update System 2014-02-09 03:54:29 UTC
dnf-0.4.13-2.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 21 Michal Jaegermann 2014-04-02 17:52:07 UTC
Ahem!  I just tried dnf-0.4.19-1.fc21.noarch.  'man dnf' says:

       dnf clean packages
              Removes any cached packages from the system.  Note that packages
              are not automatically deleted after they are downloaded.

In reality this "ARE NOT" is clearly incorrect as downloaded packages are gone after dnf exited.  OTOH both 'man dnf.conf' and http://akozumpl.github.io/dnf/api_conf.html are silent about 'keepcache'.  What gives?  Should this be reopened or anoter report about dnf documeantion filed?

Comment 22 Kevin Kofler 2014-04-02 22:07:26 UTC
The documentation clearly needs fixing indeed. I'd say open a separate bug.

Comment 23 Kevin Kofler 2014-04-02 22:08:09 UTC
(The documentation was already wrong before the option was introduced. Now at least you have the option to make it correct. ;-) But the documentation should reference the option.)

Comment 24 Michal Jaegermann 2014-04-02 22:36:38 UTC
(In reply to Kevin Kofler from comment #22)
> The documentation clearly needs fixing indeed. I'd say open a separate bug.

All right.  Bug 1083767.