Hi, it caches metadata. You see the checking if metadata are up-to-date. You can set different metadata expire policy by `metadata_expire` option in each repo config file (etc/yum.repos.d/*.repo) or wait for bug 850896 to be implemented.

metadata_expire is set to 0 which works fine in yum. Higher values cause too large latencies and fetching the small repomd.xml (< 10KiB) does not matter.

Using other protocols for metadata download (as discussed in 850896) is an orthogonal problem which deals with downloading *new* metadata.

We should not download metadata with the same checksums even when they are marked as expired.

IMO all metadata and even repomd.xml should be requested with header If-Modified-Since.
See section 14.25 of:
  http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html
When correctly implemented and files are not modified, then only HTTP headers are transfered and no real data.

Since everybody (including me) are affected, even i386 version, the priority of this bug should be increased
Also, I suspect Fedora 23 and Rawhide would be equally affected (though not verified)

It appears to me that DNF uses metadata_expire entirely differently than yum. Please read the following quote from "man yum.conf":

===
metadata_expire Time (in seconds) after which the metadata  will expire. So that if the current metadata downloaded is less than this many seconds old then yum  will not update the metadata against the repository. If you find that yum is not downloading information on updates as often as you would like lower the value of this option.
(...truncated...)
The default is 6 hours, to compliment yum-updatesd running once an hour.
===

Again: "So that if the current metadata downloaded is less than this many seconds old then yum  will  not  update  the  metadata against the repository."

This IMHO means that yum uses metadata_expire to avoid excessively frequent downloading of metadata. Once fresh metadata are downloaded, yum does not download metadata from the repository, even when newer metadata are possibly available in the repository. Once the metadata_expire time interval expires, yum absolutely does not force the metadata re-downloading for no reason, e.g. if there are no newer metadata available in the repo.

In contrast, dnf IMHO uses metadata_expire in such a way that it really expires the local copy of metadata by declaring them as outdated and unconditionally (and possibly for no practical reason, if the local copy of the metadata still matches the repo metadata) causes the unchanged metadata to be redownload again and againg, with really strange and annoying result like
Fedora 23 - i386                                898 kB/s |  39 MB     00:45
If there is only small number of small .drpms to be downloaded, the ~40MB size of unnecessary re-download of just this Fedore release repo is frequently multiple of the size of .drpms to be downloaded. Provided you are connected through 8Mbit ADSL line, it took 45 seconds in this example. If there is possibly FUP omn your line, you have lost not only the time, but also megabytes from yout FUP limit.

Taken into account how long this issue is reported and discussed again and again, you should definitely to fix this, I hate to say ASAP, but I say really soon.

(Koukejte, chlapi v Brne, neco s tim konecne udelat. Jak to je ted, je to skutecne rozcilujici, otravne, zere to cas, megabajty... A snad nemuze byt zas az tak tezke to nejak spravit.)

Brgds,

Eduard

*** Bug 1293910 has been marked as a duplicate of this bug. ***

This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Fedora 22 changed to end-of-life (EOL) status on 2016-07-19. Fedora 22 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.

still with dnf-1.1.9-2.fc23.noarch

Hello.

In my opinion, this is one of the most annoying bugs during some past years. Please just put your Fedora systems behing some 8, 4 or 2Mbit DSL line and you will see what I am talking about. Sometimes, if there are only few small DRPMs in the update set, the metadata are much bigger than updates to be downloaded. 

Please read my comment #6 in this BZ entry and *fix*this*bug*.

Thanks in advance,

Eduard

This bug appears to have been reported against 'rawhide' during the Fedora 25 development cycle.
Changing version to '25'.

Hello,

I am not able to reproduce described behaviour. Do you still experience the issue? Do you have some non-standard network environment (http proxies, ipv6 env, etc.)?

Still with dnf-1.1.10-4.fc25.noarch

| Fedora 25 - x86_64 - Base                      4.1 MB/s |  26 MB     00:06    

Proxy is usually required, but mirror can be reached directly.

Hello.

It may be just my opinion, but this bug does no longer manifest in my environment, even after "dnf clean expire-cache" which was the way to force it to always manifest on the next "dnf update".

Brgds,

Ed

We prepared a major improvement with dnf-2.3 that is available for rawhide and fc26. We also provide it for Fedora 24+ from our testing repository (dnf copr enable rpmsoftwaremanagement/dnf-nightly). Please can you try the latest version of dnf-2.3 and report if your problem appears again?

Hi Jaroslav.

Once I ugrade to nightly build of dnf-2.3 on F25, will it be downgradeable back to the current F25 dnf version via "dnf  distro-sync" ? E.g., databases backward compatility and similar things?

I would like to help with testing, but I have to try on physical computer and do not wish to wait some two months for F26 with nightly build of dnf installed.

Thanks,

Eduard

Just tried "what will happen if I try to upgrade dnf t version 2...

dnf copr enable rpmsoftwaremanagement/dnf-nightly
dnf update

offers skipping about three fullscreens of dnf-2 packages and the suggested

dnf update -- best --allowerasing

is only slightly better with offer of installing/upgrading/replacing lots of packages, but also

Skipping packages with broken dependencies:
 dnf               src    2.4.0-0.28ge949aaa.fc25
                                        rpmsoftwaremanagement-dnf-nightly 552 k
 dnf-plugins-core  src    2.0.0-0.30gb26c810.fc25
                                        rpmsoftwaremanagement-dnf-nightly 128 k
 dnf-plugins-extras
                   src    2.0.0-0.24gc320b3a.fc25
                                        rpmsoftwaremanagement-dnf-nightly  76 k

Transaction Summary
================================================================================
Install  11 Packages
Upgrade  23 Packages
Remove    9 Packages
Skip      3 Packages

Total download size: 3.0 M

It does not look OK to me to proceed, so for now I did

dnf copr disable  rpmsoftwaremanagement/dnf-nightly

and I am ready to give it another try in the future. These three src packages are only sources not needed for dnf-2.3 operation? Your advice pls?

Please note that the above happened on physical computer with up-to-date 32bit F25 installed.

[vopicka@localhost tmp]$ rpm -qa|grep ^dnf|sort
dnf-automatic-1.1.10-6.fc25.noarch
dnf-conf-1.1.10-6.fc25.noarch
dnfdaemon-selinux-0.3.16-3.fc25.noarch
dnfdaemon-0.3.16-3.fc25.noarch
dnf-plugins-core-0.1.21-5.fc25.noarch
dnf-plugins-extras-0.0.12-4.fc25.noarch
dnf-plugin-system-upgrade-0.7.1-4.fc25.noarch
dnf-yum-1.1.10-6.fc25.noarch
dnf-1.1.10-6.fc25.noarch
[vopicka@localhost tmp]$ 


Brgds,

Eduard

Thanks for your try. Skipped packages src - this is bug in dnf-1.1.x . This packages are uninstallable because they contain source code. With dnf-2.3 you will not see them and dnf will not try to install them.
How to downgrade dnf-2.3 to dnf-1.1:
Just do "dnf downgrade dnf-1.1*". Or may be + --allowerasing.

What could happen with downgrade:
Probably some of plugins could be removed, but then you can install them back (depends on installed plugins). 
What is incompatible in Fedora25 with dnf-2.3:
yumex-dnf
anaconda

Personally I did downgrade dnf-2.X to dnf-1.1 many times for testing purposes, but only I experience removal of plugins (the packages were renamed, therefore obsoletes were used. But obsoletes cannot handle downgrades to restore original situation).
Thanks a lot for your cooperation.

So here it goes.

[root@localhost dnf]# dnf --version
2.4.0
  Installed: dnf-0:2.4.0_1-1g48506ee.fc25.noarch at 2017-05-03 20:15
  Built    :  at 2017-05-03 01:45

  Installed: rpm-0:4.13.0.1-1.fc25.i686 at 2017-03-01 17:58
  Built    : Fedora Project at 2017-02-24 12:49
[root@localhost dnf]# 

And my first observations: Promising, the following command sequence does not cause unneeded metadata redownload:

dnf clean all
dnf update
# Here fresh repo meadata ale downloaded for all repos
dnf update
dnf clean expire-cache
dnf update
dnf update
rm -f /var/cache/dnf/expired_repos.json
dnf update
dnf update

Now I must wait 6 or more hours (my metadata_exire interval in dnf.conf).

My experiences are negative ... Using mock for development, DNF refresh the metadata all the time.

Hmmm... Looks to me like that metadata of "expired" repos are now correctly checked and downloaded only if server has newer version.

@vit Ondruch: Please chech especially if metadata of "fedora" repo (these metadata should be static) are redownloaded for no apparent reason?

[root@czlx-vopicka dnf]# dnf --version
2.4.0
  Installed: dnf-0:2.4.0_1-4gbb5ed9a.fc25.noarch at 2017-05-04 10:04
  Built    :  at 2017-05-04 01:50

  Installed: rpm-0:4.13.0.1-1.fc25.x86_64 at 2017-03-01 08:58
  Built    : Fedora Project at 2017-02-24 12:48
[root@czlx-vopicka dnf]# dnf update
Last metadata expiration check: 0:00:24 ago on Thu May 04 12:22:41 2017 CEST.
Dependencies resolved.
Nothing to do.
Complete!
[root@czlx-vopicka dnf]# dnf clean expire-cache
Cache was expired
0 files removed
[root@czlx-vopicka dnf]# cat expired_repos.json ;echo ""
["rpmfusion-nonfree-updates", "rpmfusion-free-updates", "updates", "rpmsoftwaremanagement-dnf-nightly", "heliocastro-hack-fonts", "rpmfusion-nonfree", "google-chrome", "adobe-linux-x86_64", "fedora", "google64", "rpmfusion-free"]
[root@czlx-vopicka dnf]# dnf update
Last metadata expiration check: 0:00:00 ago on Thu May 04 12:23:30 2017 CEST.
Dependencies resolved.
Nothing to do.
Complete!
[root@czlx-vopicka dnf]#

(In reply to Eduard Vopicka from comment #22)
> @vit Ondruch: Please chech especially if metadata of "fedora" repo (these
> metadata should be static) are redownloaded for no apparent reason?

You are probably right and the issue I am hitting is different then this. I'm going to reopen my bug 1293910 since I believe that I found the reason for the issues I observe.