I set up a package cache using the Polipo proxy (and Apache frontend to remap urls nicely). It seemed to work, but later dnf failed to update. I believe I can show there is a problem in dnf (or the mirror). Version: dnf-1.1.5-1.fc23.noarch The problem was the cache returned an old repodata.xml, pointing to files which no longer existed. However Polipo is supposed to follow the HTTP specification, and I will show you where dnf (and yum) fall foul of the spec. I wouldn't bother reproducing the exact failure I had. It's sensitive to timing and maybe other things. But it would be so simple to prevent, using a cache-buster when fetching repodata.xml. # dnf install dconf-editor Error: Failed to synchronize cache for repo 'updates' from 'http://fedora-mirror.carrier.duckdns.org/pub/fedora/linux/updates/23/x86_64/': Yum repo downloading error: Downloading error(s): repodata/865e8db25f987620cf1885c87e34aba9a85c80832cbaa38111101046339ce939-prestodelta.xml.xz - Cannot download, all mirrors were already tried without success # curl --head http://fedora-mirror/pub/fedora/linux/updates/23/x86_64/repodata/repomd.xml HTTP/1.1 200 OK Date: Tue, 12 Jan 2016 08:46:14 GMT Server: Apache/2.4.7 (Ubuntu) Content-Length: 4847 ETag: "12ef-5290cd296d540" Last-Modified: Mon, 11 Jan 2016 10:58:53 GMT Accept-Ranges: bytes Content-Type: application/xml Age: 7845 I verified the out-of-date repodata.xml was pointing to 865e8db2*.prestodelta.xml.xz. For reasons I cannot find, requesting directly through Polipo busted the cache. Here is the up-to-date response: # http_proxy=http://brick:8123 curl --head http://mirrorservice.org/sites/dl.fedoraproject.org/pub/fedora/linux/updates/23/x86_64/repodata/repomd.xml HTTP/1.1 200 OK Content-Length: 4847 ETag: "12ef-5291df6fb2d80" Date: Tue, 12 Jan 2016 10:57:23 GMT Last-Modified: Tue, 12 Jan 2016 07:25:58 GMT Server: Apache/2.4.7 (Ubuntu) Accept-Ranges: bytes Content-Type: application/xml Connection: keep-alive The problem is that without an explicit `Expires` header, HTTP caches get to use heuristics, to decide whether to revalidate or not.[1] The solution for HTTP/1.1 is simple. dnf should set `Cache-Control: max-age=0` when requesting repodata.xml. This will force caches to revalidate repodata.xml on the "origin server". The other alternative[2] is to fix the mirrors - they can send exactly the same header on responses to achieve the same effect. It'd be very nice of them to respect HTTP and the REST architecture like this, but realistically dnf should be more tolerant of random mirrors. --- [1] HTTP references: * Point 3 under https://www.mnot.net/cache_docs/#WORK * Latest spec. documents "Calculating Heuristic Freshness" at http://tools.ietf.org/html/rfc7234#section-4.2.2 [2] A third alternative would be to fix the mirroring protocol. Keep previous versions of prestodelta.xml.xz etc. around for a time. This would satisfy the usual HTTP heuristic, and allow the cache to generate a constant load on origin servers for *any* number of clients behind the cache. Clients don't have to rely on the cache's heuristic either; they can set `Cache-Control: max-age=` to allow for the minimum expected interval between updates. (I.e. expecting old files to be removed daily, set max-age to 12 hours). But for that case you probably want a full mirror anyway, so it's probably over-engineered.
Change component to librepo. The function responsible is lr_yum_download_repomd(). (In turn, it calls lr_downloadtarget_new(), which has no options for caching / HTTP header override). Additional note: Yum didn't use Cache-Control either, so AFAICT this bug isn't a regression.
Patch: https://github.com/rpm-software-management/librepo/pull/77
Fixed in upstream in commits: 473b92f964f33734555ca9f7f5093cd2ab17d98e 6f669c0afead047068ec03af8783483a6aa77eea 0141392e9f8f7ad9135b62d92d75c4cb65d57191 a47370b370ba5ae95fb6220d09529a9a42b90ec5
This message is a reminder that Fedora 23 is nearing its end of life. Approximately 4 (four) weeks from now Fedora will stop maintaining and issuing updates for Fedora 23. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '23'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 23 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.