Bug 1297762

I set up a package cache using the Polipo proxy (and Apache frontend to remap urls nicely).  It seemed to work, but later dnf failed to update.  I believe I can show there is a problem in dnf (or the mirror).


Version: dnf-1.1.5-1.fc23.noarch


The problem was the cache returned an old repodata.xml, pointing to files which no longer existed.

However Polipo is supposed to follow the HTTP specification, and I will show you where dnf (and yum) fall foul of the spec.

I wouldn't bother reproducing the exact failure I had.  It's sensitive to timing and maybe other things.  But it would be so simple to prevent, using a cache-buster when fetching repodata.xml.  


# dnf install dconf-editor
Error: Failed to synchronize cache for repo 'updates' from 'http://fedora-mirror.carrier.duckdns.org/pub/fedora/linux/updates/23/x86_64/': Yum repo downloading error: Downloading error(s): repodata/865e8db25f987620cf1885c87e34aba9a85c80832cbaa38111101046339ce939-prestodelta.xml.xz - Cannot download, all mirrors were already tried without success

# curl --head http://fedora-mirror/pub/fedora/linux/updates/23/x86_64/repodata/repomd.xml
HTTP/1.1 200 OK
Date: Tue, 12 Jan 2016 08:46:14 GMT
Server: Apache/2.4.7 (Ubuntu)
Content-Length: 4847
ETag: "12ef-5290cd296d540"
Last-Modified: Mon, 11 Jan 2016 10:58:53 GMT
Accept-Ranges: bytes
Content-Type: application/xml
Age: 7845

I verified the out-of-date repodata.xml was pointing to 865e8db2*.prestodelta.xml.xz.

For reasons I cannot find, requesting directly through Polipo busted the cache. Here is the up-to-date response:

# http_proxy=http://brick:8123 curl --head http://mirrorservice.org/sites/dl.fedoraproject.org/pub/fedora/linux/updates/23/x86_64/repodata/repomd.xml
HTTP/1.1 200 OK
Content-Length: 4847
ETag: "12ef-5291df6fb2d80"
Date: Tue, 12 Jan 2016 10:57:23 GMT
Last-Modified: Tue, 12 Jan 2016 07:25:58 GMT
Server: Apache/2.4.7 (Ubuntu)
Accept-Ranges: bytes
Content-Type: application/xml
Connection: keep-alive


The problem is that without an explicit `Expires` header, HTTP caches get to use heuristics, to decide whether to revalidate or not.[1]

The solution for HTTP/1.1 is simple.  dnf should set `Cache-Control: max-age=0` when requesting repodata.xml.  This will force caches to revalidate repodata.xml on the "origin server".

The other alternative[2] is to fix the mirrors - they can send exactly the same header on responses to achieve the same effect.  It'd be very nice of them to respect HTTP and the REST architecture like this, but realistically dnf should be more tolerant of random mirrors.


---

[1] HTTP references:

* Point 3 under https://www.mnot.net/cache_docs/#WORK
* Latest spec. documents "Calculating Heuristic Freshness" at http://tools.ietf.org/html/rfc7234#section-4.2.2

[2] A third alternative would be to fix the mirroring protocol.  Keep previous versions of prestodelta.xml.xz etc. around for a time.  This would satisfy the usual HTTP heuristic, and allow the cache to generate a constant load on origin servers for *any* number of clients behind the cache.  Clients don't have to rely on the cache's heuristic either; they can set `Cache-Control: max-age=` to allow for the minimum expected interval between updates.    (I.e. expecting old files to be removed daily, set max-age to 12 hours).  But for that case you probably want a full mirror anyway, so it's  probably over-engineered.