Bug 183977

Summary: Start using delta RPMS and ISOs for updates
Product: [Retired] Fedora Infrastructure Reporter: Richard Lloyd <rkl>
Component: update systemAssignee: Luke Macken <lmacken>
Status: CLOSED UPSTREAM QA Contact: Bill Nottingham <notting>
Severity: medium Docs Contact:
Priority: medium    
Version: unspecifiedCC: alexandr.kara, bugs+fedora, imtiaz.rahi, jschrode, mhild, nobody+pnasrat, notting, pfrields, rvokal
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
URL: ftp://ftp.suse.com/pub/projects/deltarpm/
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2008-01-30 20:07:31 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Richard Lloyd 2006-03-04 13:16:58 UTC
Description of problem:
Downloading large ISOs or a large number of RPMs to apply updates between
releases of the entire distro or individual packages can take a very long time
(with Rawhide updates in particular being the worst offender) and can be
discouraging for people who have bandwidth limits with ISPs (which is become a
lot more common as broadband speeds increase). SuSE Linux has, for a long time,
been using its "deltarpm" package to quite nicely solve this problem - the
binary differences are downloaded and applied to existing RPMs or ISOs and the
whole updating process is significantly quicker than Red Hat/Fedora's.

Version-Release number of selected component (if applicable):
deltarpm, written by SuSE, is currently at version 3.3.

How reproducible:
Every time.

Steps to Reproduce:
1. Just track Rawhide in your yum configuration and then don't do any updates
for a week. Or...
2, Download and install, say, the Fedora Core 4 ISOs and then run "yum update"
(i.e. many months after the distro media release).

  
Actual results:
In both cases, updates can run in into the hundreds of MBs and take many hours
if you don't have a fast very broadband connection.

Expected results:
You should be able to download the binary differences and reduce the
bandwidth/time needed to update software.

Additional info:
SuSE not only use deltarpms as part of its YaST Online Update, but it also
supplies delta ISOs (if they save enough space - sometimes they skip them if
there's been a big package reorg) to allow you to go between, say, Beta N and
Beta N+1 versions of their distro. Example - I'm downloading the delta ISOs for
Beta 5 -> Beta 6 of SuSE 10.1 - it's a 167MB download, which I can achieve in a
few hours. If I want to get Beta 6 from scratch (which I had to do for Fedora
Core 5 Test 3 recently for example), we're talking about a 3GB+ download and
probably a few days.

Personally, I'd start with just delta RPMs for Rawhide and only keep a single
set of deltas (i.e. between the previous version and the current version of each
package) - you may also want to not provide deltas if they're too big (e.g. more
than 50% the size of the full latest RPM). Obviously, yum would need modifying
to able to look for delta RPMs and fallback to the full RPM if one wasn't
available. After that, I'd expand it to the normal update repositories, but this
time keeping up to 2 deltas - one like Rawhide (between previous and current
version) and one to go from the base install media version and the current
version - this would cover 99% of delta updates people would require. Finally,
I'd expand the concept to match the delta ISOs that SuSE provide, but keep this
strictly as deltas between test release ISOs only (6-8 months elapse between
full releases, which would mean the delta would probably be too big).

Further URLs for reading:
http://en.opensuse.org/Download_Help - explains how SuSE users can apply delta ISOs.
http://dag.wieers.com/packages/deltarpm/ - yes, good old Dag provides Red
Hat/Fedora RPMs for deltarpm, but it's all a bit sad that RH/FC don't use them!

Comment 1 Alexandr Kara 2006-08-03 09:39:20 UTC
I strongly vote for that feature as well.
It is really a big difference between SuSE and Fedora. Even if there are
different policies to packages update (SuSE doesn't update a package just to
provide a new version), the times it takes for update to do its jobs are very
different.
Downloading over 120MB of data (including about 40MB for OpenOffice updates)
after two weeks that the Fedora system has not been connected is quite an lot.
And that is being done automatically, I haven't run yum or told it to run.
On the contrary, with SuSE you can often see updates to 10MB-like files that
only have about 50kB or so.
For example a language-pack update for OpenOffice has only about 2.5MB of files
changed (and most of the content is unchanged) of the total 23MB (uncompressed).
I suspect that sometimes the whole package contents can be the same (except
version number) - for example when upgrading a program to the newest version,
all packages, including -artwork, are updated.
And this stresses update server mirrors as well as users.

Comment 2 Imtiaz Rahi 2006-12-12 20:37:12 UTC
I also vote for this feature as well.
I am in Bangladesh and fast internet connection is rare.
We can reduce the download nightmare for updates by using delta rpms.
SuSE has been using this nice feature for quite a long time.
And I was just wondering why RedHat/Fedora did not picked this up.
Please please consider this one and I mean strongly.

Comment 3 Luke Macken 2007-04-18 23:02:44 UTC
Much work has been done recently with presto[0], and there are currently
unofficial deltas being produced for FC6 {updates,extras} and rawhide.  I opened
a ticket[1] for bodhi (the new updates system) to get presto integrated so we
can start pushing out official deltas for updates.

If anyone is interested in helping integrate presto with bodhi, please let me
know.  If not, I'll get to it eventually :)

[0]: https://hosted.fedoraproject.org/projects/presto
[1]: https://hosted.fedoraproject.org/projects/bodhi/ticket/11

Comment 4 Luke Macken 2008-01-30 20:07:31 UTC
Work is being done in bodhi to help facilitate generating deltarpms using presto
for Fedora updates.  Development can be tracked in the following bodhi ticket:

    https://hosted.fedoraproject.org/bodhi/ticket/160