Bug 704561 - applydeltaiso fails
Summary: applydeltaiso fails
Alias: None
Product: Fedora
Classification: Fedora
Component: deltarpm
Version: 14
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
Assignee: Jonathan Dieter
QA Contact: Fedora Extras Quality Assurance
Keywords: Reopened
Depends On:
TreeView+ depends on / blocked
Reported: 2011-05-13 15:34 UTC by John Reiser
Modified: 2012-08-16 21:27 UTC (History)
1 user (show)

Clone Of:
Last Closed: 2012-08-16 21:27:24 UTC

Attachments (Terms of Use)

Description John Reiser 2011-05-13 15:34:05 UTC
Description of problem: Using applydeltaiso to go from Fedora-15RC1 to Fedora-15RC2 fails.

Version-Release number of selected component (if applicable):

How reproducible: every time

Steps to Reproduce:
1. applydeltaiso Fedora-15RC1-i386-DVD.iso Fedora-15-Final.RC1_RC2-i386-DVD.diso foo  >foo.stdout  2>foo.stderr
2. echo $?; cat foo.stderr
Actual results:
1     ## failure
md5sum mismatch, iso is corrupt

Expected results: success

Additional info:
$ ls -l foo Fed*
-rw-rw-r--. 1  3650618108 May 13 08:05 foo
-rw-rw-r--. 1   204913954 May 12 19:41 Fedora-15-Final.RC1_RC2-i386-DVD.diso
-rw-rw-r--. 1         148 May 13 08:13 Fedora-15-Final.RC1_RC2-i386-MD5SUM
-rw-rw-r--. 1  3666591744 May 11 14:53 Fedora-15RC1-i386-DVD.iso
-rw-rw-r--. 1  3650484224 May 13 07:56 Fedora-15RC2-i386-DVD.iso
$ sha256sum foo Fedora-15RC1-i386-DVD.iso Fedora-15RC2-i386-DVD.iso
30f9deefe361796dc22b75161fe2471cf3b6c73631c633a35d83259c465f6924  foo
6b96fe040bce6855ee82a1996716865d9445496213177cc3de1634f31dff185f  Fedora-15RC1-i386-DVD.iso
daaed9ebf5b803151fe69fefed1c736bafe1a3309bf916c218c142a36ac1878a  Fedora-15RC2-i386-DVD.iso
$ sed 1q  < Fedora-15RC1-i386-CHECKSUM     ## matches RC1
6b96fe040bce6855ee82a1996716865d9445496213177cc3de1634f31dff185f *Fedora-15-i386-DVD.iso
$ sed 1q  < Fedora-15RC2-i386-CHECKSUM     ## matches RC2, but mismatches 'foo'
daaed9ebf5b803151fe69fefed1c736bafe1a3309bf916c218c142a36ac1878a *Fedora-15-i386-DVD.iso
$ sed 1q  < Fedora-15-Final.RC1_RC2-i386-MD5SUM
c4e72fc9bb1d8cd094bdced5f54c0620  Fedora-15-Final.RC1_RC2-i386-DVD.diso
$ md5sum Fedora-15-Final.RC1_RC2-i386-DVD.diso     ## matches
c4e72fc9bb1d8cd094bdced5f54c0620  Fedora-15-Final.RC1_RC2-i386-DVD.diso
$ rpm -qf /usr/bin/applydeltaiso
$ rpm --verify deltaiso; echo $?
$ grep -v unchanged foo.stdout
reading 329057840 bytes from old iso...done
dconf.i686 (xz.2): applying delta
dracut.noarch (xz.2): applying delta
dracut-caps.noarch (xz.2): applying delta
dracut-fips.noarch (xz.2): applying delta
dracut-network.noarch (xz.2): applying delta
dracut-tools.noarch (xz.2): applying delta
fedora-release-notes.noarch (xz.2): applying delta

Comment 1 John Reiser 2011-05-13 15:50:48 UTC
Works with deltaiso-3.6-0.6.20110223git.fc15.x86_64

Comment 2 Jonathan Dieter 2011-05-13 16:36:50 UTC
The problem is that deltarpm in F14 is linked against xz-4.999.9, while in F15 it's linked against xz-5.0.1.  In the switchover from 4.999 to 5.0, xz modified its compression algorithm so that compressed files are (very) slightly different.  If you're interested, see http://cedarandthistle.wordpress.com/2011/02/16/brittle-deltas-a-possible-solution for more detail.

So, basically, we're stuck.  If you apply a F15 deltaiso using deltaiso from F15, it works (as you've already seen).  If you use the F14 one, it won't.  One thought Andre Robatino and I were throwing around was to host a F14 copy of deltaiso that was statically linked against xz-5.0.x so you could use it for F15 deltaisos, but then that version of deltaiso will no longer work with F14 deltaisos.

Anyhow, there's no easy way that I can see to fix this.  Afraid I'm going to have to close this as CANTFIX.  If you can see a workable solution, please reopen.

Comment 3 John Reiser 2011-05-13 17:53:15 UTC
Please comment on these two possibilities which have slightly different goals.

1) Goal: generate intended output (byte-for-byte identical).  Method: add a commandline parameter "--external-xz='<path_and_paramters>'" which lets the user specify a substitute xz and its command-line flags.  Then the expert user could use the other xz installed somewhere else, or could specify flags that would cause xz to choose its settings to be compatible with specific .isos.

2) Goal: generate "usable" output: something that will pass MediaCheck and will install correctly, but may be different from the intended .iso because of xz differences.  Method: detect non-equality as now, but then recover.  Run implantisomd5 on the output so that MediaCheck passes, change the exit code from 1 to 2, change the stderr message to something such as "The output is not byte-for-byte identical with the intended .iso because of differences in xz compression.  However, the output .iso is equivalent: the results of an install will be the same, and the .iso also passes MediaCheck."

Comment 4 John Reiser 2011-05-14 17:41:45 UTC
Choice 2) will require significant work.  "implantisomd5 --force" succeeds and the resulting .iso boots, passes MediaCheck, and begins an install.  However, most of Packages/*.rpm is garbage; begining with:
error: deja-dup-18.1.1-1.fc15.i686.rpm: not an rpm package (or package manifest)
error: dejavu-fonts-common-2.33-1.fc15.noarch.rpm: not an rpm package (or package manifest)
error: dejavu-lgc-sans-fonts-2.33-1.fc15.noarch.rpm: not an rpm package (or package manifest)
error: dejavu-lgc-sans-mono-fonts-2.33-1.fc15.noarch.rpm: not an rpm package (or package manifest)
as diagnosed by "rpm --verify -p Packages/*.rpm";  2471 out of 2944 are bad.

This looks like applydeltaiso has shifted the blocks of an ISO9660 filesystem without adjusting the directory entries.  Applydeltaiso claims "copying unchanged" for almost every *.rpm (all except the 7 mentioned in original Description above) but it seems that the copy did not go to the correct blocks of the .iso per the ISO9660 directory.

Altogether, it seems to me that applydeltaiso should adopt a more-sceptical attitude.  Examine the names of the .rpms, and refuse to proceed when a known bad case is detected (such as this case, with mismatched xz for .fc15 on Fedora 14).  Even more, use evidence-based tests: pick the 25th-largest .rpm, uncompress it, re-compress it, and refuse to proceed unless it matches the original.

Comment 5 Fedora End Of Life 2012-08-16 21:27:27 UTC
This message is a notice that Fedora 14 is now at end of life. Fedora 
has stopped maintaining and issuing updates for Fedora 14. It is 
Fedora's policy to close all bug reports from releases that are no 
longer maintained.  At this time, all open bugs with a Fedora 'version'
of '14' have been closed as WONTFIX.

(Please note: Our normal process is to give advanced warning of this 
occurring, but we forgot to do that. A thousand apologies.)

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, feel free to reopen 
this bug and simply change the 'version' to a later Fedora version.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we were unable to fix it before Fedora 14 reached end of life. If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora, you are encouraged to click on 
"Clone This Bug" (top right of this page) and open it against that 
version of Fedora.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 

Note You need to log in before you can comment on or make changes to this bug.