Bug 1017205

Summary: --instrepo makes fedup crash
Product: [Fedora] Fedora Reporter: Kamil Páral <kparal>
Component: fedupAssignee: Will Woods <wwoods>
Status: CLOSED DUPLICATE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 19CC: kparal, tflink, wwoods
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-10-22 09:47:26 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
fedup crash log
none
fedup OK log none

Description Kamil Páral 2013-10-09 12:40:16 UTC 
Description of problem:
I installed F19 GNOME and run:

$ fedup --network 20 --instrepo http://download.eng.brq.redhat.com/pub/fedora/linux/development/20/x86_64/os/

Fedup crashed while downloading packages. Log excerpt:

[    72.843] (II) fedup:message() Downloading failed: Errors were encountered while downloading packages.
[    72.845] (II) fedup:message()   rtkit-0.11-6.fc20.x86_64: failure: Packages/r/rtkit-0.11-6.fc20.x86_64.rpm from cmdline-instrepo: [Errno -1] Package does not match intended download. Suggestion: run yum --enablerepo=cmdline-instrepo clean metadata
[    72.845] (II) fedup:message()   highlight-3.15-1.fc20.x86_64: failure: Packages/h/highlight-3.15-1.fc20.x86_64.rpm from cmdline-instrepo: [Errno -1] Package does not match intended download. Suggestion: run yum --enablerepo=cmdline-instrepo clean metadata
[    72.845] (II) fedup:message()   pyliblzma-0.5.3-10.fc20.x86_64: failure: Packages/p/pyliblzma-0.5.3-10.fc20.x86_64.rpm from cmdline-instrepo: [Errno -1] Package does not match intended download. Suggestion: run yum --enablerepo=cmdline-instrepo clean metadata
[    72.845] (DD) fedup:<module>() Exception:
Traceback (most recent call last):
  File "/bin/fedup", line 181, in <module>
    main(args)
  File "/bin/fedup", line 126, in main
    pkgs = download_packages(f)
  File "/bin/fedup", line 66, in download_packages
    f.download_packages(updates, callback=output.DownloadCallback())
  File "/usr/lib/python2.7/site-packages/fedup/download.py", line 200, in download_packages
    updates = self._downloadPackages(callback)
  File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 6393, in _downloadPackages
    raise Errors.YumDownloadError, errstr
YumDownloadError: [u'rtkit-0.11-6.fc20.x86_64: failure: Packages/r/rtkit-0.11-6.fc20.x86_64.rpm from cmdline-instrepo: [Errno -1] Package does not match intended download. Suggestion: run yum --enablerepo=cmdline-instrepo clean metadata', u'highlight-3.15-1.fc20.x86_64: failure: Packages/h/highlight-3.15-1.fc20.x86_64.rpm from cmdline-instrepo: [Errno -1] Package does not match intended download. Suggestion: run yum --enablerepo=cmdline-instrepo clean metadata', u'pyliblzma-0.5.3-10.fc20.x86_64: failure: Packages/p/pyliblzma-0.5.3-10.fc20.x86_64.rpm from cmdline-instrepo: [Errno -1] Package does not match intended download. Suggestion: run yum --enablerepo=cmdline-instrepo clean metadata']
[    72.855] (II) fedup:<module>() /bin/fedup exiting at Wed Oct  9 11:53:25 2013


If I don't specify --instrepo, everything works fine.

Version-Release number of selected component (if applicable):
fedup 0.7.3-4.fc19

How reproducible:
seems always (at the moment)

Steps to Reproduce:
1. install F19 GNOME
2. run fedup --network 20 --instrepo http://download.eng.brq.redhat.com/pub/fedora/linux/development/20/x86_64/os/
3. fedup crashes
Comment 1 Kamil Páral 2013-10-09 12:40:40 UTC 
Created attachment 809889 [details]
fedup crash log
Comment 2 Kamil Páral 2013-10-09 12:41:24 UTC 
Created attachment 809890 [details]
fedup OK log

This is a log of fedup run without --instrepo (just fedup --network 20).
Comment 3 Will Woods 2013-10-09 18:06:08 UTC 
...this log looks like a cleanly-handled YumDownloadError, caused by corrupt/incomplete files in your instrepo.

What was the output on-screen? Did it actually emit a traceback, or just an error message?
Comment 4 Kamil Páral 2013-10-10 09:31:59 UTC 
The console output is:

(1210/1212): zenity-3.8.0-3.fc20.x86_64.rpm                                              | 3.4 MB  00:00:00     
(1211/1212): zip-3.0-9.fc20.x86_64.rpm                                                   | 261 kB  00:00:00     
(1212/1212): zlib-1.2.8-3.fc20.x86_64.rpm                                                |  90 kB  00:00:00     
(1/2): highlight-3.15-1.fc20.x86_64.rpm                                                  | 603 kB  00:00:00     
(2/2): pyliblzma-0.5.3-10.fc20.x86_64.rpm                                                |  46 kB  00:00:00     

Downloading failed: Errors were encountered while downloading packages.
  highlight-3.15-1.fc20.x86_64: failure: Packages/h/highlight-3.15-1.fc20.x86_64.rpm from cmdline-instrepo: [Errno -1] Package does not match intended download. Suggestion: run yum --enablerepo=cmdline-instrepo clean metadata
  pyliblzma-0.5.3-10.fc20.x86_64: failure: Packages/p/pyliblzma-0.5.3-10.fc20.x86_64.rpm from cmdline-instrepo: [Errno -1] Package does not match intended download. Suggestion: run yum --enablerepo=cmdline-instrepo clean metadata


The funny thing is that fedora.repo should point to the same mirror as --instrepo url, it's our private mirror. So I disabled fedora-updates.repo to directly compare the two approaches.

With "fedup --network 20 --instrepo http://download.brq..." the problem is still there, yum complains about broken packages.

But if I run "fedup --network 20" (which should use the _same_ mirror, only upgrade.img is downloaded from a different path - F20 Alpha), the process goes smoothly and there is no error.

I can tell that both approaches use the same mirror, because in both cases the packages are downloaded using 1 Gbps speed (our local LAN) instead of using much slower Internet connection.

So, both approaches should work fine (they most probably operate on the same package set), but one of them crashes.
Comment 5 Will Woods 2013-10-10 20:12:09 UTC 
If that's the case, the only real difference would be the repo id (default-installrepo vs cmdline-installrepo).

The repo id also gets used in the repodata cache path, e.g.:

  /var/tmp/system-upgrade/default-installrepo
  /var/tmp/system-upgrade/cmdline-instrepo

Possibly the cmdline-instrepo isn't expiring its cache correctly and you have out-of-date repodata? Can you compare the metadata in cmdline-instrepo/ to
what's on the server?

Also, would you be kind enough to test a couple things for me?

Forcibly expire the metadata cache:

  fedup --network 20 --instrepo $URL --expire-cache

and see if fedup works with --instrepo. If that fails, remove *all* the metadata:

  fedup --network 20 --instrepo $URL --clean-metadata

And try again. If it *still* fails:

  fedup --clean

And try once more.

I'd love to see the fedup.log after/if you get one of those to work.

(NOTE: You might want to back up /var/tmp/system-upgrade/* first to save your downloaded data - or just cmdline-instrepo/ so we can examine the metadata later. It might also be a good idea to move fedup.log to fedup.log.old so I don't have to look through *all* your fedup logs to find the most recent ones.)
Comment 6 Kamil Páral 2013-10-21 16:30:19 UTC 
I haven't gotten to re-testing this yet, but this might be just another occurrence of bug 1005895.
Comment 7 Kamil Páral 2013-10-22 09:47:26 UTC 
This is really a duplicate of bug 1005895. Applying deltarpm patch fixed the problem.

*** This bug has been marked as a duplicate of bug 1005895 ***