Bug 442064 - need 2 passes: current downloads aren't good enough to continue
need 2 passes: current downloads aren't good enough to continue
Status: CLOSED NEXTRELEASE
Product: Fedora
Classification: Fedora
Component: pungi (Show other bugs)
9
All Linux
low Severity low
: ---
: ---
Assigned To: David Cantrell
Fedora Extras Quality Assurance
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2008-04-11 11:18 EDT by John Reiser
Modified: 2013-01-09 20:49 EST (History)
0 users

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2008-07-16 13:42:11 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description John Reiser 2008-04-11 11:18:58 EDT
Description of problem: I have to run twice to get the .rpms, then twice to get
the .src.rpms, before a compose succeeds.  Somehow the packages which are
downloaded aren't good enough the first time, but are OK the second.


Version-Release number of selected component (if applicable):
pungi-1.2.13-1.fc9.noarch


How reproducible:
at least 4 times this week, on both i386 and x86_64

Steps to Reproduce:
1. pungi -c /usr/share/pungi/rawhide-fedora.ks --destdir=/data/Fedora9 --name
Fedora --ver 9 --discs 6

2.
3.
  
Actual results: Downloads of each kind (binary, source) "succeed" but somehow
are not good enough to continue the current compose.  Re-running (after cleanup
of destination directory) succeeds with no additional downloads of the
previously-failing type.  Example:

-----first run of pungi
  <<several binary packages (30 or so) for i386 were downloaded, including the
one below:>>
Pungi.Gather:ERROR: Errors were encountered while downloading packages.
Pungi.Gather:ERROR: 50:aspell-sl-0.50-3.fc9.i386: failure:
Packages/aspell-sl-0.50-3.fc9.i386.rpm from rawhide: [Errno 256] No more mirrors
to try.
  <<pungi exits>>
-----

-----second run of pungi
  <<no binary packages downloaded>>
  <<source package downloads follow:>>
(1/11): selinux-policy-3. 100% |=========================| 710 kB    00:04
(2/11): parted-1.8.8-5.fc 100% |=========================| 1.5 MB    00:10
(3/11): gnome-session-2.2 100% |=========================| 789 kB    00:05
(4/11): gdm-2.21.10-0.200 100% |=========================| 3.3 MB    00:22
(5/11): cairo-1.6.0-1.fc9 100% |=========================| 4.8 MB    00:36
(6/11): mesa-7.1-0.23.fc9 100% |=========================| 5.8 MB    00:40
(7/11): mediawiki-1.10.4- 100% |=========================| 4.2 MB    00:27
(8/11): anaconda-11.4.0.7 100% |=========================| 4.0 MB    00:27
(9/11): eclipse-3.3.2-9.f 100% |=========================|  83 MB    09:29
(10/11): gnome-panel-2.22 100% |=========================| 3.2 MB    00:29
(11/11): glibc-2.7.90-15. 100% |=========================|  15 MB    01:46
Pungi.Gather:ERROR: Errors were encountered while downloading packages.
Pungi.Gather:ERROR: 1:eclipse-3.3.2-9.fc9.src: failure:
eclipse-3.3.2-9.fc9.src.rpm from rawhide-source: [Errno 256] No more mirrors to try.
Pungi.Gather:ERROR: parted-1.8.8-5.fc9.src: failure: parted-1.8.8-5.fc9.src.rpm
from rawhide-source: [Errno 256] No more mirrors to try.
Pungi.Gather:ERROR: glibc-2.7.90-15.src: failure: glibc-2.7.90-15.src.rpm from
rawhide-source: [Errno 256] No more mirrors to try.
Pungi.Gather:ERROR: mesa-7.1-0.23.fc9.src: failure: mesa-7.1-0.23.fc9.src.rpm
from rawhide-source: [Errno 256] No more mirrors to try.
Pungi.Gather:ERROR: selinux-policy-3.3.1-33.fc9.src: failure:
selinux-policy-3.3.1-33.fc9.src.rpm from rawhide-source: [Errno 256] No more
mirrors to try.
Pungi.Gather:ERROR: anaconda-11.4.0.71-1.src: failure:
anaconda-11.4.0.71-1.src.rpm from rawhide-source: [Errno 256] No more mirrors to
try.
Pungi.Gather:ERROR: gnome-session-2.22.1.1-1.fc9.src: failure:
gnome-session-2.22.1.1-1.fc9.src.rpm from rawhide-source: [Errno 256] No more
mirrors to try.
Pungi.Gather:ERROR: cairo-1.6.0-1.fc9.src: failure: cairo-1.6.0-1.fc9.src.rpm
from rawhide-source: [Errno 256] No more mirrors to try.
Pungi.Gather:ERROR: gnome-panel-2.22.1.2-1.fc9.src: failure:
gnome-panel-2.22.1.2-1.fc9.src.rpm from rawhide-source: [Errno 256] No more
mirrors to try.
Pungi.Gather:ERROR: mediawiki-1.10.4-39.fc9.src: failure:
mediawiki-1.10.4-39.fc9.src.rpm from rawhide-source: [Errno 256] No more mirrors
to try.
Pungi.Gather:ERROR: 1:gdm-2.21.10-0.2008.04.08.4.fc9.src: failure:
gdm-2.21.10-0.2008.04.08.4.fc9.src.rpm from rawhide-source: [Errno 256] No more
mirrors to try.
  <<pungi exits>>
-----

-----third run of pungi
  <<no complaints; entire compose succeeds>>
-----




Expected results:
Binary or source packages downloaded during a compose are used successfully in
subsequent phases of the same run.

Additional info:  I accumulate downloaded packages in /var/cache/pungi/rawhide*:
currently something like 5693 .rpms and 1751 .src.rpms ["find . | wc"]  Each day
before I start on a new rawhide, I "rm -rf /data/Fedora9/*" to clean the
destination directory (but leave the directory itself.)  Upon failure of a pass,
I remove the 'logs' and 'work' directories, and the 9/i386 or 9/x86_64
directory.  On the second architecture, I leave the results from directory
9/<other_arch>. Then I rerun the failing pass (binary download or source
download), and it succeeds the second time.

The .ks file specifies one particular repository; typically I had problems with
inconsistencies among the mirrors.
-----/usr/share/pungi/rawhide-fedora.ks
#repo --name=rawhide        
--mirrorlist=http://mirrors.fedoraproject.org/mirrorlist?repo=rawhide&arch=$basearch
#repo --name=rawhide-source 
--mirrorlist=http://mirrors.fedoraproject.org/mirrorlist?repo=rawhide-source&arch=$basearch
repo --name=rawhide        
--baseurl=http://download.fedora.redhat.com/pub/fedora/linux/development/$basearch/os
repo --name=rawhide-source 
--baseurl=http://download.fedora.redhat.com/pub/fedora/linux/development/source/SRPMS
-----
Comment 1 Jesse Keating 2008-04-11 13:56:05 EDT
Yeah, there is a bug that got introduced I think because of yum some point
recently.  The code I have for downloading looks for any error out put and will
act as if the download failed.  But it seems that something is leaking failures
but the download is OK.  I haven't had a lot of time to track it down, I hope to
after F9 goes out the door and I'll put out an F9 update.
Comment 2 Bug Zapper 2008-05-14 05:18:59 EDT
Changing version to '9' as part of upcoming Fedora 9 GA.
More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 3 Jesse Keating 2008-07-15 22:24:21 EDT
hrm, I can't duplicate now.  Is this still an issue?
Comment 4 John Reiser 2008-07-16 11:01:46 EDT
Yes, there is still an issue about communication between buildinstall and pungi.
 pungi quits, saying that buildinstall gave an error, but I don't see anything
wrong.

-----tail of console log
Pungi.Gather:INFO: Finished downloading packages.
Pungi.Pungi:INFO: Making repodata
Previous repo file missing: 
1750/1750 - Packages/libesmtp-1.0.4-7.fc9.i386.rpm                     rpm
Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
Sqlite DBs complete
Pungi.Pungi:INFO: Running /usr/bin/repoview --quiet --state-dir
/var/cache/pungi/repoviewcache --title "Fedora 10 - i386" /data/Fedora10/10/i386/os
Pungi.Pungi:INFO: Linking release note file GPL
Pungi.Pungi:INFO: Linking release note file README-BURNING-ISOS-en_US.txt
Pungi.Pungi:INFO: Linking release note file RPM-GPG-KEY-fedora-i386
Pungi.Pungi:INFO: Linking release note file RPM-GPG-KEY-fedora-test-x86_64
Pungi.Pungi:INFO: Linking release note file RPM-GPG-KEY-fedora-ppc
Pungi.Pungi:INFO: Linking release note file RPM-GPG-KEY-fedora-test-ppc64
Pungi.Pungi:INFO: Linking release note file RPM-GPG-KEY-fedora-primary
Pungi.Pungi:INFO: Linking release note file RPM-GPG-KEY-fedora-test-ia64
Pungi.Pungi:INFO: Linking release note file RPM-GPG-KEY-fedora-x86_64
Pungi.Pungi:INFO: Linking release note file RPM-GPG-KEY-fedora-ia64
Pungi.Pungi:INFO: Linking release note file RPM-GPG-KEY-fedora-test-i386
Pungi.Pungi:INFO: Linking release note file RPM-GPG-KEY-fedora-test-primary
Pungi.Pungi:INFO: Linking release note file RPM-GPG-KEY-fedora-ppc64
Pungi.Pungi:INFO: Linking release note file RPM-GPG-KEY-fedora-test-ppc
Pungi.Pungi:INFO: Linking release note file RPM-GPG-KEY-fedora-test
Pungi.Pungi:INFO: Linking release note file RPM-GPG-KEY-fedora
Pungi.Pungi:INFO: Running /usr/lib/anaconda-runtime/buildinstall --product
Fedora --version 10 --release "Fedora 10" --bugurl http://bugzilla.redhat.com
/data/Fedora10/10/i386/os
Pungi.Pungi:ERROR: Got an error from /usr/lib/anaconda-runtime/buildinstall
Pungi.Pungi:ERROR: 
Traceback (most recent call last):
  File "/usr/bin/pungi", line 189, in <module>
    main()
  File "/usr/bin/pungi", line 98, in main
    mypungi.doBuildinstall()
  File "/usr/lib/python2.5/site-packages/pypungi/pungi.py", line 168, in
doBuildinstall
    pypungi._doRunCommand(buildinstall, self.logger) #, env={"TMPDIR":
self.workdir})
  File "/usr/lib/python2.5/site-packages/pypungi/__init__.py", line 70, in
_doRunCommand
    raise OSError, "Got an error from %s: %s" % (command[0], err)
OSError: Got an error from /usr/lib/anaconda-runtime/buildinstall: 
# 
-----end of console log
Comment 5 Jesse Keating 2008-07-16 11:15:34 EDT
What's in the log file itself?
Comment 6 John Reiser 2008-07-16 12:59:36 EDT
Fedora10/logs/i386.log has 82094 lines, ending with:
-----
Pungi.Pungi.DEBUG: Running buildinstall...
/tmp/buildinstall.tree.ELec68 /tmp
No Match for argument anaconda-runtime
Nothing to download

Pungi.Pungi.ERROR: Got an error from /usr/lib/anaconda-runtime/buildinstall
Pungi.Pungi.ERROR:
-----
where that last line is "Pungi.Pungi.ERROR: " with one space after the colon.
Comment 7 John Reiser 2008-07-16 13:01:28 EDT
Perhaps I didn't give enough context.  The two preceding lines are:
-----
Pungi.Pungi.INFO: Linking release note file RPM-GPG-KEY-fedora
Pungi.Pungi.INFO: Running /usr/lib/anaconda-runtime/buildinstall --product
Fedora --version 10 --release "Fedora 10" --bugurl http://bugzilla.redhat.com
/data/Fedora10/10/i386/os
-----
Comment 8 Jesse Keating 2008-07-16 13:13:16 EDT
Are you using the Fedora 9 version of pungi for this?  What's your pungi
version, because the buildinstall call doesn't seem to have the repos you used
to do your compose passed to it, which is necessary.  You'll need the rawhide
version of buildinstall too.

You can't use F9 to compose f10 unless you do it in a chroot comprised of f10
packages.
Comment 9 John Reiser 2008-07-16 13:31:26 EDT
Yes, that was under Fedora 9 with pungi-1.2.18.1-1.fc9.noarch.  With only
medium-speed DSL, rawhide is hard to use until a little bit before alpha.

Why cannot pungi check these things, then complain if necessary?  It would be so
nice if vanilla F9 were enough to compose F10.

Comment 10 Jesse Keating 2008-07-16 13:42:11 EDT
You're right, I should have /some/ way to determine that you're doing something
that isn't the native running system.  I just don't know of a clever way of
doing that as of yet.

Anyway, since pungi uses the system installed anaconda-runtime for doing the
buildinstall work, it really needs the version of anaconda that will be used for
the install locally installed.  That's why you need the f10 package set.  YOu
can do this in a mock chroot so that you don't have to 'run rawhide'.

I'm closing this as NEXTRELEASE since it's not a problem in rawhide and the F10
package.

Note You need to log in before you can comment on or make changes to this bug.