Bug 718635 - Kojid unable to regen-repo in Fedora 15 using createrepo-0.9.9-3.fc15.noarch
Summary: Kojid unable to regen-repo in Fedora 15 using createrepo-0.9.9-3.fc15.noarch
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Fedora
Classification: Fedora
Component: koji
Version: 15
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Dennis Gilmore
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-07-04 05:34 UTC by Anthony Messina
Modified: 2014-01-21 23:18 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2012-08-07 19:49:32 UTC
Type: ---


Attachments (Terms of Use)
koji logs for createrepo using rawhide (18.17 KB, text/x-log)
2011-07-29 20:41 UTC, Anthony Messina
no flags Details

Description Anthony Messina 2011-07-04 05:34:22 UTC
As described on the buildsys mailing list in the following thread: http://lists.fedoraproject.org/pipermail/buildsys/2011-May/003660.html

I'm running a Kojid instance, freshly upgraded from Fedora 14 -> 15.  My
koji hub is still running Fedora 14.

When running a regen-repo, I now get this error:

2011-05-25 20:20:41,405 [WARNING] koji.TaskManager: TRACEBACK: Traceback
(most recent call last):
  File "/usr/lib/python2.7/site-packages/koji/daemon.py", line 1114, in
runTask
    response = (handler.run(),)
  File "/usr/lib/python2.7/site-packages/koji/tasks.py", line 146, in run
    return self.handler(*self.params,**self.opts)
  File "/usr/sbin/kojid", line 2552, in handler
    self.create_local_repo(rinfo, arch, pkglist, groupdata, oldrepo)
  File "/usr/sbin/kojid", line 2606, in create_local_repo
    % parseStatus(status, ' '.join(cmd))
GenericError: failed to create repo: /usr/bin/createrepo -vd -o
/tmp/koji/tasks/1796/1796/repo -u http://messinet.com/koji-mnt/packages
-i /mnt/koji/repos/dist-f14-build/221/x86_64/pkglist -g
/mnt/koji/repos/dist-f14-build/221/groups/comps.xml --update --skip-stat
/mnt/koji/packages/ exited with status 1


The following version works:
createrepo-0.9.8-5.fc14.noarch

The following does not work with the error as above:
createrepo-0.9.9-3.fc15.noarch

Comment 1 seth vidal 2011-07-29 19:16:19 UTC
I've gotten partial confirmation that createrepo from rawhide fixes this.

I'd appreciate more testing.

thanks

Comment 2 Anthony Messina 2011-07-29 20:10:38 UTC
I looked into trying rawhide previously, but it also pulls in an update for yum.  Perhaps you have some guidance, but should I allow the update of createrepo and yum from the rawhide repo on this f15 stable machine?

Comment 3 Anthony Messina 2011-07-29 20:41:19 UTC
Created attachment 515942 [details]
koji logs for createrepo using rawhide

I've upgraded to createrepo-0.9.9-5.fc16.noarch (and yum for deps) on my f15 koji builders and the attached koji regen-repo output is different, but it appears to at least create the build repos without failing.

Comment 4 Anthony Messina 2011-07-31 09:13:28 UTC
After upgrading to createrepo-0.9.9-5.fc16.noarch on my koji builders, as stated above, I can get through the repo creation without any errors, however, when trying to build a package with the new repos, I get the error below about missing the 'pkgorigins.gz' file.  Sure enough, there is no file named 'pkgorigins.gz' as shown:


Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/koji/daemon.py", line 1114, in runTask
    response = (handler.run(),)
  File "/usr/lib/python2.7/site-packages/koji/tasks.py", line 146, in run
    return self.handler(*self.params,**self.opts)
  File "/usr/sbin/kojid", line 986, in handler
    broot.init()
  File "/usr/sbin/kojid", line 373, in init
    self.session.host.setBuildRootList(self.id,self.getPackageList())
  File "/usr/sbin/kojid", line 456, in getPackageList
    self.markExternalRPMs(ret)
  File "/usr/sbin/kojid", line 559, in markExternalRPMs
    fo = koji.openRemoteFile(relpath, **opts)
  File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 1376, in openRemoteFile
    fo = open(fn)
IOError: [Errno 2] No such file or directory: '/mnt/koji//repos/dist-f15-build/336/i386/repodata/pkgorigins.gz'

Comment 5 Anthony Messina 2011-07-31 09:25:26 UTC
What I do have in the /mnt/koji//repos/dist-f15-build/336/i386/repodata/ directory is:
1e4d72283c5621497f933ad550d1b45a205c6eae-comps.xml
25cacd5f497c06d1687113dd3dbe9c5c09ef55ec-primary.xml.gz
34c3c5c09b0ff133c05f466faf3d0d721f59338a-other.sqlite.bz2
392fef88484c0f0c69677dee6923925707a7d177-filelists.sqlite.bz2
74553315be2cdb807e4848f462d5a91402ada7ad-pkgorigins.gz
79228c8d6f33e7ebdcf66747caaee21b34e4adf9-other.xml.gz
b96bf1153302f65733cb79732c23e5eca742298f-comps.xml.gz
f49c7c9fdb18021b69ddbb26cc2c9f0ea1cc77e6-primary.sqlite.bz2
fad66ebb8f1effa032aa8ada21ab9eec2658e55c-filelists.xml.gz
repomd.xml


In previous versions of createrepo (createrepo-0.9.8-5.fc14.noarch), the list looked like:
comps.xml
comps.xml.gz
filelists.sqlite.bz2
filelists.xml.gz
other.sqlite.bz2
other.xml.gz
pkgorigins.gz
primary.sqlite.bz2
primary.xml.gz
repomd.xml

Comment 6 seth vidal 2011-08-01 16:22:51 UTC
1. the traceback from comment4 is not from createrepo. The file is just missing, this isn't something from createrepo.

2. yes - the new createrepo requires a new yum.

Comment 7 Anthony Messina 2011-08-01 17:52:22 UTC
(In reply to comment #6)
> 1. the traceback from comment4 is not from createrepo. The file is just
> missing, this isn't something from createrepo.

Perhaps not, but that is what happens when a F15 koji builder (koji-builder-1.6.0-2.fc15.noarch) attempts to regenerate a repo using createrepo-0.9.9-5.fc16.noarch.

In earlier versions of createrepo (as used by koji), the sums weren't prepended to the files in the repodata directory.

Does koji need to be updated in order to work with createrepo-0.9.9-5.fc16.noarch?

Comment 8 seth vidal 2011-08-01 21:06:31 UTC
not that I am aware of, no.

Comment 9 Anthony Messina 2011-08-05 14:05:41 UTC
Ok, I've looked into this a bit and may be going about this in
completely the wrong way, but...

Modifying line 2577
(https://fedorahosted.org/koji/browser/builder/kojid#L2577) does not work.

cmd = ['/usr/bin/createrepo', '-vd', '--simple-md-filenames', '-o',
self.outdir, '-u', self.options.topurl]

However, modifying mergerepos to add the following after line 96
(https://fedorahosted.org/koji/browser/builder/mergerepos#L96) allows
the koji repo to be regenerated AND allows packages to be built from
that repo:

self.mdconf.unique_md_filenames = False

I am not certain that this is THE way to fix it, but it is a workaround
for now.

Since this seems to be a change in Koji, rather than createrepo, how should I go about including the Koji group on this?

Comment 10 seth vidal 2011-08-08 16:50:53 UTC
mergerepos in koji is From koji - not from createrepo.

Might want to reassign this bug?

Comment 11 Anthony Messina 2011-08-09 04:18:22 UTC
Modified component to Koji.  Thanks, Seth.

Comment 12 Nathanael Noblet 2011-10-06 06:37:15 UTC
I can confirm both the behaviour and proposed fix seem to work for me... whether the fix is correct I have no idea.

Comment 13 Michael Cronenworth 2011-10-17 15:56:07 UTC
This bug is still present as of today's Fedora updates.

Comment 14 Michael Cronenworth 2011-11-28 18:05:59 UTC
Issue seems to be resolved with all the latest updates.

Comment 15 Ken Dreyer 2011-11-28 18:29:48 UTC
(In reply to comment #14)
> Issue seems to be resolved with all the latest updates.

I'm still seeing this on my F16 Koji setup. Koji-builder is at 1.6.0-2. I don't think it's fixed...

Comment 16 Michael Cronenworth 2011-11-28 18:38:03 UTC
(In reply to comment #15)
> I'm still seeing this on my F16 Koji setup. Koji-builder is at 1.6.0-2. I don't
> think it's fixed...

I must have been seeing a different issue then. I've called regen-repo several times without any trouble on tags that have F15 and F16 with local and external repos. I can also build packages against those tags.

Comment 17 Ken Dreyer 2011-11-28 19:04:40 UTC
Yeah, I confirmed it on my setup. amessina's patch fixes it for me. Filed "upstream" at https://fedorahosted.org/koji/ticket/227

Comment 18 Fedora Admin XMLRPC Client 2012-01-31 12:02:02 UTC
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Comment 19 Fedora End Of Life 2012-08-07 19:49:34 UTC
This message is a notice that Fedora 15 is now at end of life. Fedora
has stopped maintaining and issuing updates for Fedora 15. It is
Fedora's policy to close all bug reports from releases that are no
longer maintained. At this time, all open bugs with a Fedora 'version'
of '15' have been closed as WONTFIX.

(Please note: Our normal process is to give advanced warning of this
occurring, but we forgot to do that. A thousand apologies.)

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, feel free to reopen
this bug and simply change the 'version' to a later Fedora version.

Bug Reporter: Thank you for reporting this issue and we are sorry that
we were unable to fix it before Fedora 15 reached end of life. If you
would still like to see this bug fixed and are able to reproduce it
against a later version of Fedora, you are encouraged to click on
"Clone This Bug" (top right of this page) and open it against that
version of Fedora.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

The process we are following is described here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping


Note You need to log in before you can comment on or make changes to this bug.