Bug 636697 - [MASH/KOJI] what about imported RPMs without a build ?
Summary: [MASH/KOJI] what about imported RPMs without a build ?
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: mash
Version: 12
Hardware: All
OS: Linux
low
medium
Target Milestone: ---
Assignee: Bill Nottingham
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2010-09-22 23:42 UTC by tguthmann
Modified: 2014-03-17 03:25 UTC (History)
3 users (show)

Fixed In Version: mash-0.5.20-1.fc13
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2010-10-13 05:57:55 UTC


Attachments (Terms of Use)
Possible workaround (1.29 KB, patch)
2010-09-23 00:42 UTC, tguthmann
no flags Details | Diff
slightly reworked change (864 bytes, text/plain)
2010-09-24 18:57 UTC, Bill Nottingham
no flags Details
Final patch (862 bytes, patch)
2010-09-27 20:39 UTC, tguthmann
no flags Details | Diff

Description tguthmann 2010-09-22 23:42:43 UTC
Description of problem:
I've just discovered that mash does not include packages without a koji 
build despite the fact that a RPM has been imported (koji import 
--create-build [package]). Mash seems to complain because there is no SRPM associated to the package/RPM. Saying sources=false in the repository config doesn't help. See code below.


Version-Release number of selected component (if applicable):
- mash 0.5.17
- koji 1.4.0


How reproducible:
Always reproducible. Import an RPM without any SRPM (and obviously no build) into koji. Then try to mash the repository containing this RPM (already packaged by a 3rd party).


Steps to Reproduce:
1. koji add-tag --owner=you yourrepo packagename
2. koji import --create-build packagename-1.0.0-1.noarch.rpm
3. koji tag-pkg yourepo packagename-1.0.0
4. mash yourrepo


Actual results:
mash throws an exception and exit.
# koji tag-pkg repoX package-17.00-00
# mash repoX
2010-09-22 11:14:40 mash: Getting package lists for repoX...
2010-09-22 11:14:40 mash: Sorting packages...
Traceback (most recent call last):
   File "/usr/bin/mash", line 97, in ?
     main()
   File "/usr/bin/mash", line 77, in main
     rc = themash.doCompose()
   File "/usr/lib/python2.4/site-packages/mash/__init__.py", line 339, 
in doCompose
     srcpkg = src_hash[pkg['build_id']]
KeyError: 389
mash failed in /var/tmp/mash/repoX


Expected results:
mash should work properly and the RPM without build _and_ without SRPM is listed in the generated mash repository.


Additional info:
The issue of bug can be found in mash/__init__.py around line 335. Indeed src_hash does not contain a hash (the package key) for the pkg without SRPM (srcpkg = src_hash[pkg['build_id']]). Later in the code, you do a test to see if there is a SRPM but we never reach this state (fn = _get_reference(srcpkg, builds_hash)). This snippet below should be slighty redone but sadly I've reached my python skills limits so I can't really help anymore. Sorry.

        src_hash = dict([(x['build_id'],x) for x in source.packages()])
        excludearch = dict([(x['build_id'],'') for x in source.packages()])
        exclusivearch = excludearch.copy()
        # Get excludearch/exclusivearch lists for noarch packages
        for pkg in noarch.packages():
            # TOM DEBUG
            self.logger.info("buildid=%s, srcpkg=%s" % (pkg['build_id'],src_hash[pkg['build_id']]) )
            srcpkg = src_hash[pkg['build_id']]
            self.logger.debug("Checking %s for Exclude/ExclusiveArch" % (nevra(pkg),))
            fn = _get_reference(srcpkg, builds_hash)
            # if build has no source rpm, check the binary
            if fn == None:
                fn = _get_reference(pkg, builds_hash)
            try:
                hdr = koji.get_rpm_header(fn)
            except:
                self.logger.error("Couldn't read header from %s (%s)" % (nevra(pkg), fn))
                if fn:
                    fn.close()
                continue
            excludearch[pkg['build_id']] = hdr['EXCLUDEARCH']
            exclusivearch[pkg['build_id']] = hdr['EXCLUSIVEARCH']
            fn.close()

Thanks
Thomas

Comment 1 tguthmann 2010-09-23 00:42:21 UTC
Created attachment 449074 [details]
Possible workaround

Hi,

You will find in attachment a workaround to fix this issue. However I am 
looking forward for a proper fix.

This patch only defaults the dictonnary with 0 if the key is not 
present. Then I do a test to see if the value is 0 and use the RPM 
instead to get the necessary info. 

Cheers

Comment 2 Bill Nottingham 2010-09-24 18:57:41 UTC
Created attachment 449491 [details]
slightly reworked change

Your patch is in the right direction; here's a reworked version. Does this work for you?

Comment 3 tguthmann 2010-09-26 23:59:52 UTC
Hi Bill,

Sorry but it does not work. 

See output below (python is 2.4.3 on el5.5)

# mash REPO
2010-09-27 09:57:09 mash: Getting package lists for REPO...
2010-09-27 09:57:09 mash: Sorting packages...
Traceback (most recent call last):
  File "/usr/bin/mash", line 97, in ?
    main()
  File "/usr/bin/mash", line 77, in main
    rc = themash.doCompose()
  File "/usr/lib/python2.4/site-packages/mash/__init__.py", line 340, in doCompose
    srcpkg = src_hash.get([pkg['build_id']])
TypeError: list objects are unhashable
mash failed in /var/tmp/mash/REPO

Cheers,
Thomas

Comment 4 Bill Nottingham 2010-09-27 17:04:02 UTC
Whoops, remove the braces around 'pkg['build_id']. Sorry about that (wonder how it worked in a brief test here.)

Comment 5 tguthmann 2010-09-27 20:39:13 UTC
Created attachment 450032 [details]
Final patch

Fixed a typo.

Comment 6 tguthmann 2010-09-27 20:42:56 UTC
Hey Bill,

Yes it works now. It's all good. I let you change the status to closed and I hope to see the next release soon :) Thanks for your time and for mash.

Comment 7 Bill Nottingham 2010-09-27 21:02:06 UTC
OK, will be in 0.5.20-1. (Still working out one issue there before release.)

Comment 8 Fedora Update System 2010-09-28 16:17:34 UTC
mash-0.5.20-1.el5 has been submitted as an update for Fedora EPEL 5.
https://admin.fedoraproject.org/updates/mash-0.5.20-1.el5

Comment 9 Fedora Update System 2010-09-28 16:21:55 UTC
mash-0.5.20-1.fc14 has been submitted as an update for Fedora 14.
https://admin.fedoraproject.org/updates/mash-0.5.20-1.fc14

Comment 10 Fedora Update System 2010-09-28 16:33:58 UTC
mash-0.5.20-1.fc13 has been submitted as an update for Fedora 13.
https://admin.fedoraproject.org/updates/mash-0.5.20-1.fc13

Comment 11 Fedora Update System 2010-09-28 17:31:46 UTC
mash-0.5.20-1.fc14 has been pushed to the Fedora 14 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update mash'.  You can provide feedback for this update here: https://admin.fedoraproject.org/updates/mash-0.5.20-1.fc14

Comment 12 Fedora Update System 2010-10-13 05:55:56 UTC
mash-0.5.20-1.el5 has been pushed to the Fedora EPEL 5 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 13 Fedora Update System 2010-10-13 12:49:03 UTC
mash-0.5.20-1.fc14 has been pushed to the Fedora 14 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 14 Fedora Update System 2011-04-14 20:58:36 UTC
mash-0.5.20-1.fc13 has been pushed to the Fedora 13 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.