Bug 1073256

Actually, this is even weirder.

The above one-line patch doesn't fix the problem, it creates a new one:

Traceback (most recent call last):
  File "/usr/bin/mash", line 100, in <module>
    main()
  File "/usr/bin/mash", line 77, in main
    rc = themash.doCompose()
  File "/usr/lib/python2.6/site-packages/mash/__init__.py", line 422, in doCompose
    pid = _write_files(source.packages(), path, path, repocache = repocache, arch = 'SRPMS')
  File "/usr/lib/python2.6/site-packages/mash/__init__.py", line 245, in _write_files
    status = self._makeMetadata(repo_path, repocache, arch, comps, previous = previous_path)
  File "/usr/lib/python2.6/site-packages/mash/__init__.py", line 131, in _makeMetadata
    md.run(path)
  File "/usr/lib/python2.6/site-packages/mash/metadata.py", line 247, in run
    self.obj.run(path)
  File "/usr/lib/python2.6/site-packages/mash/metadata.py", line 200, in run
    self.repomatic.doRepoMetadata()
  File "/usr/lib/python2.6/site-packages/createrepo/__init__.py", line 1097, in doRepoMetadata
    fo.write(repomd.dump_xml())
  File "/usr/lib/python2.6/site-packages/yum/repoMDObject.py", line 228, in dump_xml
    for (cpeid, item) in self.tags['distro'].items():
AttributeError: 'list' object has no attribute 'items'

So clearly, self.tags['distro'] is a list.

But in the __init__ function of the class:

        self.tags      = {'content' : set(), 'distro' : {}, 'repo': set()}

So, self.tags['distro'] is declared as a dict, but it becomes a list at some point. :-/

With a bit of pdb magic, I find that at the point of the backtrace:

(Pdb) self.tags['distro']
[['cpe:/o:networkbox:nbrs:5.0', 'nb5.0-free']]

That's weird, The current code would expect that to be ['cpe:/o:networkbox:nbrs:5.0', 'nb5.0-free'] instead...

I'm at a loss, here. :-/

Scratch those last 2 paragraphs, the thing I saw in pdb is what the code expects.

----

A bit more background, I'm running mash through Bodhi here, and that's what triggers this error.

So I've just added a mere print in yum/repoMDObject.py (just before that line 228), and run a new updates push with my Bodhi instance.

And I can see that during the push, I'm going several times through that code, and self.tags['distro'] will have a different value, one of:

    [['cpe:/o:networkbox:nbrs:5.0', 'nb5.0-free']]
    {'cpe:/o:networkbox:nbrs:5.0': set(['n'])}

The former is what the current code expects, the second one is not, triggering the exception I pasted in comment 0.

There's something wrong here, why would this value change like this?

Ha! Found it!

The culprit is createrepo. In /usr/lib/python2.6/site-packages/createrepo/__init__.py, at line 919:

        if self.conf.distro_tags:
            repomd.tags['distro'] = self.conf.distro_tags

Here, self.conf.distro_tags is [['cpe:/o:networkbox:nbrs:5.0', 'nb5.0-free']]

So, I guess the problem is two fold:

1. createrepo should set repomd.tags['distro'] to a dictionary, as expected by yum

2. yum should be changed to add the .items() I had suggested in comment 0.

Hi Mathieu, thank you for the investigation. Could you give me the exact steps how to reproduce this so that I can check your suggested changes? Precise details would be appreciated, since I don't have much experience with mash.

Hmm...

Right now I'm reproducing it by pushing updates to a repository with Bodhi... except it only happens with some experimental code I am working on for Bodhi.

I'll try to cook up a reproducer for you. :)

Hey Valentina, here's a trivial reproducer, using only standard tools. :)

1. Create an empty updateinfo.xml file (the problem exists also with non-empty files, but it's faster that way for a reproducer) :

    $ cat updateinfo.xml
    <?xml version="1.0" ?><updates/>

2. Create an empty repository (the problem exists also with non-empty ones, but it's faster that way for a reproducer) :

    $ mkdir testrepo
    $ createrepo --distro="cpeid,foobar" foobar

3. Now try adding the updateinfo metadata to the repository:

    $ modifyrepo --mdtype=updateinfo updateinfo.xml testrepo/repodata
    Wrote: /store/koji/mash/mashed/testrepo/repodata/updateinfo.xml.gz
               type = updateinfo
           location = repodata/5744b86a8645cec8c870f2091488abe79734a5708a0ce67a50cd39b97466c853-updateinfo.xml.gz
           checksum = 5744b86a8645cec8c870f2091488abe79734a5708a0ce67a50cd39b97466c853
          timestamp = 1394101015.16
      open-checksum = c55e699655c8ed1858ee32f2436cbecae58120385c81aa7e4d44ab85b4e82161
    Traceback (most recent call last):
      File "/usr/share/createrepo/modifyrepo.py", line 219, in <module>
        ret = main(sys.argv[1:])
      File "/usr/share/createrepo/modifyrepo.py", line 212, in main
        repomd.add(metadata, mdtype=opts.mdtype)
      File "/usr/share/createrepo/modifyrepo.py", line 148, in add
        self._write_repomd()
      File "/usr/share/createrepo/modifyrepo.py", line 77, in _write_repomd
        outmd.write(self.repoobj.dump_xml())
      File "/usr/lib/python2.6/site-packages/yum/repoMDObject.py", line 228, in dump_xml
        for (cpeid, item) in self.tags['distro']:
    ValueError: too many values to unpack

----------

Both createrepo and modifyrepo call the same Yum code:

    File "/usr/lib/python2.6/site-packages/yum/repoMDObject.py", line 228, in dump_xml

However, when createrepo gets to that line, self.tags['distro'] is:

    [('cpeid', 'foobar')]

But when modifyrepo does it, self.tags['distro'] is:

    {'cpeid': set(['f'])}

Created attachment 871336 [details]
distro_tags should be a dict

This is the createrepo side of the change.

Created attachment 871337 [details]
repoMDObject: tags['distro'] is a dict

This is the Yum side of the change.

The two patches I just attached fix the problem for me, and apply cleanly on the latest source RPMs for createrepo and yum.

Obviously, these should be pushed together, otherwise one will break the other.

In case that helps makes this fix be released faster, we need this in Bodhi (used at least by the Fedora infrastructure) :

https://github.com/fedora-infra/bodhi/pull/35#issuecomment-36818510

Mathieu, thank you! I have found an upstream commit which fixes the problem for me - http://yum.baseurl.org/gitweb?p=yum.git;a=commitdiff;h=6a65f13183ee77a32f68c65ce5f47025f84509c7

Please let me know if it works for you.

Yup, it fixes the problem for me too.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2014-1410.html