Bug 444882 - yum: AttributeError: 'NoneType' object has no attribute 'lower'
yum: AttributeError: 'NoneType' object has no attribute 'lower'
Product: Fedora
Classification: Fedora
Component: yum (Show other bugs)
All Linux
low Severity medium
: ---
: ---
Assigned To: Seth Vidal
Fedora Extras Quality Assurance
Depends On:
  Show dependency treegraph
Reported: 2008-05-01 10:47 EDT by Robert Story
Modified: 2014-01-21 18:02 EST (History)
10 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2008-05-08 12:58:51 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
rpmforge metadat (3.26 MB, application/octet-stream)
2008-05-07 14:25 EDT, Robert Story
no flags Details

  None (edit)
Description Robert Story 2008-05-01 10:47:09 EDT
Description of problem:
I installed the rpmforge repo spec and yum started failing with a stacktrace
that ended with:
  File "/usr/lib/python2.5/site-packages/yum/packages.py", line 68, in
    e = e.lower()
AttributeError: 'NoneType' object has no attribute 'lower'

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
2.rpm -ivh rpmforge*
3.yum update
Actual results:
above error

Expected results:
regular yum process

Additional info:
I simply changed line 68 to be:

            if e is not None:
                e = e.lower()

there should probably be an else case there.. all packages from that repo are
listed with "None" for the epoch... e.g.

perl-HTTP-CryptoCookie.noarch            None:1.14-1.rh9.rf     rpmforge
perl-Tie-EncryptedHash.noarch            1.21-4.fc9             updates
Comment 1 Seth Vidal 2008-05-07 12:37:18 EDT
Can you make it happen only when rpmforge for f9/rawhide is enabled?

Can you attach a copy of the metadata for that repo?

Comment 2 Robert Story 2008-05-07 14:25:57 EDT
Created attachment 304792 [details]
rpmforge metadat

yes, it only happens after the rpmforge repo is added. fedora and livna are
fine. I hope this is the right file for metadata. If not, please be specific
about what you want...
Comment 3 Seth Vidal 2008-05-07 14:55:11 EDT
Yah, as I suspected the metadata is not including an 'epoch' in the <version>
lines with each package unless the package explicitly lists an epoch. This leads
me to believe to question how they're generating the metadata.

I'll go see if I can find out.
Comment 4 Seth Vidal 2008-05-07 15:09:16 EDT
this makes me sad:


note the comments.

this metadata is broken, badly.

please do not use it.

Adding Thorsten, too. I think he has some contacts within rpmforge. I hope.
Comment 5 Thorsten Leemhuis 2008-05-08 00:46:30 EDT
(In reply to comment #4)
> Adding Thorsten, too. I think he has some contacts within rpmforge. I hope.

Sorry, I'm no help here ( rpmforge != rpmfusion);
Comment 7 Seth Vidal 2008-05-08 12:58:51 EDT
Ah, such fun invective. 

okay, closing this as notabug as I'm not interested in getting into that

the repo metadata at rpmforge is not compliant -> notabug.

Comment 8 Seth Vidal 2008-05-08 13:05:14 EDT
And as one more point.

the DTD in the section about the VERSION tag says:

<!ELEMENT version (#PCDATA)>
                  ver CDATA #REQUIRED
                  rel CDATA #REQUIRED>

The version tags in the primary metadata at rpmforge do not all include an epoch
Therefore it is invalid metadata.
Comment 9 Robert Story 2008-05-08 13:57:53 EDT
So what ever happened to writing robust programs that handle bad input
gracefully? I mean, really... it's a 2 line patch to deal with an unset variable.
Comment 10 Jeff Johnson 2008-05-08 14:00:12 EDT
Note the CDATA in the DTD.

An XML DTD sez' almost zero about what contents are assigned to the epoch field because of
the CDATA. The only conformance that can be assumed by citing a DTD is that the CDATA is properly 
escaped for XML parsing.

    <rpm:entry name="perl" flags="GE" epoch="None" ver="5.008001"/>
is likely both DTD conformant and useless as markup for repository contents.
Comment 11 James Antill 2008-05-08 14:16:01 EDT
> So what ever happened to writing robust programs that handle bad input gracefully?

 The world moved on from Postel's law, because when you have many people sending
you crap it's just much better to say "no" than to try guessing at what everyone
 Hell this is _XML_ it's pretty much the definition of a strict format.

> Note the CDATA in the DTD.

 Yeh, so? Your argument is that because the DTD can't check all cases of
worthless data, you are excused when it does catch this specific case?
 I think not.
Comment 12 Jeff Johnson 2008-05-08 15:13:38 EDT
I'm not arguing anything, just pointing out one (of many) limitations
in the existing repomd markup and the createrepo.py implementation.

My example indicates an UNSPECIFIED repomd corner case, nothing more.
Comment 13 James Antill 2008-05-08 16:24:41 EDT
 Unsurprisingly, you seem very confused.

 Is it a "limitation" of the spec. that it doesn't iterate everything that is
not allowed? No, certainly not ... there are basically zero specs. which do that.

 Is it a "limitation" of _some versions of_ createrepo that it allows you to
create output that current yum does not like? No more so that it'd be a bug in
vim to allow you to do the same. That is even more true when it _documents_ the
pros. and cons. of -n in the --help documentation.

 Anyone who thinks what you posted is "unspecified" needs a new definition of
"unspecified", nothing more.
Comment 14 Jeff Johnson 2008-05-08 16:34:43 EDT
No reason to be ad hominem. I'm hardly confused. We appear to be saying the same thing.

Meanwhile, if even 1% of the energy spent in bugzilla had been applied to an actual "fix",
the 2 line change would have been applied and everyone would have benefited. But YMMV ...
Comment 15 Panu Matilainen 2008-05-09 03:37:01 EDT
My 5c... and yes, several years in retrospective.

That epoch is a mandatory field in repodata is a bit of a mistake: as it is, it
doesn't accurately describe the package contents, it describes implicit behavior
of rpm >= 4.2.2 (or thereabouts).

Tracebacking on a *known* case is silly and just looks like a bug to the casual
observer. If you want to be explicit about repodata spec violation then catch it
and error out with "Invalid repository metadata" or something to that effect. Or
just silently add the zero epoch like rpm does. For the latter, patch already
Comment 16 Dag Wieers 2008-05-09 05:30:09 EDT
> the repo metadata at rpmforge is not compliant -> notabug.

The metadata was generated with "createrepo -n"

     -n, --noepoch = don't add zero epochs for non-existent epochs
                    (incompatible with yum and smart but required for
                     systems with rpm < 4.2.1)
Comment 17 Seth Vidal 2008-05-09 07:37:22 EDT
Dag: yah -n (the cli option) still exists in createrepo 0.9.X but it doesn't do

I can only assume you're using 0.4.X at rpmforge. Any particular reason you're
running createrepo on a system with rpm < 4.2.1?

Panu: I already have a patch in my local tree to handle it. Considering my
question about what was generating this metadata was met with a flame about me
being a horrible person I wasn't all that keen on listening to the rest.
Comment 18 Jeff Johnson 2008-05-09 10:28:04 EDT
Panu: nice to see someone finally saying in public that exposing Epoch: in repomd
as #REQUIRED was a "mistake". 'Twas, but script kiddies learn the hard way.

FYI: there are other design flaws in repomd including
    1) rounding RPMTAG_SIZE to next higher disk blocksize in
          <size package=\"%{SIZE}\" installed=\"%{SIZE}\" archive=\"%{ARCHIVESIZE}\"/>
         There is no way a priori to know definitely what fs block size is used. The complexity
          in rounding (and scaling, SuSE uses # of 1K blocks, not bytes, in same data) in
          a data value in repo markup cannot be justified for the (presumed) purpose of
          trying to accurately reflect the actual disk space used.
    2) using the *.rpm file digest, not readily available pre-computed digest values in
          <checksum type=\"sha\" pkgid=\"NO\">%|HDRID?{%{HDRID}}|</checksum>
          Computing a digest on huge quantities of data (as contained in repo's) is extremely
          costly, and is useful iff one needs to verify repo storage _COMPLETELY_. Verifying
           repo storage is perfectly sane, just not every time. See the litter of "caching" speedup
          CLI options in createrepo (and the equivalent in apt-rpm utils), all of which are likely
          used for "speed", voiding the need for <checksum> verification as default behavior.
    3)  adding the start/end header range offsets in
          <rpm:header-range start=\"%{HEADERSTARTOFF}\" end=\"%{HEADERENDOFF}\"/>
          There may have been a need at some point, not any more.
    4) destroying sort order in dependency metadata
          Providing pre-sorted dependency info avoids having to sort the data on every load,
          and simplifies (thereby maximizing usability of repomd) writing efficient dependency
          metadata store retrievals.
    5) no markup for package files stat(2) information
          Not defining stat(2) markup for package payloads limits repomd to the class of "depsolvers" 
          rather than "installers"
But now that you've heard the suggestions explicitly from me, please feel free to claim that I'm 
confused. *shrug*
Comment 19 Dag Wieers 2008-05-09 15:59:59 EDT
Seth: createrepo 0.4.6 is the only decent version that works on RHEL4 for me.
Although I must say I haven't tried anything newer than 0.4.10 (iirc) mostly
because createrepo is not on freshmeat so I cannot subscribe to receive new
release info.

If newer releases of createrepo support RPM < 4.2.1 then maybe I should look
into that, however it needs to run RHEL4 because that is where the metadata is
being created.

PS Life would be much easier if there was an afterthought about older python
releases in general. Having the current yum/createrepo on RHEL3 to RHEL5 would
not split a userbase and allows to reuse documentation/plugins/scripting.
Comment 20 Dag Wieers 2008-05-09 16:09:11 EDT
> Any particular reason you're running createrepo on a system with rpm < 4.2.1?

Being able to use the same version of apt on RHEL2.1, RH7, RH8, RH9, RHEL3,
RHEL4 and RHEL5 using repomd metadata.

Yes, RHEL2.1 is still in support until mid-2009 and some companies may not even
replace those by then. And RH7 is still part of VMware ESX Service Console so
will be around for some more time.

We are not quite rid of RPM < 4.2.1...
Comment 21 James Antill 2008-05-09 18:43:40 EDT
Panu: Not encoding the epoch value explicitly means we have to add the "implicit
behavior of rpm >= 4.2.2" to everything that reads the MD. Personally, I don't
see how that's better.


> but script kiddies learn the hard way.

 I can only assume you don't count as you just created the buffer overflows in rpm?
 And with the introductions out of the way...

#1 is not true, AFAICS. No rounding is done anywhere on size values.

#2 is confusing, you admit that the only way we can see that what we downloaded
matches what we wanted is by using a full file checksum ... but you are arguing
that sometimes when we download things we should magically trust the network?

#3 is needed for yum-3.0.1, which right now is still the current version in
RHEL-5/CentOS-5 ... both of which have people working on yum. So a slightly
different situation than dealing with RHL9 which is _long_ unsupported and noone
who cares about it works on yum, AFAICS.

#4 Sorry, you used "efficiency" and "XML" in the same sentence, you lose. If you
want to do this, you are free to propose it though ... I don't see what harm it
would do.

#5 Assuming that sqlite+bzip2 saves you from the resulting XML data explosion, I
don't see another reason not to include that ... or at least create an extra
filelists_stat_info or whatever. Feel free to propose it, if you really care.

> please feel free to claim that I'm confused.

 Ok, you're confused.
Comment 22 Jeff Johnson 2008-05-10 04:23:05 EDT
James: Please point out the "buffer overflows" I created so that they can be fixed. There are
certainly older versions of rpm (on RHEL2.1 and RHL7 judging from this bug report) that
are in need of maintenance even if you and Fedora and Red Hat son't think the issuae are worth fixing. 

The underlying topic on bz#444882  is whether a known useful 2 line fix to createrepo.py
should/could be back ported, another failed maintenance task (so far, its promising that
the issue may actually be resolved rather than mocked).

rpmfusion is the only repository I'm aware of that is trying to credibly support the large number of
 currently deployed legacy RH distributions rather than advocating the advantages of the latest
and greatest python release. The rpmfusion effort is worthy of support, not twitchy scarcasm.
Comment 23 Jeff Johnson 2008-05-10 04:25:00 EDT
Pardon, I meant rpmforge. In this case I am confused.
Comment 24 James Antill 2008-05-11 22:32:19 EDT
 Seth added the workarounds in yum for this broken MD, so it should work again
after the first Fed-9 update and maybe Fed-8 if we backrelease it.
Comment 25 Ralph Angenendt 2008-05-12 07:40:18 EDT
I'm just wondering if yum versions which are vailable for Red Hat 9 actually
support that "broken" metadata and if we're not just seeing a problem here just
because someone mistook rh9 for "rawhide 9" - otherwise I don't see why the
original reporter would try to use that on Fedora 9 ...

AFAICS you'd probably still use yum 2.0.x on RH9 (as you would on CentOS 3),
which doesn't use the repomd metadata at all, but still relies on the .hdr stuff
which is also there for that repository.

Just my 2 cents ...

Note You need to log in before you can comment on or make changes to this bug.