Can you make it happen only when rpmforge for f9/rawhide is enabled?

Can you attach a copy of the metadata for that repo?

Created attachment 304792 [details]
rpmforge metadat

yes, it only happens after the rpmforge repo is added. fedora and livna are
fine. I hope this is the right file for metadata. If not, please be specific
about what you want...

Yah, as I suspected the metadata is not including an 'epoch' in the <version>
lines with each package unless the package explicitly lists an epoch. This leads
me to believe to question how they're generating the metadata.

I'll go see if I can find out.

this makes me sad:

http://apt.sw.be/redhat/9/en/i386/rpmforge/repodata/repomd.xml

note the comments.

this metadata is broken, badly.

please do not use it.

Adding Thorsten, too. I think he has some contacts within rpmforge. I hope.

(In reply to comment #4)
> Adding Thorsten, too. I think he has some contacts within rpmforge. I hope.

Sorry, I'm no help here ( rpmforge != rpmfusion);

fyi,
http://lists.rpmforge.net/pipermail/users/2008-May/001584.html

Ah, such fun invective. 

okay, closing this as notabug as I'm not interested in getting into that
ridiculousness.

the repo metadata at rpmforge is not compliant -> notabug.

And as one more point.

the DTD in the section about the VERSION tag says:

<!ELEMENT version (#PCDATA)>
<!ATTLIST version epoch CDATA #REQUIRED
                  ver CDATA #REQUIRED
                  rel CDATA #REQUIRED>

The version tags in the primary metadata at rpmforge do not all include an epoch
attribute.
Therefore it is invalid metadata.

So what ever happened to writing robust programs that handle bad input
gracefully? I mean, really... it's a 2 line patch to deal with an unset variable.

Note the CDATA in the DTD.

An XML DTD sez' almost zero about what contents are assigned to the epoch field because of
the CDATA. The only conformance that can be assumed by citing a DTD is that the CDATA is properly 
escaped for XML parsing.

IOW,
    <rpm:entry name="perl" flags="GE" epoch="None" ver="5.008001"/>
is likely both DTD conformant and useless as markup for repository contents.

> So what ever happened to writing robust programs that handle bad input gracefully?

 The world moved on from Postel's law, because when you have many people sending
you crap it's just much better to say "no" than to try guessing at what everyone
wants.
 Hell this is _XML_ it's pretty much the definition of a strict format.

> Note the CDATA in the DTD.

 Yeh, so? Your argument is that because the DTD can't check all cases of
worthless data, you are excused when it does catch this specific case?
 I think not.

I'm not arguing anything, just pointing out one (of many) limitations
in the existing repomd markup and the createrepo.py implementation.

My example indicates an UNSPECIFIED repomd corner case, nothing more.

 Unsurprisingly, you seem very confused.

 Is it a "limitation" of the spec. that it doesn't iterate everything that is
not allowed? No, certainly not ... there are basically zero specs. which do that.

 Is it a "limitation" of _some versions of_ createrepo that it allows you to
create output that current yum does not like? No more so that it'd be a bug in
vim to allow you to do the same. That is even more true when it _documents_ the
pros. and cons. of -n in the --help documentation.

 Anyone who thinks what you posted is "unspecified" needs a new definition of
"unspecified", nothing more.

No reason to be ad hominem. I'm hardly confused. We appear to be saying the same thing.

Meanwhile, if even 1% of the energy spent in bugzilla had been applied to an actual "fix",
the 2 line change would have been applied and everyone would have benefited. But YMMV ...

My 5c... and yes, several years in retrospective.

That epoch is a mandatory field in repodata is a bit of a mistake: as it is, it
doesn't accurately describe the package contents, it describes implicit behavior
of rpm >= 4.2.2 (or thereabouts).

Tracebacking on a *known* case is silly and just looks like a bug to the casual
observer. If you want to be explicit about repodata spec violation then catch it
and error out with "Invalid repository metadata" or something to that effect. Or
just silently add the zero epoch like rpm does. For the latter, patch already
exists.

> the repo metadata at rpmforge is not compliant -> notabug.

The metadata was generated with "createrepo -n"

     -n, --noepoch = don't add zero epochs for non-existent epochs
                    (incompatible with yum and smart but required for
                     systems with rpm < 4.2.1)

Dag: yah -n (the cli option) still exists in createrepo 0.9.X but it doesn't do
anything.

I can only assume you're using 0.4.X at rpmforge. Any particular reason you're
running createrepo on a system with rpm < 4.2.1?

Panu: I already have a patch in my local tree to handle it. Considering my
question about what was generating this metadata was met with a flame about me
being a horrible person I wasn't all that keen on listening to the rest.

Panu: nice to see someone finally saying in public that exposing Epoch: in repomd
as #REQUIRED was a "mistake". 'Twas, but script kiddies learn the hard way.

FYI: there are other design flaws in repomd including
    1) rounding RPMTAG_SIZE to next higher disk blocksize in
          <size package=\"%{SIZE}\" installed=\"%{SIZE}\" archive=\"%{ARCHIVESIZE}\"/>
         There is no way a priori to know definitely what fs block size is used. The complexity
          in rounding (and scaling, SuSE uses # of 1K blocks, not bytes, in same data) in
          a data value in repo markup cannot be justified for the (presumed) purpose of
          trying to accurately reflect the actual disk space used.
    2) using the *.rpm file digest, not readily available pre-computed digest values in
          <checksum type=\"sha\" pkgid=\"NO\">%|HDRID?{%{HDRID}}|</checksum>
          Computing a digest on huge quantities of data (as contained in repo's) is extremely
          costly, and is useful iff one needs to verify repo storage _COMPLETELY_. Verifying
           repo storage is perfectly sane, just not every time. See the litter of "caching" speedup
          CLI options in createrepo (and the equivalent in apt-rpm utils), all of which are likely
          used for "speed", voiding the need for <checksum> verification as default behavior.
    3)  adding the start/end header range offsets in
          <rpm:header-range start=\"%{HEADERSTARTOFF}\" end=\"%{HEADERENDOFF}\"/>
          There may have been a need at some point, not any more.
    4) destroying sort order in dependency metadata
          Providing pre-sorted dependency info avoids having to sort the data on every load,
          and simplifies (thereby maximizing usability of repomd) writing efficient dependency
          metadata store retrievals.
    5) no markup for package files stat(2) information
          Not defining stat(2) markup for package payloads limits repomd to the class of "depsolvers" 
          rather than "installers"
         
But now that you've heard the suggestions explicitly from me, please feel free to claim that I'm 
confused. *shrug*

Seth: createrepo 0.4.6 is the only decent version that works on RHEL4 for me.
Although I must say I haven't tried anything newer than 0.4.10 (iirc) mostly
because createrepo is not on freshmeat so I cannot subscribe to receive new
release info.

If newer releases of createrepo support RPM < 4.2.1 then maybe I should look
into that, however it needs to run RHEL4 because that is where the metadata is
being created.

PS Life would be much easier if there was an afterthought about older python
releases in general. Having the current yum/createrepo on RHEL3 to RHEL5 would
not split a userbase and allows to reuse documentation/plugins/scripting.

> Any particular reason you're running createrepo on a system with rpm < 4.2.1?

Being able to use the same version of apt on RHEL2.1, RH7, RH8, RH9, RHEL3,
RHEL4 and RHEL5 using repomd metadata.

Yes, RHEL2.1 is still in support until mid-2009 and some companies may not even
replace those by then. And RH7 is still part of VMware ESX Service Console so
will be around for some more time.

We are not quite rid of RPM < 4.2.1...

Panu: Not encoding the epoch value explicitly means we have to add the "implicit
behavior of rpm >= 4.2.2" to everything that reads the MD. Personally, I don't
see how that's better.

JBJ:

> but script kiddies learn the hard way.

 I can only assume you don't count as you just created the buffer overflows in rpm?
 And with the introductions out of the way...

#1 is not true, AFAICS. No rounding is done anywhere on size values.

#2 is confusing, you admit that the only way we can see that what we downloaded
matches what we wanted is by using a full file checksum ... but you are arguing
that sometimes when we download things we should magically trust the network?

#3 is needed for yum-3.0.1, which right now is still the current version in
RHEL-5/CentOS-5 ... both of which have people working on yum. So a slightly
different situation than dealing with RHL9 which is _long_ unsupported and noone
who cares about it works on yum, AFAICS.

#4 Sorry, you used "efficiency" and "XML" in the same sentence, you lose. If you
want to do this, you are free to propose it though ... I don't see what harm it
would do.

#5 Assuming that sqlite+bzip2 saves you from the resulting XML data explosion, I
don't see another reason not to include that ... or at least create an extra
filelists_stat_info or whatever. Feel free to propose it, if you really care.

> please feel free to claim that I'm confused.

 Ok, you're confused.
 HTH. HAND.

James: Please point out the "buffer overflows" I created so that they can be fixed. There are
certainly older versions of rpm (on RHEL2.1 and RHL7 judging from this bug report) that
are in need of maintenance even if you and Fedora and Red Hat son't think the issuae are worth fixing. 

The underlying topic on bz#444882  is whether a known useful 2 line fix to createrepo.py
should/could be back ported, another failed maintenance task (so far, its promising that
the issue may actually be resolved rather than mocked).

rpmfusion is the only repository I'm aware of that is trying to credibly support the large number of
 currently deployed legacy RH distributions rather than advocating the advantages of the latest
and greatest python release. The rpmfusion effort is worthy of support, not twitchy scarcasm.

Pardon, I meant rpmforge. In this case I am confused.

 Seth added the workarounds in yum for this broken MD, so it should work again
after the first Fed-9 update and maybe Fed-8 if we backrelease it.

I'm just wondering if yum versions which are vailable for Red Hat 9 actually
support that "broken" metadata and if we're not just seeing a problem here just
because someone mistook rh9 for "rawhide 9" - otherwise I don't see why the
original reporter would try to use that on Fedora 9 ...

AFAICS you'd probably still use yum 2.0.x on RH9 (as you would on CentOS 3),
which doesn't use the repomd metadata at all, but still relies on the .hdr stuff
which is also there for that repository.

Just my 2 cents ...