Bug 478907 - Please remove lead type check again (for srpms only having RPMTAG_SOURCERPM)
Please remove lead type check again (for srpms only having RPMTAG_SOURCERPM)
Product: Fedora
Classification: Fedora
Component: rpm (Show other bugs)
All Linux
low Severity medium
: ---
: ---
Assigned To: Panu Matilainen
Fedora Extras Quality Assurance
Depends On:
  Show dependency treegraph
Reported: 2009-01-05 17:34 EST by Simon
Modified: 2009-02-24 15:52 EST (History)
6 users (show)

See Also:
Fixed In Version: 4.6.0-1.fc10
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2009-02-24 15:52:08 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Simon 2009-01-05 17:34:19 EST
Description of problem:
With git commit 50a4ed783c300a92948f7933a3c726d4bd2d643d (http://rpm.org/
gitweb?p=rpm.git;a=commit;h=50a4ed783c300a92948f7933a3c726d4bd2d643d), the
check to seperate between binary and source rpm was changed heavily:

> Work around broken packages which dont have RPMTAG_SOURCERPM
> - check the lead type and headerIsSource() agree on the type and if not,
>  fix it up based on the information in lead  

Let me remember, that the rpm lead is an abandoned data structure already
since 1997 when looking to "Maximum RPM" e.g. on rpm.org. The section "The
lead: An Abandoned Data Structure" (http://www.rpm.org/max-rpm/s1-rpm-file-
format-rpm-file-format.html) describes that and explains why it shouldn't
be used at all - now we're 12 years later and there's something requiring
now the rpm lead in case if there isn't maybe a rpm lead type written.

You're argumenting in your commit, that there are some binary packages out
in the wild that don't have a RPMTAG_SOURCERPM. I would like to know, how
they have been created and which confusion you've seen there.

If looking to my own issue, your workaround breaks for me usage of latest
OpenPKG source rpm packages which are now treated as binary packages, as
they don't have a rpm lead type - but a RPMTAG_SOURCERPM inside headers as 
required...until that breaking commit I'm hereby referring to.

Oh, to avoid misunderstandings: "The next two bytes (0000) determine what
type of RPM file the file is. There are presently two types defined: Binary 
package file (type = 0000) or Source package file (type = 0001)". I would
say this is what you're now checking with the mentioned git commit and what
breaks usage for me.

Version-Release number of selected component (if applicable):
rpm >= 4.6.0-rc3

How reproducible:
Everytime, just try to install a newer OpenPKG source rpm package on Fedora:
It unluckily will poorly fail, because it is treated wrongly as a binary rpm
package and thus fails to install because of wrong detection.

Expected results:
Please undo git commit 50a4ed783c300a92948f7933a3c726d4bd2d643d or change it
so, that rpm source packages without rpm lead type for source rpm are handled
as source rpm as in former times. You maybe can refer to RPMTAG_SOURCERPM. If
it is set, but no rpm lead type is available, it's still a source rpm. Thanks.
Comment 1 Jeff Johnson 2009-01-06 10:46:11 EST
Note that OpenPKG uses rpm-5.x, which is trying to eliminate
an abandoned data structure 12 years later aggressively (by not populating the lead
of *.src.rpm to demonstrably show that the information is not used).

Re-adding the lead information to rpm-5.x (as used by OpenPKG) to achieve
compatibility with whatever rpm.org rpm-4.6.0 (and rpm-
is very not hard to achieve, and most definitely will be done as necessary.

However, the git check-in in 4.6.0 is doing a test that has never
been in any version of rpm before, and is in fact using info in a lead to test for a
SRPM in a new and different way.

In all cases, for all packages ever produced by RPM itself, the
single best (as in works everywhere and always) test is the non-existence
of RPMTAG_SOURCERPM. All binary packages have RPMTAG_SOURCERPM
for at least a decade, no SRPM includes that tag.

OpenPKG SRPM's also include RPMTAG_SOURCEPACKAGE explicitly in
a SRPM header, which was (until rpm-4.6.0 changed its test) sufficient
to achieve compatiblility with all versions of rpm other than rpm-4.6.0.
Comment 2 Panu Matilainen 2009-01-07 05:44:04 EST
The oddball case of jclasslib package can be found in bug 475214, almost certainly not produced by rpm itself, or at least a modified rpm used. Up to the commit in question, rpm 4.6 would think of it as a source rpm too (as only RPMTAG_SOURCERPM was looked at), but 4.4.x and older do recognize it as a binary package due to looking into lead, although in slightly different way (note the system in bug 475215 was upgraded from F9, the package originally installed with rpm 4.4.x). 

We've a nice little mess here:
- both rpm.org and rpm5.org use mainly RPMTAG_SOURCERPM existence for determining type
- rpm5.org doesn't produce lead info at all
- rpm5.org adds explicit RPMTAG_SOURCEPACKAGE *if* built to do so
- rpm.org retrofits RPMTAG_SOURCEPACKAGE at runtime on packages determined as source, based on RPMTAG_SOURCERPM (non-)existence
- LSB (yeah yeah, I know...) stating that the type of rpm package is determined by the lead, RPMTAG_SOURCERPM is informational, ie "does not contribute to processing of package"

I'm not exactly overjoyed about depending on the info from lead either... but we've a package here that conforms to a written standard (obsolete as it may be), but just not to the unwritten de-facto standard used by rpm itself. And the package has worked up until now.

One possibility that would treat this as the oddball case it is: see if RPMTAG_RPMVERSION is present and if not, only then resort to using the lead for anything. RPMTAG_RPMVERSION is of course another "informational only" tag, but AFAIK all rpm versions always add it, so it's a fairly good indicator whether a package was produced with rpm or with something else.
Comment 3 Jeff Johnson 2009-01-07 10:00:30 EST
RPMTAG_RPMVERSION is not in the lead is most of what I care.

And someone ought to point out the "oddball case" to whomever made the package.
rpm does not install partially damaged packages, and its not too hard to define
lack of RPMTAG_SOURCERPM as a form of partial damage no matter
how the package came to be installed on FC9.
Comment 4 Simon 2009-01-08 15:27:20 EST
You hopefully can understand, that I don't care that much about the developer mess you mentioned. You've made an exception in Fedora for jrpm created rpm packages, so another exception or workaround should not matter that much, right? I would like to see getting rid of this in Fedora rpm without breaking anything anywhere else - thank you.
Comment 5 Robert Scheck 2009-01-16 14:32:46 EST
Since when is Fedora *really* taking care regarding LSB? Didn't somebody of
your colleagues somewhen officially tell, that the LSB is anyway crap? ;-)

Panu, can we move forward with this bug, please? I really would like to see
this regression solved in a non-destroying way making everybody happy.
Comment 6 Robert Scheck 2009-01-24 11:06:31 EST
Panu, can you please show up on this bug report?

Currently it seems to me, that supporting a single jpackage created RPM
package is more important for Fedora rather compatibility with rpm5.org.
Comment 7 R P Herrold 2009-01-24 11:35:21 EST
As to comment 5, and the reference to the dismissal of the LSB by 'spot', and the recent ones as to the FHS, I have worked rather hard to seek to draw Fedora folk into interest in LSB, and to address the criticism of the FHS to make the FHS relevant and able to be responsive to recent criticism.  

I take it from comment that the problem is 'jrpm created rpm' pacages do not implement all the header structures -- Shouldn't this be reassigned to whatever the owning package for the offnder, so that it may be fixed?

If bugzilla and rpm development under Red Hat are bent to serve the 'Balkanizing' political desires of part of Fedora, we are all lost.
Comment 8 Jeff Johnson 2009-01-24 11:42:26 EST
Note that compatibility will be dealt with @rpm5.org if/when rpm-4.6.0 is
"officially" released. There are other incompatibilities in rpm.org's rpm-4.6.0
that are at least as important as discussing a value in an "abandoned" rpmlead
data structure.
Comment 9 Robert Scheck 2009-01-24 11:43:14 EST
Oh right, I'm complaining here about, that it seems to be more important, that 
a single jrpm created RPM package seems to be more important for Fedora rather 
compatibility with rpm5.org. Sorry jpackage guys, I meant jrpm. I got pointed,
that this is a huge difference. So jrpm not jpackage.
Comment 10 Panu Matilainen 2009-01-30 02:19:56 EST
If you folks have finished flaming me for bogus lead in your packages causing problems for the time being, can we try to be constructive for a moment?

I turned the check around a bit, RPMTAG_SOURCEPACKAGE in srpms is sufficient to prevent from being thought as a binary no matter what the lead says which AFAICT takes care of this bug.

Like I said in comment #2 I've no love the lead either, but I do care about packages conforming to widely existing documentation continuing to work. The "weird" jrpm packages do fullfill the widely existing documentation about rpm file format, RPMTAG_SOURCERPM is documented as "information only". Thats the issue to me, not single package being more important than compatibility or lack or thereof with rpm5.org.

This issue does demonstrate the feebleness of checking against non-existence and zeros though. What would avoid this is Yet Another Tag in the header, explicitly stating the type of the package. Eg. RPMTAG_PKGTYPE, values 1 for binary and 2 for source or so and push it to get documented in LSB. When not present, fall back to whatever ping-pong-retrofit-guesswork an implementation chooses to do - such as simply taking whatever the lead says. Wont help with the existing package universe of course but it would avoid this kind of issue from showing up ever again for new packages.
Comment 11 Robert Scheck 2009-01-30 02:34:01 EST
Panu, thank you first for showing up and changing your type check. Do you have
a git commit for me, so that I can test/verify it?

Regarding the idea of e.g. RPMTAG_PKGTYPE: Maybe Jeff can explain how it is 
handled for rpm5.org - maybe you like the idea (or maybe you already know how
rpm5.org handles this, as it's open source, yes).
Comment 12 Panu Matilainen 2009-01-30 02:59:29 EST
Here's the commit from 4.6.x branch: http://rpm.org/gitweb?p=rpm.git;a=commitdiff;h=49c2e8a79d14181f0398f7f4d85bff4d69260f55

Unless something has very recently changed, AFAIK rpm5.org decides simply by (non-)existence of RPMTAG_SOURCERPM (which is what rpm.org did too before all this). Which is feeble as you cannot know by looking at a package whether the tag is left out on purpose or not, which is largely the reason why we have this bug :)
Comment 13 Jeff Johnson 2009-01-30 06:34:59 EST
There are no flames from me here ... but if you insist, I will most
definitely flame you for making RPMTAG_FILEDIGESTALGO a scalar;
you could have *easily* added a new tag number without boogering
up a pre-existing (the work was done 2+ years ago) implementation
just because you can.

Just add RPMTAG_SOURCEPACKAGE to SRPMS during rpmbuild and be done with the issue.

Adding RPMTAG_SOURCEPACKAGE explicitly during build avoids the pokery-jiggery testing
for non-existence of Something Else, which has always been flawed
(but is in fact what has been done by RPM for years).

I see no reason for RPMTAG_PKGTYPE to be added to content. There are
already positive tests for "source" w RPMTAG_SOURCEPACKAGE, and

No matter what identifier is devised, there will always be corner-cases,
like pubkey headers, and jrpm created "buggy" packages, and maliciously
fuzzed header content, etc.

No reliable "standard" exists for types of RPM packages, nor can RPM
packages be adequately semantically typed as "source", "binary", or ...
Consider the kernel-source package which was a "binary" RPM
containing "source" code, for one well known confusion.

Any enumeration of the characteristics of "source" or "binary" RPM
packages quickly identifies the jrpm as neither "source" nor "binary"
RPM package.
Comment 14 Jeff Johnson 2009-01-30 07:00:10 EST
Re "recently changed" in comment #12:

rpm-5.0 was released @rpm5.org explicitly adding RPMTAG_SOURCEPACKAGE to
*.src.rpm headers for all vendors who wished compatibility with rpm-4.x. rpm-5.0
has _ALWAYS_ been that way.
Comment 15 Panu Matilainen 2009-01-30 07:18:14 EST
Jeff, others flamed not you (as is quite obvious by reading the comments), that wasn't addressed to you.

The RPMTAG_FILEDIGESTALGO[S] clash is outside of the scope of this bug but just for the record: yes, it was a dumb thing from me, and is now resolved in rpm.org, 4.6.0 will go out with a different tag for it. Fortunately nobody uses it yet for anything but early testing.
Comment 16 Jeff Johnson 2009-01-30 08:00:47 EST
Cool, one less issue for me to fix. I was really hoping you'ld achieve db-4.7.x, c'est dommage ...

There's still headerGet() with different arguments, I did squat that ELF symbol first,
but I'l leave that flammage for later. Deliberately poisoning API/ABI linkage just
means "Two men in, one man out." I most definitely plan my exit by walking out ...
Comment 17 Fedora Update System 2009-02-07 17:22:45 EST
rpm-4.6.0-1.fc10 has been pushed to the Fedora 10 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update rpm'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/F10/FEDORA-2009-1478
Comment 18 Robert Scheck 2009-02-10 06:09:14 EST
rpm-4.6.0-1 seems to solve my issue so far - thank you.
Comment 19 Fedora Update System 2009-02-24 15:51:27 EST
rpm-4.6.0-1.fc10 has been pushed to the Fedora 10 stable repository.  If problems still persist, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.