Bug 395961 - find-provides/find-requires miss important dependencies (SONAMEs)
find-provides/find-requires miss important dependencies (SONAMEs)
Status: CLOSED WONTFIX
Product: Fedora
Classification: Fedora
Component: rpm (Show other bugs)
11
All Linux
low Severity high
: ---
: ---
Assigned To: Panu Matilainen
Fedora Extras Quality Assurance
:
: 487958 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2007-11-22 12:53 EST by Eddy Petrișor
Modified: 2014-08-12 22:52 EDT (History)
7 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2010-06-28 06:29:35 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
package using a dummy wrapper around find-* (622.44 KB, application/x-rpm)
2007-11-22 12:53 EST, Eddy Petrișor
no flags Details
package using the internal generator (622.20 KB, application/x-rpm)
2007-11-22 12:55 EST, Eddy Petrișor
no flags Details

  None (edit)
Description Eddy Petrișor 2007-11-22 12:53:25 EST
Description of problem:
find-requires and find-provides do not even come close to find the right
reqs/provs as the internal dependency generator. This is problematic when
trying to filter out reqs/provs since some of them referring to SONAMEs are
missing altogether.

More than that, there is no indication in rpm --showrc that, by default, RPM
really uses a different generator since the __find_provides and
__find_requires macros are defined to point to
/usr/lib/rpm/find-{requires,provides}; This lack of hints and information
made me waste 4 days trying to understand why the requires and provides change
even when I use a dummy wrapper around the find-* scripts.


Also, there is no clear way to disable the internal generator only for one of
the two, since there is a common switch for both.


Version-Release number of selected component (if applicable):
Any rpm version including RPM version 4.4.2.2 (and probably later, if any). I
have tested this (I know *now* this sounds stupid) on rpm starting from rh9,
fc4, fc7 and finally fc8.


How reproducible:
Rebuild the example src.rpm packages (one doesn't disable the internal
generator, the other just wraps find-requires). Note that both find-requires
and find-provides are broken.


Steps to Reproduce:
1.download the example src.rpms
2.rebuild them both (they have different releases)
3.compare the resulted requires and provides for
  vmailmgr-daemon-0.97-2.i386.rpm (has dummy find-requires)
  and vmailmgr-daemon-0.97-3.i386.rpm (uses internal dep gen)

You will see these differences:

diff -ruN 2/PROVIDES 3/PROVIDES
--- 2/PROVIDES  2007-11-22 19:33:41.000000000 +0200
+++ 3/PROVIDES  2007-11-22 19:33:35.000000000 +0200
@@ -1 +1,2 @@
+config(vmailmgr-daemon)
 vmailmgr-daemon
diff -ruN 2/REQUIRENAME 3/REQUIRENAME
--- 2/REQUIRENAME       2007-11-22 19:33:20.000000000 +0200
+++ 3/REQUIRENAME       2007-11-22 19:33:25.000000000 +0200
@@ -1,4 +1,5 @@
 /bin/sh
+config(vmailmgr-daemon) = 0.97-3
 libcrypt.so.1
 libcrypt.so.1(GLIBC_2.0)
 libc.so.6
@@ -8,8 +9,10 @@
 libgcc_s.so.1
 libm.so.6
 libstdc++.so.6
+libstdc++.so.6(CXXABI_1.3)
 libstdc++.so.6(GLIBCXX_3.4)
 rpmlib(CompressedFileNames) <= 3.0.4-1
 rpmlib(PayloadFilesHavePrefix) <= 4.0-1
+rtld(GNU_HASH)
 supervise-scripts >= 2.2
 ucspi-unix


  
Actual results:

Not the same results.

bash-3.2# echo vmailmgrd | /usr/lib/rpm/find-requires
libcrypt.so.1
libcrypt.so.1(GLIBC_2.0)
libc.so.6
libc.so.6(GLIBC_2.0)
libc.so.6(GLIBC_2.1.3)
libc.so.6(GLIBC_2.3)
libgcc_s.so.1
libm.so.6
libstdc++.so.6
libstdc++.so.6(GLIBCXX_3.4)


Expected results:

Should be the same as the internal generator:

bash-3.2# /usr/lib/rpm/rpmdeps -R vmailmgrd
libc.so.6
libc.so.6(GLIBC_2.0)
libc.so.6(GLIBC_2.1.3)
libc.so.6(GLIBC_2.3)
libcrypt.so.1
libcrypt.so.1(GLIBC_2.0)
libgcc_s.so.1
libm.so.6
libstdc++.so.6
libstdc++.so.6(CXXABI_1.3)
libstdc++.so.6(GLIBCXX_3.4)
rtld(GNU_HASH)


Additional info:

This is a duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=134374, but
the resons for closing the bug was not taking into account the SONAME issues I
am rasing here.


I think there can be more than one way to fix this:
- make the find-* scripts wrappers over the internal dependency generator
  --- broken packages will not know they need a rebuild against the new find-*
  +++ non-intusive for existent packages
  +++ keeps compatbility
  +++ easy to implement in a shell script :
      ( while read FILE ; do /usr/lib/rpm/rpmdeps -R $I ; done | sort -u )
      or
      ( while read FILE ; do /usr/lib/rpm/rpmdeps -P $I ; done | sort -u )
      depending on the find-* name (they can even be the same file and pass
      the appropriate parameter depending on what `basename $0` is


- drop find-* scripts entirely
  --- many packages will break
  --- needs a lot of work for packages which rely on find-* scripts
  --- hell freezes
  +++ packages which need to be rebuilt will be found immediately and this
      will give the occasion to people to actually become aware of the issue
Comment 1 Eddy Petrișor 2007-11-22 12:53:26 EST
Created attachment 267141 [details]
package using a dummy wrapper around find-*
Comment 2 Eddy Petrișor 2007-11-22 12:55:35 EST
Created attachment 267151 [details]
package using the internal generator

This src.rpm package uses the internal generator
Comment 3 Eddy Petrișor 2007-11-22 14:59:19 EST
(In reply to comment #0)
>   +++ easy to implement in a shell script :
>       ( while read FILE ; do /usr/lib/rpm/rpmdeps -R $I ; done | sort -u )
>       or
>       ( while read FILE ; do /usr/lib/rpm/rpmdeps -P $I ; done | sort -u )

of course, these should read:

( while read FILE ; do /usr/lib/rpm/rpmdeps -R $FILE ; done | dort -u )
and respectively,
( while read FILE ; do /usr/lib/rpm/rpmdeps -P $FILE ; done | dort -u )

Comment 4 Jeff Johnson 2007-11-24 23:41:41 EST
All packages in RHL 9 were built with the internal dependency generator.

At that point find-provides and find-requires were end-of-life.

Sad that you wasted 4 days discovering what is already well known.
Comment 5 Jeff Johnson 2007-11-24 23:47:29 EST
FWIW, rpmdeps was written to be a drop-in replacement for find-requires and find-provides
if necessary.

There are other reasons to use *only* the internal dependency generator, not the least of which
is that it is impossible to produce a multilib ready package using external scripts or helpers.
Comment 6 Eddy Petrișor 2007-11-26 06:10:41 EST
(In reply to comment #4)
> All packages in RHL 9 were built with the internal dependency generator.
> 
> At that point find-provides and find-requires were end-of-life.

For me end-of-life would have meant that they shouldn't have been *provided* at all.

> Sad that you wasted 4 days discovering what is already well known.

I haven't seen not even a warning in any of the docs that I have found about RPM
packaging. As I said, there are absolutely 0 hints/pointers/notes about this
issue and, even worse, those scripts are still part of the rpm package.

More than that, I think you are exagerating when you're saying this is a well
known issue since I have asked on #fedora-devel about this and not even one soul
was able to put me on the right track.
Comment 7 Eddy Petrișor 2007-11-26 06:16:05 EST
(In reply to comment #5)
> FWIW, rpmdeps was written to be a drop-in replacement for find-requires and
find-provides
> if necessary.

drop-in replacement means that there's absolutely no change in the interface and
all one has to do to use "the other" is to make sure is called.

rpmdeps is *not* a drop-in replacement of find-* scripts since they have
different interfaces.

> There are other reasons to use *only* the internal dependency generator, not
the least of which
> is that it is impossible to produce a multilib ready package using external
scripts or helpers.

Is that still a problem if rpmdeps is wrapped like I proposed?
Since rpm does not have a feature such as dpkg's suggests or recommends, I am
forced to remove a requires from one of the packages I am building since only in
seldom cases the component needing the requirement is used.
Comment 8 Jeff Johnson 2007-11-28 10:34:02 EST
rpmdeps certainly is a drop in replacement when rpm is configured correctly.

The API is exactly the same, file paths gozzinta on stdin, dependencies comesouta on stdout,
with rpmdeps using CLI switches to determine whether Provides: or Requires: is desired,
and macros used to configure rpm as always.

If up to me, I would have ripped find-provides long ago. Lusers want the old crapola,
which has several flaws, like the one pointed out here with missing
    Requires: rtld(GNU_HASH)
Other flaws include problems with using ldd or objdump in buildroot's being
influenced by the system build environment causing the scriptie scraping to fail
in odd ways.

And rpm certainly has Suggests:, just not the rpm you happen to be using.

And yes, its impossible to use an external helper to generate multilib ready
packages exactly _BECAUSE_ the API to the external helpers did not change.

All files are attached to their dependencies when using the internal dependency
generator, and that required a complex construction with multiply linked sorted structures
that cannot be created from the Good Old Stuff spewed by find-requires.
Comment 9 Jeff Johnson 2007-11-28 10:44:20 EST
hint to fix your filtering problem:
   Use the internal dependency generator.
   Run rpm -qp --filerequire  pkgwithbaddeep*.rpm | grep baddep
   Locate the path(s) of the files that are causing the bad dependency to be generated.
   chmod -x each of those file paths at the end of %install.

If you absoultely _MUST_ package executable files with missing dependencies, then
    Add %attr{755,-,-} /path/to/file/with/bad/dependency explicitly in %files.

You're welcome.
Comment 10 Eddy Petrișor 2007-11-30 09:54:51 EST
(In reply to comment #9)
> hint to fix your filtering problem:
>    Use the internal dependency generator.

I have already suggested[1] and gave a patch (see comment 3) for how I would
consider the problem fixed for everybody, not just for me and just for one case.

>    Run rpm -qp --filerequire  pkgwithbaddeep*.rpm | grep baddep
>    Locate the path(s) of the files that are causing the bad dependency to be
generated.
>    chmod -x each of those file paths at the end of %install.

not possible since there's one file that has, among others, a possible
dependency, but is not used in most cases.

> If you absoultely _MUST_ package executable files with missing dependencies, then
>     Add %attr{755,-,-} /path/to/file/with/bad/dependency explicitly in %files.

Again not possible, the file has other deps, too.


Please, would you please answer if replacing find-* scripts with the wrappers i
suggested in #3 is not an acceptable solution, and if so, why?



[1] by changing the find-* scripts to wrap the internal dependency generator -
see comment 3: https://bugzilla.redhat.com/show_bug.cgi?id=395961#c3
Comment 11 Jeff Johnson 2007-12-01 08:06:14 EST
Replacing the find-scripts with is not an acceptable solution (although its likely
a good idea in certain circumstances because rpmdeps will find the Requires: rtld(GNU_HASH) and other additional dependencies that the older scripts won't find).

Why? The internal dependency generator creates the necessary interlinked tag data structures for the 
following options
    --fileclass
    --fileprovide
    --filerequire
to attach dependencies (and classifications) to files. 
Comment 12 Eddy Petrișor 2007-12-03 05:24:44 EST
(In reply to comment #11)
> Replacing the find-scripts with is not an acceptable solution (although its likely
> a good idea in certain circumstances because rpmdeps will find the Requires:
rtld(GNU_HASH) and other additional dependencies that the older scripts won't find).
> 
> Why? The internal dependency generator creates the necessary interlinked tag
data structures for the 
> following options
>     --fileclass
>     --fileprovide
>     --filerequire
> to attach dependencies (and classifications) to files. 

And? What is wrong wrt to the current find-* results? Does that make the
situation any worse than it is now? I don't think it does, does it?

If someone is willing to use find-* scripts, I don't think they care less which
files pulls in which dep and they really don't have that information with the
current find-* scripts. Am I missing something?

Please, I understand that in most cases the internal dependency generator is the
way, but I think you're stumbling in details which are irrelevant for *this*
scenario. Also, please, let's stay focused on the subject at hand and compare
the current situation of find-* scripts and possible future situations.
Comment 13 Panu Matilainen 2007-12-03 06:21:21 EST
To me it seems the actual subject at hand was lost in the implementation details
to begin with: what you're really after is dependency filtering in rpm, not
"replace find-requires with rpmdeps"
Comment 14 Eddy Petrișor 2007-12-03 14:35:01 EST
(In reply to comment #13)
> To me it seems the actual subject at hand was lost in the implementation details
> to begin with: what you're really after is dependency filtering in rpm, not
> "replace find-requires with rpmdeps"

Yes, but that was the only (and probably the easiest) proposal for the
implementation.
Comment 15 Bug Zapper 2008-11-26 03:39:14 EST
This message is a reminder that Fedora 8 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 8.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '8'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 8's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 8 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 16 Panu Matilainen 2009-03-02 02:36:16 EST
*** Bug 487958 has been marked as a duplicate of this bug. ***
Comment 17 Chris Weyl 2009-03-02 10:45:59 EST
Not to seem impatient, but is this actually set to be resolved any time soon?

Alternatively, is there some way to sort/filter the internally generated provides, perhaps with a lua scriptlet or somesuch?  (still trying to get my head around what one can/cannot do via lua)

Thanks :)
Comment 18 Bug Zapper 2009-06-09 05:22:10 EDT
This bug appears to have been reported against 'rawhide' during the Fedora 11 development cycle.
Changing version to '11'.

More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 19 Bug Zapper 2010-04-27 07:50:04 EDT
This message is a reminder that Fedora 11 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 11.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '11'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 11's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 11 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 20 Bug Zapper 2010-06-28 06:29:35 EDT
Fedora 11 changed to end-of-life (EOL) status on 2010-06-25. Fedora 11 is 
no longer maintained, which means that it will not receive any further 
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of 
Fedora please feel free to reopen this bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.
Comment 21 Craig Ringer 2014-08-12 22:50:27 EDT
"At that point find-provides and find-requires were end-of-life. Sad that you wasted 4 days discovering what is already well known."

Well, they're not *documented as end-of-life* anywhere useful.

They're still in rpm:

$ rpm --eval '%__find_provides'
/usr/lib/rpm/redhat/find-provides
$ rpm --eval '%__find_requires'
/usr/lib/rpm/redhat/find-requires
$ rpm --version
RPM version 4.11.2

don't have any comments warning about wrapping them, don't emit any warnings, aren't complained about by rpmlint, etc.

IIRC they're also still discussed on 

http://www.rpm.org/wiki/PackagerDocs/Dependencies 

and even in:

http://docs.fedoraproject.org/en-US/Fedora_Draft_Documentation/0.1/html/RPM_Guide/ch-packaging-guidelines.html#id670777

... and the latter even discusses how to override them.

It looks like this changed recently, but after the Fedora 20 release:

http://pkgs.fedoraproject.org/cgit/rpm.git/commit/?id=7aa67a4

so hardly "already well known" outside the core RPM hackers and maybe fedora packagers community.
Comment 22 Craig Ringer 2014-08-12 22:52:12 EDT
... so this still applies to Fedora 20.

Note You need to log in before you can comment on or make changes to this bug.