Bug 224544 - find provides should only search in ld directories
find provides should only search in ld directories
Status: CLOSED RAWHIDE
Product: Fedora
Classification: Fedora
Component: rpm (Show other bugs)
rawhide
All Linux
medium Severity low
: ---
: ---
Assigned To: Fedora Packaging Toolset Team
bzcl34nup
: FutureFeature
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2007-01-26 05:01 EST by Patrice Dumas
Modified: 2013-05-20 06:31 EDT (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-05-20 06:31:39 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Patrice Dumas 2007-01-26 05:01:09 EST
Description of problem:

find-provides searches everywhere in the file list, not only 
where ld will search. This leads to bogus provides. It should
only search in /%_lib, %_libdir, and in directories in 
/etc/ld.so.conf and in /etc/ld.so.conf.d/*.conf

Version-Release number of selected component (if applicable):

rpm-build-4.4.2-39.fc7

How reproducible:

See, for example 
rpm -q --provides ghostscript
X11.so  

because of 
/usr/lib/ghostscript/8.15/X11.so


Steps to Reproduce:
1.
2.
3.
  
Actual results:


Expected results:


Additional info:
Comment 1 Jeff Johnson 2007-01-26 08:00:33 EST
Doing
    chmod -x /usr/lib/ghostscript/8.15/X11.so
stops the provides from being generated.

Limiting provide/require extraction by path assumes that
the search path is known on the build machine, which is
not true in general.

WORKSFORME
Comment 2 Jeff Johnson 2007-01-26 08:10:33 EST
Actually, there's another flaw here as well.

For ELF DSO's without DT_SONAME, the name of the file
is used by rpm as a Provides:, mimicking what the ELF linker does.

Adding the foo.so file name as DT_SONAME  functionality to rpmbuild was demanded in order to
support 3rd party packaging that did not correctly link libraries, adding DT_SONAME and
DT_NEEDED as appropriate.

Comment 3 Patrice Dumas 2007-01-26 08:20:04 EST
(In reply to comment #1)
> Doing
>     chmod -x /usr/lib/ghostscript/8.15/X11.so
> stops the provides from being generated.

But then debuginfo aren't generated anymore, if I recall well.

> Limiting provide/require extraction by path assumes that
> the search path is known on the build machine, which is
> not true in general.

In fact it seems to me that in common cases and in recent fedora, 
it could be possible to simply take everything in 
cat /etc/ld.so.conf.d/*
+standard dirs, maybe removing comments. Do you see a reason why 
it could fail?

It is not completely obvious, however, that the search path on
the build machine is relevant for deps generation. It could
be possible to cross-build rpms for different systems, or even 
platforms, for example.

However it seems to me that it could still be reasonable to use 
the path on the build host to autogenerate the provides, and let
the user fiddle with the corner cases, like what is routinely 
done in perl packages.
Comment 4 Patrice Dumas 2007-01-26 08:25:24 EST
(In reply to comment #2)
> Actually, there's another flaw here as well.
> 
> For ELF DSO's without DT_SONAME, the name of the file
> is used by rpm as a Provides:, mimicking what the ELF linker does.

In my personal opinion, the ELF linker shouldn't do that to begin
with.

> Adding the foo.so file name as DT_SONAME  functionality to rpmbuild was
demanded in order to
> support 3rd party packaging that did not correctly link libraries, adding
DT_SONAME and
> DT_NEEDED as appropriate.

That's indeed completely broken, and certainly not relevant for fedora.
Should I fill a separate bug?
Comment 5 Matthias Clasen 2007-01-26 09:51:40 EST
As a data point, one third of all provides in Core is generated by non-library
DSOs, and will never by useful for satisfying requires, and can only introduce
errors in dependency resolution.
Comment 6 Patrice Dumas 2007-01-26 10:08:47 EST
(In reply to comment #5)
> As a data point, one third of all provides in Core is generated by non-library
> DSOs, and will never by useful for satisfying requires, and can only introduce
> errors in dependency resolution.

Given the high number, it is also likely that removing them would 
improve the rpm/yum... efficiency.
Comment 7 Jeff Johnson 2007-01-30 07:21:25 EST
The number of Provides: does not affect performance significantly, bsearch
and hash are used. The number of operations scales from the number of
Requires:, not the number of Provides:.

And tracking all provided file paths is many times larger than the number
of actual Provides:, from DSO's or not.

What errors will be introduced in dependency resolution? How are you determining
"never useful for satisfying requires"?
Comment 8 Jeff Johnson 2007-01-30 07:33:25 EST
Whether using basename as DT_SONAME is "completely broken" or not, it is the
existing ELF linker behavior. All rpm is doing is tracking the existing behavior
through dependencies.

Many large commercial vendor packages are built without DT_SONAMES. Changing Provides:
extraction to not use the basename when DT_SONAME is not present is very
likely to have significant impact.

If debuginfo extraction fails because the executable bit is not set, then that needs
fixing in debuginfo. The convention of using the executable bit to determine whether
dependencies should be extracted is useful.
Comment 9 Patrice Dumas 2007-01-30 07:50:35 EST
(In reply to comment #8)
> Whether using basename as DT_SONAME is "completely broken" or not, it is the
> existing ELF linker behavior. All rpm is doing is tracking the existing behavior
> through dependencies.

Following the ELF linked behavior in rpm is not necessarily a good
thing when it comes to automatic dependencies determination. 
Adding every .so, not only those that have a soname, and not having 
a version greatly augments the chance to have a bogus provide. 
The .so from perl packages, from dlopened plugins are used, with
an important risk that the wrong dependency is used.

> Many large commercial vendor packages are built without DT_SONAMES. Changing
Provides:
> extraction to not use the basename when DT_SONAME is not present is very
> likely to have significant impact.

Do you have an example of packages actually requiring an unversionned
.so? Besides we are here in the scope of fedora, and what is true for rpm
may not be true for rpm in fedora, in fedora unversionned .so requires should
never be used. Outside of fedora it is also bad practice, and I can't see a 
case where it could be useful, ie where it isn't bad packaging.

> If debuginfo extraction fails because the executable bit is not set, then that
needs
> fixing in debuginfo. The convention of using the executable bit to determine
whether
> dependencies should be extracted is useful.

Isn't it also useful to determine whether debuginfo information
should be extracted or not?
Comment 10 Jeff Johnson 2007-01-31 08:46:21 EST
Adding dependencies to packages to prevent ELF linker failures is exactly the raison d'etre for
ELF soname dependencies.

What is a "bogus provide"? You have yet to point out an explicit failure case
from extracting provides from DSO's on non-library paths.

Sure Fedora can change however it wants. I am supplying technical reasons why
the basename of DSO's was added as a Provides: in the first place. There are
Fedora users other than me using out-of-scope Fedora packages that rely on
what you are calling "bogus Provides:"

Look in /usr/lib/rpm/find-debuginfo.sh if you wish to determine whether
doing "chmod -x ..." on DSO's prevents producing -debuginfo packages.
Comment 11 Red Hat Bugzilla 2007-08-21 01:31:18 EDT
User pnasrat@redhat.com's account has been closed
Comment 12 Panu Matilainen 2007-08-22 02:34:19 EDT
Reassigning to owner after bugzilla made a mess, sorry about the noise...
Comment 13 Bug Zapper 2008-04-03 14:57:45 EDT
Based on the date this bug was created, it appears to have been reported
against rawhide during the development of a Fedora release that is no
longer maintained. In order to refocus our efforts as a project we are
flagging all of the open bugs for releases which are no longer
maintained. If this bug remains in NEEDINFO thirty (30) days from now,
we will automatically close it.

If you can reproduce this bug in a maintained Fedora version (7, 8, or
rawhide), please change this bug to the respective version and change
the status to ASSIGNED. (If you're unable to change the bug's version
or status, add a comment to the bug and someone will change it for you.)

Thanks for your help, and we apologize again that we haven't handled
these issues to this point.

The process we're following is outlined here:
http://fedoraproject.org/wiki/BugZappers/F9CleanUp

We will be following the process here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping to ensure this
doesn't happen again.
Comment 14 Patrice Dumas 2008-04-04 04:10:42 EDT
Still present.
Comment 15 Bug Zapper 2008-05-13 22:34:19 EDT
Changing version to '9' as part of upcoming Fedora 9 GA.
More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 16 Hans de Goede 2008-10-15 10:41:24 EDT
Panu,

Given some recent discussion on fedora-devel about problems caused by this misfeature of rpm, I'm really interested in seeing this fixed. As in interested up to the point where I'm willing to spend time writing code.

We've had some discussion about this in Brno during the Fudcon. From what I remember then rpmbuild has both a built in (the default) and an external dependency generator, and the built in is the default.

So if I patch both generators to:
only do the elf .so file Provides generation under %{_lib} and %{libdir}, and also under any dirs listed in any /etc/ld.so.conf.d/* files which are part of the same (sub) package to not break users of that hack.

Would you be willing to take a look at such a patch, and more importantly if its not too ugly, would you be willing to add such a patch to rpmbuild ?
Comment 17 Bug Zapper 2009-06-09 18:25:32 EDT
This message is a reminder that Fedora 9 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 9.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '9'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 9's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 9 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 18 Hans de Goede 2009-06-10 03:27:40 EDT
This is still hapenning in rawhide, so setting version to rawhide.

Panu my help offer from comment #16 still stands.
Comment 19 Fedora Admin XMLRPC Client 2012-04-13 19:12:31 EDT
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.
Comment 20 Fedora Admin XMLRPC Client 2012-04-13 19:14:05 EDT
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.
Comment 21 Panu Matilainen 2013-05-20 06:31:39 EDT
The "unwanted provides from dlopen'ed modules" issue has been addressed in upstream + rawhide now, but differently (filter by dependency name, not path):
http://lists.fedoraproject.org/pipermail/devel/2013-May/182919.htm

Note You need to log in before you can comment on or make changes to this bug.