Bug 924938 - wrong provides in latest build
Summary: wrong provides in latest build
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: perl
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Petr Pisar
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 924882 925918 925969 927014 927170 (view as bug list)
Depends On:
Blocks: 914215 925121 925761 926037 926996
TreeView+ depends on / blocked
 
Reported: 2013-03-22 21:19 UTC by Thomas Moschny
Modified: 2013-04-27 00:10 UTC (History)
30 users (show)

Fixed In Version: perl-5.16.3-266.fc20
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2013-03-25 15:18:03 UTC


Attachments (Terms of Use)

Description Thomas Moschny 2013-03-22 21:19:35 UTC
perl-5.16.3-265.fc20 provides perl(Carp), but does not contain Carp.pm. This causes perl-XML-Parser to fail.

Comment 1 Vít Ondruch 2013-03-22 23:11:35 UTC
What is worser, it breaks autoconf:

http://koji.fedoraproject.org/koji/watchlogs?taskID=5158977

Comment 2 Jerry James 2013-03-23 02:27:52 UTC
And it also breaks makeinfo.

Comment 3 Jerry James 2013-03-23 02:28:25 UTC
*** Bug 925918 has been marked as a duplicate of this bug. ***

Comment 4 Mamoru TASAKA 2013-03-23 06:16:06 UTC
With rebuilding perl-5.16.3-265 on f19, then

$ rpmdiff --ignore T ./perl-5.16.3-265.fc19.i686.rpm ./perl-5.16.3-265.fc20.i686.rpm 2>&1 | tee 30.txt
removed     REQUIRES perl-libs = 4:5.16.3-265.fc19
added       REQUIRES perl-libs = 4:5.16.3-265.fc20
removed     PROVIDES perl(x86-32) = 4:5.16.3-265.fc19
added       PROVIDES perl(Carp)  
added       PROVIDES perl(DB) = 1.37
added       PROVIDES perl(DB::fake)  
added       PROVIDES perl(bytes)  
added       PROVIDES perl(utf8)  
added       PROVIDES perl(x86-32) = 4:5.16.3-265.fc20
S.5........ /usr/bin/a2p
S.5........ /usr/bin/perl
S.5........ /usr/bin/perl5.16.3
..5........ /usr/bin/perlbug
..5........ /usr/bin/perlthanks
S.5........ /usr/lib/perl5/Config_heavy.pl
S.5........ /usr/lib/perl5/_h2ph_pre.ph
..5........ /usr/lib/perl5/auto/B/B.so
S.5........ /usr/lib/perl5/auto/Devel/PPPort/PPPort.so
..5........ /usr/lib/perl5/auto/Devel/Peek/Peek.so
..5........ /usr/lib/perl5/auto/Fcntl/Fcntl.so
..5........ /usr/lib/perl5/auto/File/Glob/Glob.so
..5........ /usr/lib/perl5/auto/GDBM_File/GDBM_File.so
..5........ /usr/lib/perl5/auto/Hash/Util/FieldHash/FieldHash.so
S.5........ /usr/lib/perl5/auto/Hash/Util/Util.so
..5........ /usr/lib/perl5/auto/I18N/Langinfo/Langinfo.so
..5........ /usr/lib/perl5/auto/IO/IO.so
S.5........ /usr/lib/perl5/auto/IPC/SysV/SysV.so
..5........ /usr/lib/perl5/auto/MIME/Base64/Base64.so
..5........ /usr/lib/perl5/auto/Math/BigInt/FastCalc/FastCalc.so
..5........ /usr/lib/perl5/auto/NDBM_File/NDBM_File.so
..5........ /usr/lib/perl5/auto/ODBM_File/ODBM_File.so
..5........ /usr/lib/perl5/auto/Opcode/Opcode.so
..5........ /usr/lib/perl5/auto/POSIX/POSIX.so
..5........ /usr/lib/perl5/auto/PerlIO/encoding/encoding.so
..5........ /usr/lib/perl5/auto/PerlIO/mmap/mmap.so
..5........ /usr/lib/perl5/auto/PerlIO/scalar/scalar.so
..5........ /usr/lib/perl5/auto/PerlIO/via/via.so
..5........ /usr/lib/perl5/auto/SDBM_File/SDBM_File.so
..5........ /usr/lib/perl5/auto/Storable/Storable.so
..5........ /usr/lib/perl5/auto/Sys/Hostname/Hostname.so
S.5........ /usr/lib/perl5/auto/Sys/Syslog/Syslog.so
..5........ /usr/lib/perl5/auto/Tie/Hash/NamedCapture/NamedCapture.so
S.5........ /usr/lib/perl5/auto/Time/HiRes/HiRes.so
..5........ /usr/lib/perl5/auto/Unicode/Collate/Collate.so
S.5........ /usr/lib/perl5/auto/Unicode/Normalize/Normalize.so
..5........ /usr/lib/perl5/auto/arybase/arybase.so
..5........ /usr/lib/perl5/auto/attributes/attributes.so
..5........ /usr/lib/perl5/auto/mro/mro.so
..5........ /usr/lib/perl5/auto/re/re.so

So something changed outside of perl.

Comment 5 Remi Collet 2013-03-23 06:28:28 UTC
*** Bug 925969 has been marked as a duplicate of this bug. ***

Comment 6 Mamoru TASAKA 2013-03-23 06:35:33 UTC
Also, perl-5.16.3-264.fc20 does not provide perl(Carp), however rebuilding perl-5.16.3-264.fc20 with _current_ f20 now provides perl(Carp).

Comment 7 Yanko Kaneti 2013-03-23 08:12:44 UTC
It seems to be an accidental %if/%endif breakage inroduced here
http://pkgs.fedoraproject.org/cgit/perl.git/commit/?id=b4857c8d7ca4f03d3bdd0e9fd885a5e8bdc1f96b

Comment 8 Yanko Kaneti 2013-03-23 10:22:45 UTC
Ok, sorry for the noise. This time I actually tested this and its the rawhide change from file-5.11-9 to file-5.14-1 that causes the change in provides

Comment 9 Thomas Moschny 2013-03-23 13:29:59 UTC
*** Bug 924882 has been marked as a duplicate of this bug. ***

Comment 10 Mamoru TASAKA 2013-03-23 17:04:26 UTC
Well, 
 - with first trying rpmbuild -bc perl-5.16.3-265.fc20 with file-5.11-9
 - and rpmbuild -bi --short-circuit with the same file-5.11-9 does not show
   perl(Carp)
 - however now after upgrading file to 5.14-1 and rpmbuild -bi --short-circuit now adds perl(Carp)

so definitely related to file rpm change. And "rpm -qlp file-5.16.3-265.fc20.rpm | grep -v /usr/share/doc | /usr/lib/rpm/find-provides" does not show perl(Carp) provides, so this seems related to rpm internal dependency generator.

file maintainer, any idea?

Comment 11 Thomas Moschny 2013-03-23 17:29:12 UTC
Imho the file package is not blame here. 5.11 misdetected Carp.pm as an AWK script, while 5.14 correctly says it is a Perl script.

So what happens iiuc is (that despite the %exclude) RPMs internal dependency generator now considers Carp.pm for Perl provides (which it did not before), and so the perl(Carp) provides gets into the final RPM (and also some others).

Or to say it differently: The bug fix in the file package uncovered a bug in the perl specfile, and some additional filters need to be added.

As a side note, there could be a bug in the RPM dependency generator, which should probably ignore %excluded files.

Comment 12 Kevin Fenzi 2013-03-23 21:01:21 UTC
This simple patch seems to fix it here: 

diff --git a/perl.spec b/perl.spec
index 2db2cc9..16b7cb7 100644
--- a/perl.spec
+++ b/perl.spec
@@ -14,7 +14,7 @@
 # intentionally (unversioned perl(DB) is removed and versioned one is kept)
 %global __provides_exclude_from .*/auto/.*\\.so$|.*/%{perl_archlib}/.*\\.so$|%{_docdir}
 %global __requires_exclude_from %{_docdir}
-%global __provides_exclude perl\\((VMS|Win32|BSD::|DB\\)$)
+%global __provides_exclude perl\\((Carp|VMS|Win32|BSD::|DB\\)$)
 # unicore::Name - it's needed by perl, maybe problem of rpm
 # FCGI is external dependency after install of perl-CGI, remove it during RC releases
 %global __requires_exclude perl\\((VMS|BSD::|Win32|Tk|Mac::|Your::Module::Here|unicore::Name|FCGI)
@@ -29,7 +29,7 @@
 Name:           perl
 Version:        %{perl_version}
 # release number must be even higher, because dual-lived modules will be broken otherwise
-Release:        265%{?dist}
+Release:        266%{?dist}
 Epoch:          %{perl_epoch}
 Summary:        Practical Extraction and Report Language
 Group:          Development/Languages
@@ -3327,6 +3327,9 @@ sed \
 
 # Old changelog entries are preserved in CVS.
 %changelog
+* Sat Mar 23 2013 Kevin Fenzi <kevin@scrye.com> - 4:5.16.3-266
+- No longer provide perl(Carp). (bug #924938)
+
 * Fri Mar 22 2013 Petr Pisar <ppisar@redhat.com> - 4:5.16.3-265
 - Conflict perl-autodie with older perl (bug #911226)
 - Sub-package Env (bug #924619)

Scratch build at: http://koji.fedoraproject.org/koji/taskinfo?taskID=5162650

Can folks confirm?

Comment 13 Jeff Makey 2013-03-24 01:07:56 UTC
perl-5.16.3-266.fc20.x86_64.rpm from the scratch build in Comment 12 requires but does not provide perl-Carp, so the latter is automatically installed by yum along with the former.  This should solve the problems raised in this bug.

There may be similar problems for the modules identified by this command sequence:

 (rpm -q --requires perl ; rpm -q --provides perl) | sort | uniq -d

with the new RPM its output is this:

perl(Config)
perl(DynaLoader)
perl(Math::BigInt)
perl(Tie::Hash)
perl(bytes)
perl(charnames)
perl(utf8)

The list included perl(Carp) with the previous RPM.

Comment 14 Ralf Corsepius 2013-03-24 16:55:19 UTC
(In reply to comment #12)
> This simple patch seems to fix it here: 
> 
> diff --git a/perl.spec b/perl.spec
> index 2db2cc9..16b7cb7 100644
> --- a/perl.spec
> +++ b/perl.spec
> @@ -14,7 +14,7 @@
>  # intentionally (unversioned perl(DB) is removed and versioned one is kept)
>  %global __provides_exclude_from
> .*/auto/.*\\.so$|.*/%{perl_archlib}/.*\\.so$|%{_docdir}
>  %global __requires_exclude_from %{_docdir}
> -%global __provides_exclude perl\\((VMS|Win32|BSD::|DB\\)$)
> +%global __provides_exclude perl\\((Carp|VMS|Win32|BSD::|DB\\)$)
>  # unicore::Name - it's needed by perl, maybe problem of rpm

> Can folks confirm?

Well, to me, this patch is more a cludge to dampen the issue than an actual fix.

I would rather see the cause (according to what other say a bug in file) fixed instead of adding cludges, because I'd expect the "actual bug" will trigger further similar issues elsewhere.

Comment 15 Thomas Moschny 2013-03-24 17:31:38 UTC
(In reply to comment #14)
> I would rather see the cause (according to what other say a bug in file)
> fixed instead of adding cludges, because I'd expect the "actual bug" will
> trigger further similar issues elsewhere.

Again, I don't think it is a bug in file. There used to be, and that was fixed, causing a bug here (perl specfile and/or rpm) to be uncovered now.

Comment 16 Paul Howarth 2013-03-24 20:34:33 UTC
*** Bug 927014 has been marked as a duplicate of this bug. ***

Comment 17 Paul Howarth 2013-03-24 20:42:15 UTC
It seems to me that rpm is generating requires/provides using files that have been %exclude-d, and that is the underlying cause of the problem.

Comment 18 Ralf Corsepius 2013-03-25 02:25:57 UTC
(In reply to comment #15)
> (In reply to comment #14)
> > I would rather see the cause (according to what other say a bug in file)
> > fixed instead of adding cludges, because I'd expect the "actual bug" will
> > trigger further similar issues elsewhere.
> 
> Again, I don't think it is a bug in file. There used to be, and that was
> fixed, causing a bug here (perl specfile and/or rpm) to be uncovered now.

I didn't say it's a bug in file, I am only saying this isn't a bug in perl.spec but is a bug elsewhere and mentioned "file", because others pointed at it. I don't know the actual cause.

FWIW: Rebuilding older perl-releases with current f20 receives the same bogus provides perl(Carp).

Comment 19 Jan Kaluža 2013-03-25 07:44:14 UTC
(In reply to comment #10)
> file maintainer, any idea?

File in rawhide should detect Perl files better. I've run my regression tests ( https://fedorahosted.org/file-tests/ ) and saw only improvements between 5.11 and 5.14.

Some Perl files previously detect as "awk script, ASCII text" are now properly detected as "Perl5 module source, ASCII text".

I have very limited understanding of interaction between File and rpmbuild. From what I know, rpmbuild checks all files using the File tool and try to match File output according to regexps in /usr/lib/rpm/fileattrs (I'm testing that matching in my test-suite too).

What happened here is that File probably (I don't have the old File installed right now to prove that) started reporting "/usr/share/perl5/perl5db.pl" as Perl script. Next thing is speculation, because I don't what exactly rpmbuild does when it finds some Perl file using File tool, but I think it probably checks for "package X" in that Perl File. And it finds this in our file:

./usr/share/perl5/perl5db.pl:        package Carp;    # Do not include us in the list

That could explain why you see that particular Provides there. Please try to remove that file, do scratch-build and test the Provides to prove this theory. If it's true, I think it's rpmbuild bug.

Comment 20 Mattias Ellert 2013-03-25 08:19:25 UTC
The bogus provides are not coming for the %excluded Carp.pm, it comes form /usr/share/perl5/perl5db.pl - and it is not the only one:

$ /usr/lib/rpm/perl.prov /usr/share/perl5/perl5db.pl
perl(Carp)
perl(DB) = 1.37
perl(DB::fake)

Some of the other extra provides are form other *.pl files.

My suggestion would be to add

.*/%{perl_archlib}/.*\\.pl$  and  .*/%{perl_privlib}/.*\\.pl$

to __provides_exclude_from

Comment 21 Vít Ondruch 2013-03-25 08:31:31 UTC
So how about untag the perl -265 as a interim solution, until the proper solution will be figured out?

Comment 22 Petr Pisar 2013-03-25 08:47:37 UTC
Untagged, please wait for build root rotation:

$ koji list-tag-history --build=perl-5.16.3-265.fc20
Fri Mar 22 16:48:47 2013: perl-5.16.3-265.fc20 tagged into f20 by ppisar
Mon Mar 25 09:42:13 2013: perl-5.16.3-265.fc20 untagged from f20 by ppisar

This issue has been caused by too optimistic dependency generator that started to get some new files due to improved `file' utility.

Difference between 264 and 265 build roots:

Removed packages:
        file-5.11-9.fc19.x86_64
        file-libs-5.11-9.fc19.x86_64
        libblkid-2.22.2-6.fc19.x86_64
        libmount-2.22.2-6.fc19.x86_64
        libuuid-2.22.2-6.fc19.x86_64
        perl-5.16.3-263.fc20.x86_64
        perl-Pod-Escapes-1.04-263.fc20.noarch
        perl-libs-5.16.3-263.fc20.x86_64
        perl-macros-5.16.3-263.fc20.x86_64
        util-linux-2.22.2-6.fc19.x86_64
Added packages:
        file-5.14-1.fc20.x86_64
        file-libs-5.14-1.fc20.x86_64
        libblkid-2.23-0.1.fc20.x86_64
        libmount-2.23-0.1.fc20.x86_64
        libuser-0.58-2.fc19.x86_64
        libuuid-2.23-0.1.fc20.x86_64
        perl-5.16.3-264.fc20.x86_64
        perl-Pod-Escapes-1.04-264.fc20.noarch
        perl-constant-1.27-1.fc20.noarch
        perl-libs-5.16.3-264.fc20.x86_64
        perl-macros-5.16.3-264.fc20.x86_64
        util-linux-2.23-0.1.fc20.x86_64

It will be fixed in way suggested by Mattias Ellert (excluding some perl.spec files from dependency generator scanning).

Comment 23 Ralf Corsepius 2013-03-25 08:50:17 UTC
(In reply to comment #20)
> The bogus provides [...] it comes form
> /usr/share/perl5/perl5db.pl - and it is not the only one:
Confirmed.

> Some of the other extra provides are form other *.pl files.
> 
> My suggestion would be to add
> 
> .*/%{perl_archlib}/.*\\.pl$  and  .*/%{perl_privlib}/.*\\.pl$
> 
> to __provides_exclude_from

May-be I am missing something and am going to say something stupid, but IMO, "Provide: perl(...)" should only be provided by *.pm's and not by *.pl's. 
This would mean all *.pl's should be excluded from any perl()-provides processing in rpm, in general.

Unfortunately file seems to have severe problems ins destinguishing "perl scripts" from "perl modules":

F20:
# file /usr/bin/automake
/usr/bin/automake: Perl5 module source, ASCII text
# file perl-5.16.3/lib/perl5db.pl
perl-5.16.3/lib/perl5db.pl: Perl5 module source, ASCII text

F18:
# file /usr/bin/automake
/usr/bin/automake: Perl script, ASCII text executable
# file perl-5.16.3/lib/perl5db.pl
perl-5.16.3/lib/perl5db.pl: awk script, ASCII text

Both results are wrong, but differently wrong ;)

Comment 24 Jan Kaluža 2013-03-25 09:16:05 UTC
It's still the same problem. File output is unreliable and I think it should not be used for use-cases where you really want to be sure. It will always detects some non-perl files as perl files and some perl files as non-perl files. For example, if you have Perl code in comment in C++ code, it will detect it as Perl. 

Currently Perl is detected using following two patterns: https://github.com/glensc/file/blob/master/magic/Magdir/perl#L29 . I don't know Perl so well to be able to write some regexp to detect typical Perl constructs which are not used by another languages.

If you think you know how to significantly improve Perl detection while not breaking detection of other formants using what "man magic" provides, feel free to open bug with patch.

What I could do is to increase the strength of "shebang" patterns, so it would detect "/usr/bin/automake" as "Perl script" again. The question is, what is the difference between Perl Module and Perl Script except the possible extension? Can Perl file with shebang contain Perl Module? If yes, we should detect it as Perl Module if there's "package", otherwise we will not set Provides right again.

Comment 25 Remi Collet 2013-03-25 09:35:15 UTC
*** Bug 927170 has been marked as a duplicate of this bug. ***

Comment 26 Ralf Corsepius 2013-03-25 10:17:18 UTC
(In reply to comment #24)
> It's still the same problem. File output is unreliable and I think it should
> not be used for use-cases where you really want to be sure.

Agreed. Rpm's file-type detection needs to be a multi-staged, heuristical process with (IMO) file being the last resort, in cases all else fail.

> What I could do is to increase the strength of "shebang" patterns, so it
> would detect "/usr/bin/automake" as "Perl script" again.
IMO, this would be a step into the correct direction.

> The question is,
> what is the difference between Perl Module and Perl Script except the
> possible extension?
The file name extension is the #1 distinction criterion. Additional ones are  the shebang and yet another one is "executable-permissions".

However, there is a difference between the general case of guessing on a file's type and on processing a specific file in context of rpm and perl-modules (c.f. the next sentence)

> Can Perl file with shebang contain Perl Module?
rpm-relevant perl-modules *must* be on the search-path provided through 
"@INC" (cf. perl -V) and *must* be named '*.pm' (cf. man perlmod).

Though, I am not aware about perl prohibiting them to carry shebangs or these file to carry executable permission, they usually don't have either.

Files named *.pl (such as the perl5db.pl case, here), however never are modules and therefore should never provide rpm perl()-provides.


In other words, I think Matthias filter proposal could be extended to exclude all files on the "@INC" path not named '*.pm' from generating "perl()'-provides, rsp. to let rpm only generate "perl()-provs" for files matching "*.pm" and being under "@INC".

[There is a corner case: While *.pls normally do not provide perl-modules, they may contain self-provided perl-modules]

Comment 27 Jan Kaluža 2013-03-25 11:05:08 UTC
> > The question is,
> > what is the difference between Perl Module and Perl Script except the
> > possible extension?
> The file name extension is the #1 distinction criterion. Additional ones are
> the shebang and yet another one is "executable-permissions".

Right, but only the shebang is acceptable for File. It does not detect files according to names or permissions. But I agree that RPM should start by checking the extension in this case.
 
> However, there is a difference between the general case of guessing on a
> file's type and on processing a specific file in context of rpm and
> perl-modules (c.f. the next sentence)

+1

> > Can Perl file with shebang contain Perl Module?
> rpm-relevant perl-modules *must* be on the search-path provided through 
> "@INC" (cf. perl -V) and *must* be named '*.pm' (cf. man perlmod).
>
> Though, I am not aware about perl prohibiting them to carry shebangs or
> these file to carry executable permission, they usually don't have either.

Ok, I will check some Perl files I have here to be really sure and detect anything with Perl shebang as "Perl script" in next File build.

> Files named *.pl (such as the perl5db.pl case, here), however never are
> modules and therefore should never provide rpm perl()-provides.

As explained above, this should be handled by rpm, not by File. File detects the type of file according to its content. It will still says "Perl module" for .pl file if it does not have shebang and defines "package".

Comment 28 Jan Kaluža 2013-03-25 11:46:43 UTC
I have create Bug 927211 to move RPM related part of this bug to RPM component.

Comment 29 Kevin Fenzi 2013-03-25 14:01:00 UTC
(In reply to comment #22)
> Untagged, please wait for build root rotation:

This is not an acceptable solution. ;( 

Once a package has gone out in a rawhide compose it's installed on lots of machines, simply untagging it papers over the buildroot issue and skews the rawhide repo from reality, as well as many other packages that are built against it. Our policy is to never do this once it's gone out in a compose. 

Please fix. ;)

Comment 30 Yanko Kaneti 2013-03-25 14:11:01 UTC
(In reply to comment #29)
> (In reply to comment #22)
> > Untagged, please wait for build root rotation:
> 
> This is not an acceptable solution. ;( 
> 
> Once a package has gone out in a rawhide compose it's installed on lots of
> machines, simply untagging it papers over the buildroot issue and skews the
> rawhide repo from reality, as well as many other packages that are built
> against it. Our policy is to never do this once it's gone out in a compose. 
> 
> Please fix. ;)

Something else that should be unacceptable is leaving rawhide in a semi broken state for three-four days including the weekend. 
And untagging has to be a valid workaround for that.

Comment 31 Dennis Gilmore 2013-03-25 14:26:44 UTC
untagging is only a valid workaround if its needed to build the fix.  untagging a build under any other situation is unacceptable.  please fix properly

Comment 32 Fedora Update System 2013-04-11 11:02:22 UTC
perl-5.16.3-262.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/perl-5.16.3-262.fc19

Comment 33 Fedora Update System 2013-04-11 11:05:55 UTC
perl-5.16.3-242.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/perl-5.16.3-242.fc18

Comment 34 Fedora Update System 2013-04-11 13:05:56 UTC
perl-5.14.4-225.fc17 has been submitted as an update for Fedora 17.
https://admin.fedoraproject.org/updates/perl-5.14.4-225.fc17

Comment 35 Fedora Update System 2013-04-20 19:15:15 UTC
perl-5.16.3-262.fc19, perl-Sys-Syslog-0.32-1.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 36 Fedora Update System 2013-04-25 00:44:59 UTC
perl-5.16.3-242.fc18, perl-Sys-Syslog-0.32-1.fc18 has been pushed to the Fedora 18 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 37 Fedora Update System 2013-04-27 00:10:57 UTC
perl-5.14.4-225.fc17 has been pushed to the Fedora 17 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.