Bug 196836 - perl-5.8.8-5 is 30X slower than perl-5.8.8-4
Summary: perl-5.8.8-5 is 30X slower than perl-5.8.8-4
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: perl
Version: 5
Hardware: i386
OS: Linux
medium
high
Target Milestone: ---
Assignee: Robin Norwood
QA Contact: David Lawrence
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2006-06-27 04:17 UTC by Colin Kuskie
Modified: 2008-08-26 17:45 UTC (History)
5 users (show)

Fixed In Version: perl-5.8.8-22.fc8
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2007-07-25 19:20:49 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
Results of perlbench-0.93 when run for perl-5.8.8-5 and perl-5.8.8-4 on a dual processor i386 FC-5 machine (1.46 KB, text/plain)
2006-07-14 22:39 UTC, Jason Vas Dias
no flags Details

Description Colin Kuskie 2006-06-27 04:17:52 UTC
Description of problem:

I write and maintain tests for WebGUI, and after upgrading my
laptop (running FC5) on June 17th I noticed a huge slowdown
in performance in the tests.

From the list of upgraded packages, I traced it down to perl
itself.

Here's data before and after a downgrade back to perl-5.8.8-4:

[colink@redwall t]$ rpm -qf /usr/bin/prove
perl-5.8.8-5
[colink@redwall t]$ prove --timer i18n/label.t
i18n/label....ok 1125.916s
All tests successful.
Files=1, Tests=6470, 1125 wallclock secs (1068.49 cusr +  1.27 csys = 1069.76 CPU)
[colink@redwall t]$ rpm -qf /usr/bin/prove
perl-5.8.8-4
[colink@redwall t]$ !prove
prove --timer i18n/label.t
i18n/label....ok   38.331s
All tests successful.
Files=1, Tests=6470, 39 wallclock secs (36.92 cusr +  0.47 csys = 37.39 CPU)

1125/39 =~ 30

Version-Release number of selected component (if applicable):
perl-5.8.8-5

How reproducible:
Every time.

Steps to Reproduce:
1. Install perl-5.8.8-5
2. Huge slowdown in perl.
3.

Additional info:

The test with the most slowdown uses the Text::Balanced module.

Comment 1 Jan Christiaan van Winkel 2006-06-28 13:25:55 UTC
I can confirm this for the X86_64 version.  I Use Slim Devices' slimserver
(slimserver-6.2.2-1) runs somewhere 20 times faster when just switching between
versions perl-5.8.8-4 and perl-5.8.8-5 (and no other rpms are updated) with the
perl-5.8.8-5 version being the slower one.  Other perl related RPMs I have
installed:
perl-Digest-SHA1-2.11-1.2
perl-XML-Parser-2.34-6.1.2.2
perl-SGMLSpm-1.03ii-16.2
mod_perl-2.0.2-5.1
perl-Digest-HMAC-1.01-14.2
perl-BSD-Resource-1.24-3.2.2
newt-perl-1.08-9.2.1
perl-Compress-Zlib-1.41-1.2.2
perl-Net-DNS-0.57-1
perl-String-CRC32-1.4-1.FC5
perl-DBI-1.50-2.2
perl-Net-IP-1.24-2.2
perl-libwww-perl-5.805-1.1
perl-HTML-Parser-3.51-1.FC5
perl-DBD-MySQL-3.0004-1.FC5
perl-5.8.8-4
perl-URI-1.35-2.2
perl-HTML-Tagset-3.10-2.1

I am not a perl hacker (I do more in C++ and python), but I am more than willing
to provide any trace data that I could generate.

Comment 2 Jason Vas Dias 2006-07-14 22:39:08 UTC
Created attachment 132469 [details]
Results of perlbench-0.93 when run for perl-5.8.8-5 and perl-5.8.8-4 on a dual processor i386 FC-5 machine

The attached perlbench log shows that the performance of perl-5.8.8-4 compared
to perl-5.8.8-5 is virtually identical for common perl operations . Perlbench
is a standard benchmark test for perl.

Yes, I have duplicated the severe performance loss when running the WebGUI
t/i18n/label.t test.

Reblessing overloaded objects now incurs a performance penalty in 5.8.8-5+ : 
this is due to the fix for perlbug #34925 'overload and rebless':
   http://rt.perl.org/rt3/Ticket/Display.html?id=34925
The fix for this bug (change 27512) involves a 'brute force approach' to 
detect overloaded objects that are then reblessed; the performance penalty
is proportional to the number of active scalars in the perl program.

Testing 5.8.8-5 compiled without the 'perl-5.8.8-U27512.patch' applied
shows no performance loss when running the WebGUI t/i118n/label.t test
over 5.8.8-4 - the 27512 patch is definitely the cause.

Perlbug 34925 severely affects the CPAN DBI::Class module.

As reblessing an overloaded object is quite rare, the brute force approach
was deemed acceptable by the upstream perl maintainers, who have applied this 
overload and rebless detection mechanism to the current maintenance release
of perl.

In the upstream 'blead' perl 5.9.4, a fix with no adverse performance impact
was applied (change # 27506), but this introduces binary incompatibilities 
which impact  mod_perl; one cannot apply this fix to the 5.8.x releases without

introducing major binary incompatibilities, and I've not been able to see
an alternative fix (in 5.8.8, every SV_f* flag bit is already used - in 
5.9.4, they've changed the meaning of the flag bits and reserved one 
to flag the case of overload and rebless).

So, I think we'll have to accept the performance tradeoff for correct 
operation in the presence of overload and rebless, until perl-5.9.x 
is released as the stable version of perl-5.

Comment 3 Stephen Clouse 2006-08-22 11:45:28 UTC
The real problem seems to be that patch 27512 breaks more than it fixes.  I
believe the patch itself is bugged; the brute force search seems to be
triggering in cases where it doesn't seem to be necessary.

I have a project internal to my employer that makes heavy use of the DateTime
module, which was significantly affected by this patch.  DateTime does overload
but not reblessing, but the search is still triggered (for no apparent reason)
whenever the ->clone method is called, even though there's no reason for it (a
completely new reference is created by ->clone, it does *not* rebless anything).

DBIx::Class is also bitten by this; see
http://forums.slimdevices.com/showpost.php?p=123372&postcount=20

Personally I think this patch needs more testing from upstream before it goes
into a stable line update to Fedora.

Comment 4 Robin Norwood 2006-08-22 21:21:33 UTC
Reopened - assigning to me to keep an eye on this issue.

Comment 5 Christopher McCrory 2007-01-12 21:18:42 UTC
This shows up in RHEL5 also.

Would another bug be better, or ... overload this one  :)




Comment 6 hkoba 2007-06-19 23:07:28 UTC
perl-5.8.8-10 is 1000X slower! 
(It seems like O(2^n) slowdown where n is number of onmemory SV)
Please rethink. It can kill real apps. (Or Fedora/RedHat is not for Perl?)

# Here is simple benchmark, borrowed and modified from
# http://blog.yappo.jp/yappo/archives/000515.html

use Benchmark;
timethese(shift || 100000, {
'overload' => sub { push our @array, TestOverload->new },
'not overload' => sub { push our @array, TestNoverload->new },
});

package TestNoverload;
sub new { bless { hoge => 'hoge' }, shift }


package TestOverload;
use overload (
q{""} => sub {},
);
sub new { bless { hoge => 'hoge' }, shift }

Comment 7 Steven Pritchard 2007-06-20 17:07:47 UTC
FWIW, I just verified that perl-5.8.8-18.fc7 still has this problem, but a 
fresh compile of 5.9.4 doesn't.

Comment 8 hkoba 2007-06-21 07:26:21 UTC
It was not O(2^n) but O(n^2), sorry. See following plot(n=10000 .. 100000)

http://buribullet.net/svntrac/buribullet/attach_get/2/fedora-perl-overload.png

Of course this should be O(n). O(n^2) for this case is not acceptable overhead.

Comment 9 Robin Norwood 2007-06-22 20:20:12 UTC
This is reported upstream here:

http://rt.perl.org/rt3/Public/Bug/Display.html?id=43283

And, as it turns out, Nicholas Clark points out a patch which fixes this issue
on my test system:

Benchmark: timing 100000 iterations of not overload, overload...
not overload:  1 wallclock secs ( 0.48 usr +  0.24 sys =  0.72 CPU) @
138888.89/s (n=100000)
  overload:  2 wallclock secs ( 0.50 usr +  0.22 sys =  0.72 CPU) @ 138888.89/s
(n=100000)

Building a fixed version for rawhide/f8 - updates for F7 and FC6 should be
available soon.



Comment 10 hkoba 2007-06-23 03:59:57 UTC
Thank you for your quick action!
It will save both community of Perl and Fedora/RedHat.


Comment 11 Robin Norwood 2007-06-23 15:49:25 UTC
Update - I'm having a little trouble building the new perl - I'm still working
on it, though.

Comment 12 hkoba 2007-07-01 06:42:47 UTC
> a little trouble...

If it is so hard, simply removing U27512 & U27509 will be
good enough for most of Perl users, IMHO.

# Following patch is borrowed from http://femo.jp/kazeburo/br51s0s3qh

diff -u -r1.1 perl.spec
--- perl.spec   1 Jul 2007 01:47:23 -0000       1.1
+++ perl.spec   1 Jul 2007 01:50:05 -0000
@@ -5,7 +5,7 @@
 %define multilib_64_archs x86_64 s390x ppc64 sparc64

 %define perlver    5.8.8
-%define perlrel    10
+%define perlrel    11
 %define perlepoch  4

 %{?!perl_debugging:    %define perl_debugging 0}
@@ -160,8 +160,8 @@
 Patch27116:    perl-5.8.8-U27116.patch
 Patch27391:     perl-5.8.8-U27391.patch
 Patch27426:    perl-5.8.8-U27426.patch
-Patch27509:     perl-5.8.8-U27509.patch
-Patch27512:     perl-5.8.8-U27512.patch
+#Patch27509:     perl-5.8.8-U27509.patch
+#Patch27512:     perl-5.8.8-U27512.patch
 Patch27604:     perl-5.8.8-U27604.patch
 Patch27605:     perl-5.8.8-U27605.patch
 Patch27914:     perl-5.8.8-U27914.patch
@@ -367,9 +367,9 @@

 %patch27426 -p1

-%patch27509 -p1
+#%patch27509 -p1

-%patch27512 -p1
+#%patch27512 -p1

 %patch27604 -p1


Comment 13 Robin Norwood 2007-07-25 19:20:49 UTC
This should be fixed in the latest perl in F8/rawhide: perl-5.8.8-22.fc8 or later

Comment 14 Marcos Marado 2007-12-12 12:36:33 UTC
According to the changelog, 4:5.8.8-19 for -fc7 solves this bug, but the latest
4:5.8.8-27 still gives this warning:

WARNING: DBIx::Class::StartupCheck: This version of Perl is likely to exhibit
extremely slow performance for certain critical operations.
Please consider recompiling Perl.  For more information, see
https://bugzilla.redhat.com/show_bug.cgi?id=196836 and/or
http://lists.scsys.co.uk/pipermail/dbix-class/2007-October/005119.html.
You can suppress this message by setting DBIC_NO_WARN_BAD_PERL=1 in your
environment.

Any ideas?

Comment 15 Chris Weyl 2007-12-12 16:43:23 UTC
Following through that thread, and looking at the source of DBIC::StartupCheck,
it appears that it's more of a "MAY" than definite, and that DBIC::StartupClass
doesn't know how to check to see if the particular patches have been applied. 
(AFAICT at least.)

I have DBIx::Class up for review (bug perl-DBIx-Class); if we can verify that
indeed Fedora perl has the correct fixes in place (which it does sound like it
does) then hopefully we can:

1. override DBIC::StartupCheck's logic in Fedora's packages, until #2
2. create and funnel logic upstream that accurately figures out if this has been
applied... Note that this needs to be a near no-op, as these checks are run
everytime DBIC is use'd.


Note You need to log in before you can comment on or make changes to this bug.