Red Hat Bugzilla – Bug 192400
YAML triggers segmentation fault on data larger than 50kB
Last modified: 2014-09-02 02:42:14 EDT
Description of problem: YAML does ``segmentation fault'' when fed with a string longer than approx 28000 characters Version-Release number of selected component (if applicable): perl-YAML-0.39-2 perl-5.8.8-4 How reproducible: Always Steps to Reproduce: 1. $ perl -MYAML -e '$a="a"x10000;$a="---\ntext: \"$a\"\n"; Load($a)' 2. $ perl -MYAML -e '$a="a"x50000;$a="---\ntext: \"$a\"\n"; Load($a)' Actual results: The first example runs ok, the seconds exits with ``Segmentation fault'' Expected results: Both should work ok, 50kB of text is not much today Additional info: I'm not sure whether this is problem with YAML or with perl interpreter itself, in theory no perl package without C code should be able to trigger segmentation faults
Again, I'm not sure whether this is a bug in YAML or perl interpreter. YAML.pm barfs at YAML.pm line: if ($o->{inline} =~ /^"((?:\\"|[^"])*)"\s*(.*)$/) { Here's a microscopic program that segmentation-faults without any YAML: #! /usr/bin/perl $_ = "a" . ("b" x 50000) . "a"; /^a((?:ca|[^a])*)a/;
I would suggest opening a bug against perl. I wonder if this explains the random death of one of my perl apps...
Created attachment 129717 [details] With this patch the bug appears to be gone, there's a small performance penalty
The segfault still seems to be present on F8: # rpm -qf /usr/lib/perl5/vendor_perl/5.8.8/YAML/Loader.pm perl-YAML-0.66-1.fc8.noarch The patch for this version of YAML would be (not tested for semantical correctness): --- /usr/lib/perl5/vendor_perl/5.8.8/YAML/Loader.pm.orig 2008-02-27 13:20:06.000000000 +0100 +++ /usr/lib/perl5/vendor_perl/5.8.8/YAML/Loader.pm 2008-02-27 13:22:14.000000000 +0100 @@ -507,9 +507,26 @@ return $node; } +# Work around /regexp/ bug in perl < 5.10 +sub _parse_inline_double_quoted_perl_bug_work_around { + my $self = shift; + my @list; + local $_ = $self->inline; + s{^"}{} or croak YAML_PARSE_ERR_BAD_DOUBLE(); + push @list, $1 + while s{^((?:\\.|[^\"\\]+){1,1000})}{}; + s/\\"/"/g for @list; + s{^"}{} or croak YAML_PARSE_ERR_BAD_DOUBLE(); + $self->inline($_); + return join("", @list); +} + + # Parse the inline double quoted string. sub _parse_inline_double_quoted { my $self = shift; + return $self->_parse_inline_double_quoted_perl_bug_work_around(@_) + if $]<5.009 && length($self->inline) > 10_000; my $node; if ($self->inline =~ /^"((?:\\"|[^"])*)"\s*(.*)$/) { $node = $1;
This is with fedora rawhide from 2008-03-13 rpm -q perl perl-YAML perl-5.10.0-14.fc9.x86_64 perl-YAML-0.66-3.fc9.noarch $ perl -MYAML -e '$a="a"x50000;$a="---\ntext: \"$a\"\n"; Load($a)' Complex regular subexpression recursion limit (32766) exceeded at /usr/lib/perl5/vendor_perl/5.10.0/YAML/Loader.pm line 514. YAML Error: Can't parse double quoted string Code: YAML_PARSE_ERR_BAD_DOUBLE Line: 2 Document: 1 at /usr/lib/perl5/vendor_perl/5.10.0/YAML.pm line 33 So on rawhide it's no longer generating segfault
Fedora apologizes that these issues have not been resolved yet. We're sorry it's taken so long for your bug to be properly triaged and acted on. We appreciate the time you took to report this issue and want to make sure no important bugs slip through the cracks. If you're currently running a version of Fedora Core between 1 and 6, please note that Fedora no longer maintains these releases. We strongly encourage you to upgrade to a current Fedora release. In order to refocus our efforts as a project we are flagging all of the open bugs for releases which are no longer maintained and closing them. http://fedoraproject.org/wiki/LifeCycle/EOL If this bug is still open against Fedora Core 1 through 6, thirty days from now, it will be closed 'WONTFIX'. If you can reporduce this bug in the latest Fedora version, please change to the respective version. If you are unable to do this, please add a comment to this bug requesting the change. Thanks for your help, and we apologize again that we haven't handled these issues to this point. The process we are following is outlined here: http://fedoraproject.org/wiki/BugZappers/F9CleanUp We will be following the process here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping to ensure this doesn't happen again. And if you'd like to join the bug triage team to help make things better, check out http://fedoraproject.org/wiki/BugZappers
Per comment 5, the segmentation fault has been fixed in rawhide. On the other hand, the large input still cannot be processed correctly. Do we want another bug to track that problem or can we just change the Summary of this bug and align it to rawhide?
i'll move this to rawhide since all FC5 bugs will be closed.
Changing version to '9' as part of upcoming Fedora 9 GA. More information and reason for this action is here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping
This message is a reminder that Fedora 9 is nearing its end of life. Approximately 30 (thirty) days from now Fedora will stop maintaining and issuing updates for Fedora 9. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as WONTFIX if it remains open with a Fedora 'version' of '9'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version prior to Fedora 9's end of life. Bug Reporter: Thank you for reporting this issue and we are sorry that we may not be able to fix it before Fedora 9 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora please change the 'version' of this bug to the applicable version. If you are unable to change the version, please add a comment here and someone will do it for you. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete. The process we are following is described here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Marking as Fedora 10 issue. $ rpm -q perl perl-YAML perl-5.10.0-68.fc10.i386 perl-YAML-0.66-3.fc9.noarch $ perl -MYAML -e '$a="a"x50000;$a="---\ntext: \"$a\"\n"; Load($a)' Complex regular subexpression recursion limit (32766) exceeded at /usr/lib/perl5/vendor_perl/5.10.0/YAML/Loader.pm line 514. YAML Error: Can't parse double quoted string Code: YAML_PARSE_ERR_BAD_DOUBLE Line: 2 Document: 1 at /usr/lib/perl5/vendor_perl/5.10.0/YAML.pm line 33
Marking as Fedora 11 issue. $ rpm -q perl perl-YAML perl-5.10.0-69.fc11.i586 perl-YAML-0.68-2.fc11.noarch $ perl -MYAML -e '$a="a"x50000;$a="---\ntext: \"$a\"\n"; Load($a)' Complex regular subexpression recursion limit (32766) exceeded at /usr/lib/perl5/vendor_perl/5.10.0/YAML/Loader.pm line 514. YAML Error: Can't parse double quoted string Code: YAML_PARSE_ERR_BAD_DOUBLE Line: 2 Document: 1 at /usr/lib/perl5/vendor_perl/5.10.0/YAML.pm line 33 $
This message is a reminder that Fedora 11 is nearing its end of life. Approximately 30 (thirty) days from now Fedora will stop maintaining and issuing updates for Fedora 11. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as WONTFIX if it remains open with a Fedora 'version' of '11'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version prior to Fedora 11's end of life. Bug Reporter: Thank you for reporting this issue and we are sorry that we may not be able to fix it before Fedora 11 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora please change the 'version' of this bug to the applicable version. If you are unable to change the version, please add a comment here and someone will do it for you. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete. The process we are following is described here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Marking as Fedora 13/14 issue: $ rpm -q perl perl-YAML perl-5.10.1-118.fc14.i686 perl-YAML-0.70-4.fc14.noarch $ perl -MYAML -e '$a="a"x50000;$a="---\ntext: \"$a\"\n"; Load($a)' Complex regular subexpression recursion limit (32766) exceeded at /usr/share/perl5/YAML/Loader.pm line 519. YAML Error: Can't parse double quoted string Code: YAML_PARSE_ERR_BAD_DOUBLE Line: 2 Document: 1 at /usr/share/perl5/YAML.pm line 36 $
This bug appears to have been reported against 'rawhide' during the Fedora 14 development cycle. Changing version to '14'. More information and reason for this action is here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping
This message is a notice that Fedora 14 is now at end of life. Fedora has stopped maintaining and issuing updates for Fedora 14. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At this time, all open bugs with a Fedora 'version' of '14' have been closed as WONTFIX. (Please note: Our normal process is to give advanced warning of this occurring, but we forgot to do that. A thousand apologies.) Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, feel free to reopen this bug and simply change the 'version' to a later Fedora version. Bug Reporter: Thank you for reporting this issue and we are sorry that we were unable to fix it before Fedora 14 reached end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged to click on "Clone This Bug" (top right of this page) and open it against that version of Fedora. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete. The process we are following is described here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Issue still present on Fedora 17: $ perl -MYAML -e '$a="a"x10000;$a="---\ntext: \"$a\"\n"; Load($a)' $ perl -MYAML -e '$a="a"x50000;$a="---\ntext: \"$a\"\n"; Load($a)' Complex regular subexpression recursion limit (32766) exceeded at /usr/share/perl5/vendor_perl/YAML/Loader.pm line 517. YAML Error: Can't parse double quoted string Code: YAML_PARSE_ERR_BAD_DOUBLE Line: 2 Document: 1 at /usr/share/perl5/vendor_perl/YAML/Loader.pm line 523 $ rpm -q perl-YAML perl-YAML-0.81-2.fc17.noarch $
This message is a reminder that Fedora 17 is nearing its end of life. Approximately 4 (four) weeks from now Fedora will stop maintaining and issuing updates for Fedora 17. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as WONTFIX if it remains open with a Fedora 'version' of '17'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version prior to Fedora 17's end of life. Bug Reporter: Thank you for reporting this issue and we are sorry that we may not be able to fix it before Fedora 17 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior to Fedora 17's end of life. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.
Fedora 19. # perl -MYAML -e '$a="a"x10000;$a="---\ntext: \"$a\"\n"; Load($a)' # perl -MYAML -e '$a="a"x50000;$a="---\ntext: \"$a\"\n"; Load($a)' Complex regular subexpression recursion limit (32766) exceeded at /usr/share/perl5/vendor_perl/YAML/Loader.pm line 517. YAML Error: Can't parse double quoted string Code: YAML_PARSE_ERR_BAD_DOUBLE Line: 2 Document: 1 at /usr/share/perl5/vendor_perl/YAML/Loader.pm line 523. # rpm -q perl perl-YAML perl-5.16.3-265.fc19.x86_64 perl-YAML-0.84-5.fc19.noarch #
Still present in 20. The issue is that perl-refexes are super-linear with memory so we end up using all the ram just to parse a 50kB quoted string.
Created attachment 887606 [details] Updated long-string workaround patch for perl-YAML Updated previous patch for newer perl-YAML (fedora 20). Changes: - $o -> $self - croak -> $self->die() - Bumped per-iteration string limit from 1k to 10k - Trigger slow path even on newer versions of perl (hint: parsing quoted strings ins't a perl bug, it's a perl-YAML bug) when string length >10k
Better fix: Use YAML::XS. For git-svn users who find themselves hitting this, Git::SVN::Memoize::YAML will use YAML::Any, which will pick YAML::XS *if it is installed*. Suggest: Add perl(YAML::XS) as a dependency for perl(YAML), because it is actually sane.
perl-YAML-0.98-1.fc20 has been submitted as an update for Fedora 20. https://admin.fedoraproject.org/updates/perl-YAML-0.98-1.fc20
Any idea *how* upstream 0.98 "Fixes indexing of YAML::Any", and if that solves this issue? https://github.com/ingydotnet/yaml-pm/commit/4411c43d1b248bca657be3b914f0455b132c6036 From the diff, I'm not seeing it...
The fix for the parsing issue was introduced in 0.85 and was refined over a few subsequent releases. During the course of those releases, a regression was introduced in which YAML::Any had no VERSION, which could cause dependency issues, so I included the fix for that. The test case from Comment #19 works properly with the updated version.
Great, thanks for the update Paul.
Package perl-YAML-0.98-1.fc20: * should fix your issue, * was pushed to the Fedora 20 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing perl-YAML-0.98-1.fc20' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2014-9786/perl-YAML-0.98-1.fc20 then log in and leave karma (feedback).
Things are very good with perl-YAML-0.98-1.fc20.noarch: $ perl -MYAML -e '$a="a"x50000;$a="---\ntext: \"$a\"\n"; Load($a)' $ perl -l -MYAML -e '$a="a"x50000;$a="---\ntext: \"$a\"\n"; $d = Load($a); print length $d->{text};' 50000
perl-YAML-0.98-1.fc20 has been pushed to the Fedora 20 stable repository. If problems still persist, please make note of it in this bug report.