Bug 217833 - The Text::Tabs expand fails to expand correctly for UTF-8 data
Summary: The Text::Tabs expand fails to expand correctly for UTF-8 data
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: perl
Version: 5.0
Hardware: All
OS: Linux
medium
low
Target Milestone: rc
: ---
Assignee: Marcela Mašláňová
QA Contact: David Lawrence
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2006-11-30 08:58 UTC by Jan Pazdziora
Modified: 2010-05-21 10:32 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 493349 (view as bug list)
Environment:
Last Closed: 2009-01-20 21:11:04 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2009:0117 0 normal SHIPPED_LIVE perl bug fix update 2009-01-20 16:04:48 UTC

Description Jan Pazdziora 2006-11-30 08:58:32 UTC
Description of problem:

When used with UTF-8 characters, the Text::Tabs's expand fails to expand correctly.

Version-Release number of selected component (if applicable):

$ rpm -qf /usr/lib/perl5/5.8.8/Text/Tabs.pm
perl-5.8.8-10
$ grep VERSION /usr/lib/perl5/5.8.8/Text/Tabs.pm
use vars qw($VERSION $tabstop $debug);
$VERSION = 2005.0824;

How reproducible:

On a command line.

Steps to Reproduce:
1. perl -CS -MText::Tabs -e 'print expand("\taa\t.\n\t\x{010a}\x{010a}\t."), "\n"'
2. perl -CS -MText::Tabs -e 'print expand("\taa\t.\n\t\x{010a}\x{010a}\t."),
"\n"' | od -t x1
  
Actual results:

        aa      .
        ĊĊ       .

0000000 20 20 20 20 20 20 20 20 61 61 20 20 20 20 20 20
0000020 2e 0a 20 20 20 20 20 20 20 20 c4 8a c4 8a 20 20
0000040 20 20 20 20 20 2e 0a
0000047

Bugzilla might mess the formattin, that's why I'm showing the od output as well.
Note that after those two (UTF-8) characters, there should really be six spaces,
not seven.

Expected results:

        aa      .
        ĊĊ      .

0000000 20 20 20 20 20 20 20 20 61 61 20 20 20 20 20 20
0000020 2e 0a 20 20 20 20 20 20 20 20 c4 8a c4 8a 20 20
0000040 20 20 20 20 2e 0a
0000046

Additional info:

On RHEL 4 (U4)'s

$ rpm -qf /usr/lib/perl5/5.8.5/Text/Tabs.pm
perl-5.8.5-36.RHEL4
$ grep VERSION /usr/lib/perl5/5.8.5/Text/Tabs.pm
use vars qw($VERSION $tabstop $debug);
$VERSION = 98.112801;

the result was correct so this looks like regression in RHEL 5.

Yenya Kasprzak found the bug:
http://www.fi.muni.cz/~kas/blog/index.cgi/computers/cpan-bugs.html

Comment 1 Jan Pazdziora 2007-03-28 08:42:22 UTC
As the bug is present in RHEL 5 as well, I'm changing the Product to reflect that.

Comment 2 RHEL Program Management 2007-03-28 08:48:41 UTC
This bugzilla has Keywords: Regression.  

Since no regressions are allowed between releases, 
it is also being proposed as a blocker for this release.  

Please resolve ASAP.

Comment 3 Robin Norwood 2007-03-29 16:42:32 UTC
This is now here: http://rt.perl.org/rt3/Public/Bug/Display.html?id=42167

There is another implementation of Text::Tabs::expand in the Tabs.pm source file
that is not in use - this implementation doesn't have the problem described
here, but I'm hesitant to switch to that one until we hear more from upstream -
it may have other bugs.

The underlying issue seems to be that the pos() function seems to count
incorrectly with UTF8 characters.

Comment 7 Jan Pazdziora 2007-12-14 12:13:56 UTC
Hmmm. Per comment #4, perl was an approved component for RHEL 5.1. Yet for some
reason, the bug was moved to RHEL 5.2 in comment #5. Alas, perl is not an
approved component for RHEL 5.2 as per comment #6.

What is the suggested course of action?

Moving to RHEL 5.3?

Comment 8 RHEL Program Management 2007-12-14 12:16:14 UTC
This bugzilla has Keywords: Regression.  

Since no regressions are allowed between releases, 
it is also being proposed as a blocker for this release.  

Please resolve ASAP.

Comment 9 Jan Pazdziora 2007-12-14 12:23:12 UTC
(In reply to comment #8)
> This bugzilla has Keywords: Regression.  
> 
> Since no regressions are allowed between releases, 
> it is also being proposed as a blocker for this release.  

Soooo. Doesn't this also make this a blocker (and perl approved component) for 5.2?


Comment 10 Marcela Mašláňová 2008-07-04 09:36:31 UTC
Sooo because upstream is still quiet about it, I'll use previous version of
unexpand. It's maybe slower, but functional.

Comment 13 Marcela Mašláňová 2008-07-21 11:30:12 UTC
The final fix use change of pos from bleadperl.
http://rt.perl.org/rt3/Ticket/Display.html?id=57040

Comment 17 errata-xmlrpc 2009-01-20 21:11:04 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2009-0117.html


Note You need to log in before you can comment on or make changes to this bug.