Bug 217833

Summary: The Text::Tabs expand fails to expand correctly for UTF-8 data
Product: Red Hat Enterprise Linux 5 Reporter: Jan Pazdziora <jpazdziora>
Component: perlAssignee: Marcela Mašláňová <mmaslano>
Status: CLOSED ERRATA QA Contact: David Lawrence <dkl>
Severity: low Docs Contact:
Priority: medium    
Version: 5.0CC: kas, psplicha
Target Milestone: rcKeywords: Regression
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 493349 (view as bug list) Environment:
Last Closed: 2009-01-20 21:11:04 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jan Pazdziora 2006-11-30 08:58:32 UTC
Description of problem:

When used with UTF-8 characters, the Text::Tabs's expand fails to expand correctly.

Version-Release number of selected component (if applicable):

$ rpm -qf /usr/lib/perl5/5.8.8/Text/Tabs.pm
perl-5.8.8-10
$ grep VERSION /usr/lib/perl5/5.8.8/Text/Tabs.pm
use vars qw($VERSION $tabstop $debug);
$VERSION = 2005.0824;

How reproducible:

On a command line.

Steps to Reproduce:
1. perl -CS -MText::Tabs -e 'print expand("\taa\t.\n\t\x{010a}\x{010a}\t."), "\n"'
2. perl -CS -MText::Tabs -e 'print expand("\taa\t.\n\t\x{010a}\x{010a}\t."),
"\n"' | od -t x1
  
Actual results:

        aa      .
        ĊĊ       .

0000000 20 20 20 20 20 20 20 20 61 61 20 20 20 20 20 20
0000020 2e 0a 20 20 20 20 20 20 20 20 c4 8a c4 8a 20 20
0000040 20 20 20 20 20 2e 0a
0000047

Bugzilla might mess the formattin, that's why I'm showing the od output as well.
Note that after those two (UTF-8) characters, there should really be six spaces,
not seven.

Expected results:

        aa      .
        ĊĊ      .

0000000 20 20 20 20 20 20 20 20 61 61 20 20 20 20 20 20
0000020 2e 0a 20 20 20 20 20 20 20 20 c4 8a c4 8a 20 20
0000040 20 20 20 20 2e 0a
0000046

Additional info:

On RHEL 4 (U4)'s

$ rpm -qf /usr/lib/perl5/5.8.5/Text/Tabs.pm
perl-5.8.5-36.RHEL4
$ grep VERSION /usr/lib/perl5/5.8.5/Text/Tabs.pm
use vars qw($VERSION $tabstop $debug);
$VERSION = 98.112801;

the result was correct so this looks like regression in RHEL 5.

Yenya Kasprzak found the bug:
http://www.fi.muni.cz/~kas/blog/index.cgi/computers/cpan-bugs.html

Comment 1 Jan Pazdziora 2007-03-28 08:42:22 UTC
As the bug is present in RHEL 5 as well, I'm changing the Product to reflect that.

Comment 2 RHEL Program Management 2007-03-28 08:48:41 UTC
This bugzilla has Keywords: Regression.  

Since no regressions are allowed between releases, 
it is also being proposed as a blocker for this release.  

Please resolve ASAP.

Comment 3 Robin Norwood 2007-03-29 16:42:32 UTC
This is now here: http://rt.perl.org/rt3/Public/Bug/Display.html?id=42167

There is another implementation of Text::Tabs::expand in the Tabs.pm source file
that is not in use - this implementation doesn't have the problem described
here, but I'm hesitant to switch to that one until we hear more from upstream -
it may have other bugs.

The underlying issue seems to be that the pos() function seems to count
incorrectly with UTF8 characters.

Comment 7 Jan Pazdziora 2007-12-14 12:13:56 UTC
Hmmm. Per comment #4, perl was an approved component for RHEL 5.1. Yet for some
reason, the bug was moved to RHEL 5.2 in comment #5. Alas, perl is not an
approved component for RHEL 5.2 as per comment #6.

What is the suggested course of action?

Moving to RHEL 5.3?

Comment 8 RHEL Program Management 2007-12-14 12:16:14 UTC
This bugzilla has Keywords: Regression.  

Since no regressions are allowed between releases, 
it is also being proposed as a blocker for this release.  

Please resolve ASAP.

Comment 9 Jan Pazdziora 2007-12-14 12:23:12 UTC
(In reply to comment #8)
> This bugzilla has Keywords: Regression.  
> 
> Since no regressions are allowed between releases, 
> it is also being proposed as a blocker for this release.  

Soooo. Doesn't this also make this a blocker (and perl approved component) for 5.2?


Comment 10 Marcela Mašláňová 2008-07-04 09:36:31 UTC
Sooo because upstream is still quiet about it, I'll use previous version of
unexpand. It's maybe slower, but functional.

Comment 13 Marcela Mašláňová 2008-07-21 11:30:12 UTC
The final fix use change of pos from bleadperl.
http://rt.perl.org/rt3/Ticket/Display.html?id=57040

Comment 17 errata-xmlrpc 2009-01-20 21:11:04 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2009-0117.html