Bug 74884 - DateManip complains of Malformed UTF-8
DateManip complains of Malformed UTF-8
Status: CLOSED CURRENTRELEASE
Product: Red Hat Linux
Classification: Retired
Component: perl-DateManip (Show other bugs)
8.0
i686 Linux
medium Severity low
: ---
: ---
Assigned To: Jason Vas Dias
:
: 88017 (view as bug list)
Depends On:
Blocks: CambridgeTarget
  Show dependency treegraph
 
Reported: 2002-10-02 11:36 EDT by Tom Cross
Modified: 2007-04-18 12:47 EDT (History)
6 users (show)

See Also:
Fixed In Version: perl-5.8.0-90.4
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2006-01-19 17:33:50 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Script to find Perl Modules that iconv dislikes (309 bytes, text/x-perl)
2004-04-15 05:53 EDT, Need Real Name
no flags Details

  None (edit)
Description Tom Cross 2002-10-02 11:36:22 EDT
From Bugzilla Helper:
User-Agent: Mozilla/5.0 Galeon/1.2.6 (X11; Linux i686; U;) Gecko/20020916

Description of problem:
When using the DateManip perl module, stderr is filled with "Malformed UTF-8
character..." messages.

Version-Release number of selected component (if applicable):


How reproducible:
Always

Steps to Reproduce:
1. use Date::Manip in a perl script
2.
3.
	

Actual Results:  
Stuff like this appears on stdout: (but Date::Manip appears to actually work fine)

Malformed UTF-8 character (byte 0xfe) at
/usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5902.
Malformed UTF-8 character (unexpected non-continuation byte 0x6d, immediately
after start byte 0xee) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5902.
Malformed UTF-8 character (unexpected non-continuation byte 0x74, immediately
after start byte 0xe3) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5902.
Malformed UTF-8 character (1 byte, need 3, after start byte 0xe3) at
/usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5902.
Malformed UTF-8 character (1 byte, need 3, after start byte 0xe3) at
/usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5903.
Malformed UTF-8 character (unexpected non-continuation byte 0x6d, immediately
after start byte 0xee) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5906.
Malformed UTF-8 character (unexpected continuation byte 0xba, with no preceding
start byte) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5926.
Malformed UTF-8 character (unexpected continuation byte 0xba, with no preceding
start byte) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5927.
Malformed UTF-8 character (unexpected continuation byte 0xba, with no preceding
start byte) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5929.
Malformed UTF-8 character (unexpected continuation byte 0xba, with no preceding
start byte) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5929.
Malformed UTF-8 character (unexpected non-continuation byte 0x73, immediately
after start byte 0xe3) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5930.
Malformed UTF-8 character (unexpected non-continuation byte 0x7a, immediately
after start byte 0xe3) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5930.
Malformed UTF-8 character (unexpected non-continuation byte 0x7a, immediately
after start byte 0xe3) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5930.
Malformed UTF-8 character (unexpected continuation byte 0xba, with no preceding
start byte) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5930.
Malformed UTF-8 character (unexpected non-continuation byte 0x7a, immediately
after start byte 0xe3) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5931.
Malformed UTF-8 character (unexpected continuation byte 0xba, with no preceding
start byte) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5931.
Malformed UTF-8 character (unexpected non-continuation byte 0x7a, immediately
after start byte 0xe3) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5931.
Malformed UTF-8 character (unexpected continuation byte 0xba, with no preceding
start byte) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5931.
Malformed UTF-8 character (unexpected non-continuation byte 0x7a, immediately
after start byte 0xe3) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5932.
Malformed UTF-8 character (unexpected continuation byte 0xba, with no preceding
start byte) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5932.
Malformed UTF-8 character (unexpected non-continuation byte 0x7a, immediately
after start byte 0xe3) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5932.
Malformed UTF-8 character (unexpected continuation byte 0xba, with no preceding
start byte) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5932.
Malformed UTF-8 character (unexpected non-continuation byte 0x7a, immediately
after start byte 0xe3) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5933.
Malformed UTF-8 character (unexpected continuation byte 0xba, with no preceding
start byte) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5933.
Malformed UTF-8 character (unexpected continuation byte 0xba, with no preceding
start byte) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5933.
Malformed UTF-8 character (unexpected non-continuation byte 0x7a, immediately
after start byte 0xe3) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5933.
Malformed UTF-8 character (unexpected continuation byte 0xba, with no preceding
start byte) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5933.
Malformed UTF-8 character (unexpected continuation byte 0xba, with no preceding
start byte) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5933.
Malformed UTF-8 character (unexpected non-continuation byte 0x7a, immediately
after start byte 0xe3) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5934.
Malformed UTF-8 character (unexpected continuation byte 0xba, with no preceding
start byte) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5934.
Malformed UTF-8 character (unexpected non-continuation byte 0x7a, immediately
after start byte 0xe3) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5934.
Malformed UTF-8 character (unexpected continuation byte 0xba, with no preceding
start byte) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5934.
Malformed UTF-8 character (unexpected continuation byte 0xba, with no preceding
start byte) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5935.
Malformed UTF-8 character (unexpected non-continuation byte 0x6e, immediately
after start byte 0xee) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5943.
Malformed UTF-8 character (unexpected non-continuation byte 0x69, immediately
after start byte 0xee) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5943.
Malformed UTF-8 character (unexpected continuation byte 0xba, with no preceding
start byte) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5943.
Malformed UTF-8 character (unexpected continuation byte 0xba, with no preceding
start byte) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5943.
Malformed UTF-8 character (1 byte, need 3, after start byte 0xe3) at
/usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5944.
Malformed UTF-8 character (unexpected continuation byte 0xba, with no preceding
start byte) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5945.
Malformed UTF-8 character (unexpected continuation byte 0xba, with no preceding
start byte) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5946.
Malformed UTF-8 character (unexpected non-continuation byte 0x73, immediately
after start byte 0xe3) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5946.
Malformed UTF-8 character (unexpected non-continuation byte 0x7a, immediately
after start byte 0xe3) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5946.
Malformed UTF-8 character (unexpected non-continuation byte 0x7a, immediately
after start byte 0xe3) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5947.
Malformed UTF-8 character (unexpected continuation byte 0xba, with no preceding
start byte) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5947.
Malformed UTF-8 character (unexpected non-continuation byte 0x7a, immediately
after start byte 0xe3) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5947.
Malformed UTF-8 character (unexpected continuation byte 0xba, with no preceding
start byte) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5947.
Malformed UTF-8 character (unexpected non-continuation byte 0x7a, immediately
after start byte 0xe3) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5947.
Malformed UTF-8 character (unexpected continuation byte 0xba, with no preceding
start byte) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5947.
Malformed UTF-8 character (unexpected non-continuation byte 0x7a, immediately
after start byte 0xe3) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5948.
Malformed UTF-8 character (unexpected non-continuation byte 0x7a, immediately
after start byte 0xe3) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5948.
Malformed UTF-8 character (unexpected continuation byte 0xba, with no preceding
start byte) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5948.
Malformed UTF-8 character (unexpected non-continuation byte 0x7a, immediately
after start byte 0xe3) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5948.
Malformed UTF-8 character (unexpected continuation byte 0xba, with no preceding
start byte) at /usr/lib/perl5/vendor_perl/5.8.0/Date/Manip.pm line 5948.


Expected Results:  
Usually Date::Manip is quiet.  It does appear to work correctly however.  It's
just really noisy.  :-)

Additional info:
Comment 1 Ed Avis 2002-10-29 12:15:38 EST
A one-liner to reproduce this is:

% perl -MDate::Manip -e 0

Expected result: no output.  Actual result: all those malformed UTF-8 errors.
Comment 2 Ed Avis 2002-10-29 16:01:25 EST
The same error messages are produced when building Date::Manip from CPAN, so
it's not that the perl-DateManip package is bad in itself.  Must be some
interaction between Manip.pm and the way that your perl is built.

Setting LANG=C stops the errors, see
<http://www.ee.ethz.ch/~slist/mrtg/msg20204.html>.

(It's possible that other modules than Date::Manip have the same problem, of
course.)
Comment 3 Ed Avis 2002-11-15 08:30:03 EST
It seems to be related to the choice of UTF-8 locales.  So en_US.UTF8 causes 
the error messages, but the locale en_US doesn't.

This must be because Manip.pm contains literal high-bit-set characters, and 
when perl is running in the UTF-8 locale it expects its source code to use UTF-
8 as well.  Which may or may not be a good idea, but that's what it does.

Suggested fix is to change Manip.pm to use chr(xxx) instead of literal 
characters.  That would stop these warning messages, it wouldn't get 
Date::Manip to work correctly with UTF-8 input I think, but it's a start.
Comment 4 Richard Plana 2002-12-04 13:31:56 EST
Thanks for the tip. It seems the non-UTF-8 string literals were inside comments,
anyway. So as a fix, I just iconv'd the file to UTF-8 and no more
errors/warnings. Blissfully quiet
Comment 5 Ed Avis 2003-05-08 17:26:00 EDT
A fixed source RPM for Date::Manip is at
<http://atrpms.physik.fu-berlin.de/dist/rh9/perl-DateManip/>.
Comment 6 Michael Schwendt 2003-09-28 16:43:55 EDT
For cross-reference:
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=105866#c2

Comment 7 Miloslav Trmac 2004-03-01 04:16:05 EST
*** Bug 88017 has been marked as a duplicate of this bug. ***
Comment 8 Need Real Name 2004-04-15 05:53:52 EDT
Created attachment 99440 [details]
Script to find Perl Modules that iconv dislikes

This script will give a listing of Perl Modules that iconv doesn't like.
This might be useful for fixing other modules.

I'll create another bug if people think this is useful.
Comment 9 Warren Togami 2005-09-11 07:42:33 EDT
Is "non-UTF-8 string literals were inside comments" still fatal in modern perl,
and if so would something like the above script be useful in an automated test
to avoid this problem in our future perl packages?
Comment 10 Jason Vas Dias 2006-01-19 17:33:50 EST
This is definitely no longer a bug with all current perl versions in all
Red Hat linux releases, including the perl-5.8.0+ in RHEL-3 .
 o perl no longer complains about invalid UTF8 in comments
 o The Date::Manip module no longer contains invalid UTF-8
This bug can now be closed.

Note You need to log in before you can comment on or make changes to this bug.