Bug 1494539 - In ja_JP mode, EUC-JP Japanese characters are discarded
Summary: In ja_JP mode, EUC-JP Japanese characters are discarded
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Fedora EPEL
Classification: Fedora
Component: squirrelmail
Version: epel7
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ---
Assignee: Michal Hlavinka
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-09-22 13:04 UTC by Takashi Ichihara
Modified: 2019-05-07 08:04 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-05-07 08:04:34 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
Display sample image of squirrelmail-1.4.22-16.el7 of Japanese email (148.78 KB, image/png)
2017-09-22 13:04 UTC, Takashi Ichihara
no flags Details

Description Takashi Ichihara 2017-09-22 13:04:47 UTC
Created attachment 1329581 [details]
Display sample image of squirrelmail-1.4.22-16.el7 of Japanese email

Description of problem:
 In ja_JP mode, EUC-JP characters are discarded

Version-Release number of selected component (if applicable):

 squirrelmail-1.4.22-16.el7.noarch.rpm 

How reproducible:

 Always
 In ja_JP mode, EUC-JP characters are discarded
 Sample image of the display is attached as png file

Steps to Reproduce:
 1. set Default Language: ja_JP,  Charset: iso-2022-jp
 2. Display Japanese mail message (subject or/and main text) of
    EUC-JP characters

Actual results:
Expected results:
Additional info:

This problem and workaround is described in Japanese at the URL of
https://blog.development-network.net/ung/2017/06/squirrelmail.html
http://taka2.info/20120705/squirrelmail-php54/

In short, following modification of the source code works to fix this problem.

1) Replace of the function:  htmlspecialchars  ->  sq_htmlspecialchars

( at the /usr/share/squirrelmail directoy)
% find . -name '*.php' -print \
 | xargs egrep -l 'htmlspecialchars *\([^\)]' \
 | egrep -v 'global.php|configtest.php|login.php|class/|contrib/' \
 | xargs perl -i.bak -pe 's/htmlspecialchars *\([^\)]/sq_$&/'

2) Add a new function  sq_htmlspecialchars

% cat >> functions/i18n.php
// wrapper of htmlspecialchars
// In PHP 5.4, defualt value of htmlspecialchars 3rd parameter is changed
// from 'ISO8859-1' to 'UTF-8',
// but if in ja_JP mode, html data encoding is 'EUC-JP', not 'UTF-8'.
// htmlspecialchars discard invalid encoding characters,
// so in ja_JP mode, EUC-JP characters are discarded.
function sq_htmlspecialchars($string) {
    $encoding = 'iso8859-1';
    if (function_exists('mb_internal_encoding')) {
        $encoding = mb_internal_encoding();
    }
    return htmlspecialchars($string, ENT_COMPAT | ENT_HTML401, $encoding);
}
^D


I confirmed that following to the workaround above, the problem was solved and any Japanese character can be displayed normally at default Language: ja_JP   Charset: iso-2022-jp on servral servers of RHEL 7.3.

Comment 1 Takashi Ichihara 2017-09-26 01:48:28 UTC
At the Web site of SquirrelMail Project
https://squirrelmail.org/

It is described as follows at the beginning:

—
ANNOUNCE: PHP 5.4 & 5.5 Compatibility
May 30, 2013 by Paul Lesniewski
 PHP 5.4 and PHP 5.5 compatibility fixes are live in our nightly snapshots, 
and we strongly encourage those who'd like to help test or those who 
are having problems with version 1.4.22 to download a snapshot 
and report any problems.
—

This problem was fixed at snapshot of SquirrelMail 1.4.23-svn
as can be found at the URL of
https://squirrelmail.org/download.php

Stable version snapshots (1.4.23-svn)
squirrelmail-20170926_0200-SVN.stable.tar.gz

Comment 2 Takashi Ichihara 2019-05-07 08:04:34 UTC
Hi,

I noticed that this problem and other serious problem (sub-folder problem) appeared in PHP-5.4 have been all fixed in the

http://squirrelmail.org/download.php

Stable version snapshots (1.4.23-svn)
squirrelmail-20190507_0200-SVN.stable.tar.gz 


  Therefore, I suggest to starting from

squirrelmail-20190xxx_0200-SVN.stable.tar.gz

not from old problematic release

squirrelmail-webmail-1.4.22.tar.gz 

in the next EPEL package of squirrelmail.


Note You need to log in before you can comment on or make changes to this bug.