Bug 135428 - popt : -help outputs unaligned descriptions, in case of <cellwidth> != <byte length>
popt : -help outputs unaligned descriptions, in case of <cellwidth> != <byte ...
Status: CLOSED CANTFIX
Product: Fedora
Classification: Fedora
Component: popt (Show other bugs)
rawhide
i386 Linux
medium Severity medium
: ---
: ---
Assigned To: Caius Chance
Fedora Extras Quality Assurance
: FutureFeature, Patch
Depends On: 249352
Blocks:
  Show dependency treegraph
 
Reported: 2004-10-12 13:16 EDT by sangu
Modified: 2009-07-07 20:29 EDT (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2009-07-07 20:29:19 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
popt-multibyte-help-width.diff (1.85 KB, patch)
2004-10-12 13:21 EDT, sangu
no flags Details | Diff
new "popt-multibyte-help-width.diff" (2.24 KB, patch)
2004-10-12 13:41 EDT, sangu
no flags Details | Diff
output in ko locale (74.29 KB, image/png)
2005-11-18 02:02 EST, Lawrence Lim
no flags Details
popt ouput in the zh_TW locale (83.08 KB, image/png)
2005-11-18 02:04 EST, Lawrence Lim
no flags Details
screenshot for zh_CN (103.97 KB, image/png)
2007-09-13 08:29 EDT, A S Alam
no flags Details
Sample from `$ LANG=zh_TW.UTF-8 rpm --help > ~/Desktop/sample` (9.29 KB, application/octet-stream)
2008-03-27 01:02 EDT, Caius Chance
no flags Details
screenshot on 27/03/08 (619.79 KB, image/png)
2008-03-27 01:05 EDT, Caius Chance
no flags Details

  None (edit)
Description sangu 2004-10-12 13:16:51 EDT
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; rv:1.7.3) Gecko/20041012
Firefox/0.10.1

Description of problem:
It's because popt assumes that a string's byte length means string's
cell width in  terminal.  Though this is true in many legacy encodings
(ISO8859-*, EUC-*), not in UTF-8.

Patch attached.  This patch makes popt use correct cell widths with
mbstowcs()/wcswidth().





Version-Release number of selected component (if applicable):
popt-1.9.1-12

How reproducible:
Always

Steps to Reproduce:
1. Generate ko_KR.UTF-8 or zh_TW.UTF-8 locale.

2. Run this command (or other Gnome program) in a UTF-8 capable
terminal: 
$ LANG=zh_TW.UTF-8 gnome-session --help

3. Output looks bad; descriptions' starting column numbers are not same.

Additional info:

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=275234
Comment 1 sangu 2004-10-12 13:21:05 EDT
Created attachment 105079 [details]
popt-multibyte-help-width.diff

This patch makes popt use correct cell widths with mbstowcs()/wcswidth().

Changwoo Ryu <cwryu@debian.org>
Comment 2 sangu 2004-10-12 13:31:13 EDT
Comment on attachment 105079 [details]
popt-multibyte-help-width.diff

--- popt-1.7.orig/popthelp.c	2002-09-16 22:16:54.000000000 +0900

+++ popt-1.7/popthelp.c 2004-10-07 08:13:26.000000000 +0900

@@ -187,6 +187,49 @@

     return l;

 }



+#include <wchar.h>

+

+static int mbswidth(const char *str)

+{

+    int width, wcslen;

+    wchar_t *wcs;

+

+    wcslen = mbstowcs(NULL, str, 0);

+    if (wcslen > 0) {

+	wcs = malloc(sizeof(wchar_t) * (wcslen + 1));

+	wcslen = mbstowcs(wcs, str, wcslen);

+	width = wcswidth(wcs, wcslen);

+	free(wcs);

+    } else

+	width = strlen(str);

+    return width;

+}

+

+static int findLengthInWidth(const char *str, int width)

+{

+    const char *s = str;

+    wchar_t wc;

+    mbstate_t state = { 0 };

+    int w, len, max = 0;

+

+    while (*s) {

+	len = mbrtowc(&wc, s, strlen(s), &state);

+	if (len < 0) {

+	    /* strange situation, fallback to the byte length.	*/

+	    return strlen(str);

+	}

+	w = wcwidth(wc);

+	if (w > 0) {

+	    if ((max + w) > width)

+		break;

+	    max += w;

+	}

+	s += len;

+    }

+    return (s - str);

+    

+}

+

 /**

  * Display help text for an option.

  * @param fp		output file handle

@@ -317,7 +360,7 @@

 /*@=boundswrite@*/



     if (help)

-	fprintf(fp,"  %-*s   ", maxLeftCol, left);

+	fprintf(fp,"  %-*s   ", maxLeftCol+strlen(left)-mbswidth(left), left);

     else {

	fprintf(fp,"  %s\n", left); 

	goto out;

@@ -328,13 +371,13 @@

	help = defs; defs = NULL;

     }



-    helpLength = strlen(help);

+    helpLength = mbswidth(help);

 /*@-boundsread@*/

     while (helpLength > lineLength) {

	const char * ch;

	char format[16];



-	ch = help + lineLength - 1;

+	ch = help + findLengthInWidth(help, lineLength - 1);

	while (ch > help && !isspace(*ch)) ch--;

	if (ch == help) break;		/* give up */

	while (ch > (help + 1) && isspace(*ch)) ch--;

@@ -346,7 +389,7 @@

	/*@=formatconst@*/

	help = ch;

	while (isspace(*help) && *help) help++;

-	helpLength = strlen(help);

+	helpLength = mbswidth(help);

     }

 /*@=boundsread@*/



@@ -389,7 +432,7 @@



	    s = getArgDescrip(opt, translation_domain);

	    if (s)

-		len += sizeof("=")-1 + strlen(s);

+		len += sizeof("=")-1 + mbswidth(s);

	    if (opt->argInfo & POPT_ARGFLAG_OPTIONAL) len += sizeof("[]")-1;

	    if (len > max) max = len;

	}
Comment 3 sangu 2004-10-12 13:41:13 EDT
Created attachment 105083 [details]
new "popt-multibyte-help-width.diff"

New Patch

by Changwoo Ryu <cwryu@debian.org>
Comment 4 Jeff Johnson 2004-10-12 17:36:39 EDT
I believe the alignment has already been corrected in
a later version of popt. Could you verify whether that
is true?
Comment 5 sangu 2004-10-12 21:43:28 EDT
popt-1.9.1-12.

gnome-terminal --help in ko_KR.UTF-8
http://hellocity.net/~sangu/files/popt/ko_KR.UTF-8.png

gnome-terminal --help in en_US.UTF-8
http://hellocity.net/~sangu/files/popt/en_US.UTF-8.png
Comment 6 Jens Petersen 2005-09-27 09:20:33 EDT
Links in last comment are no longer there apparently.

Any better in fc4?
Comment 7 Lawrence Lim 2005-11-18 02:01:11 EST
Tested with popt-1.10.2-6 in ko and zh_TW, problem still exist. No problem in
en_US though.


Question:
Why is the component assigned to rpm instead of popt?
Comment 8 Lawrence Lim 2005-11-18 02:02:57 EST
Created attachment 121217 [details]
output in ko locale
Comment 9 Lawrence Lim 2005-11-18 02:04:03 EST
Created attachment 121218 [details]
popt ouput in the zh_TW locale
Comment 10 Hu Zheng 2007-02-08 00:45:46 EST
I think we should mark this bug as won't fix or can't fix.
Yes, the bytes of a chracter don't means the width of a character, such Russian
characters, ans this patch solve this problem, but this patch assume each
character's width is the same, this is true for Russian, as the width of Russian
character and English character is the same, but not true for Chinese, one
Chinese character 's width in the gnome terminal seem being twice as the English
character.
So how to determine the width of charaters? By byte, true for English; By
character, true for Russian. Then how to be true for Chinese? We can assume it
as twice as English, but this is not a good solution too as determine this is
not simple and common to other languages.
Comment 11 Jeff Johnson 2007-02-08 15:58:03 EST
See comment #4.

The latest released popt is 1.10.7.

I've asked if the problem still exists because I believe it is already fixed.

If not, I'll be happy to fix.

NEEDINFO
Comment 12 Jeff Johnson 2007-02-08 16:02:21 EST
If not fixed, I'll also need some help getting a reproducer together.
Comment 13 Robert Scheck 2007-08-10 06:08:46 EDT
Does this problem exist in popt 1.12 which I added for review (bug #249352), 
too? Please make a note, if the problem still exists otherwise please close 
this bug report. Thank you.
Comment 14 Red Hat Bugzilla 2007-08-21 01:18:58 EDT
User pnasrat@redhat.com's account has been closed
Comment 15 Panu Matilainen 2007-08-22 02:30:39 EDT
Reassigning to owner after bugzilla made a mess, sorry about the noise...
Comment 16 Robert Scheck 2007-08-23 19:27:17 EDT
Lawrence, popt 1.12-3 should reach Rawhide soon. Can you please have a look, 
whether the problem still exists? Any suggestions whether this bug report can 
be really resolved, when it isn't until now?!
Comment 17 A S Alam 2007-09-13 08:29:27 EDT
Created attachment 194591 [details]
screenshot for zh_CN

I tried to test with zh_CN
Please update me if I have no understand real bug, but it seems it not
yet fixed. 
Tested with following RPM:
popt-1.12-3.fc8
Comment 18 Robert Scheck 2007-12-30 17:15:05 EST
I don't think, the latest popt update really solves the problem, but noticing 
here: https://admin.fedoraproject.org/updates/F8/pending/popt-1.13-1.fc8

I would tend to Hu Zheng's suggestion to mark this bug as CANTFIX, if no real
solution comes up which works for every case.
Comment 19 Caius Chance 2008-03-27 01:02:06 EDT
Created attachment 299270 [details]
Sample from `$ LANG=zh_TW.UTF-8 rpm --help > ~/Desktop/sample`

$ rpm -qa |grep popt
popt-1.13-3.fc9.i386

$ uname -a
Linux cavalier 2.6.23.1-42.fc8 #1 SMP Tue Oct 30 13:55:12 EDT 2007 i686 athlon
i386 GNU/Linux

Fedora 9 (devel / rawhide) on 27 Mar 2008.
Comment 20 Caius Chance 2008-03-27 01:04:23 EDT
Hi Hu Zheng, I have just triaged this once again on Fedora 9 (devel / rawhide).
Please kindly advice if this bug is reproducible for bug closure decision. IMHO
the mis-alignment is because of mixed usage of spaces and tabs? Rgds, Caius.
Comment 21 Caius Chance 2008-03-27 01:05:38 EDT
Created attachment 299271 [details]
screenshot on 27/03/08
Comment 22 Tony Fu 2008-09-09 23:14:24 EDT
requested by Jens Petersen (#27995)
Comment 23 Caius Chance 2009-02-15 13:43:24 EST
ping
Comment 24 Robert Scheck 2009-02-15 14:31:55 EST
Cairus, I don't know how I should solve this problem as downstream maintainer.

And upstream told me, that a native JA/KO person would be needed also having 
some GNOME knowledge in order to get this hopefully really solved. Let me know
if I can do something more for you. Upstream contacts are e.g. mentioned in
the popt package itself.
Comment 25 Caius Chance 2009-02-15 20:00:56 EST
(In reply to comment #18)
> I would tend to Hu Zheng's suggestion to mark this bug as CANTFIX, if no real
> solution comes up which works for every case.

Shall we?

Note You need to log in before you can comment on or make changes to this bug.