Bug 1584647 - vim manpage got garbled in Japanese locale
Summary: vim manpage got garbled in Japanese locale
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: vim
Version: 31
Hardware: All
OS: Linux
unspecified
unspecified
Target Milestone: ---
Assignee: Karsten Hopp
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-05-31 11:36 UTC by IWATA Mutsumi
Modified: 2023-09-14 04:29 UTC (History)
3 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2019-08-19 12:08:53 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description IWATA Mutsumi 2018-05-31 11:36:06 UTC
Description of problem:
In Red Hat Bugzilla #1035606, the problem of garbled characters in some man pages has been resolved by using file command.
But recently, because of incorrect determine in file command, vim's Japanese man page garbled again.
I propose that add an option to file command in vim.spec, to determine file type without magic file.

Version-Release number of selected component (if applicable):


How reproducible:
$ LC_MESSAGES=ja_JP.utf8 man vim


Actual results:

Not displayed as Japanese, as below.

VIM(1)                        General Commands Manual                        VIM(1)

a<U+0090><U+008D>a<U+0089><U+008D>
       vim - Vi IMproved, a<U+0083><U+0097>a<U+0083>a<U+0082>°a<U+0083>ca<U+0083>
<U+009E>a<U+0081>Ra<U+0083><U+0086>a<U+0082>a<U+0082>1a<U+0083><U+0088>a<U+0082>¨a
<U+0083><U+0087>a<U+0082>£a<U+0082>?

a<U+009B>,a?<U+008F>
       vim [options] [file ..]
       vim [options] -
       vim [options] -t tag
       vim [options] -q [errorfile]

       ex
       view
       gvim gview evim eview
       rvim rview rgvim rgview

ea¬a<U+0098><U+008E>
       Vim         a<U+0081> ̄         Vi         ao<U+0092>a<U+008F><U+009B>a<U+0081>Ra<U+0083><U+0086>a<U+0082>a<U+0082>1a<U+0083><U+0088>a<U+0082>¨a<U+0083><U+0087>a
<U+0082>£a<U+0082>?a<U+0081>§a<U+0081><U+0099>a<U+0080><U+0082>


Expected results:

Displayed as Japanese.

VIM(1)                        General Commands Manual                        VIM(1)

名前
       vim - Vi IMproved, プログラマのテキストエディタ

書式
       vim [options] [file ..]
       vim [options] -
       vim [options] -t tag
       vim [options] -q [errorfile]

       ex
       view
       gvim gview evim eview
       rvim rview rgvim rgview

説明
       Vim  は Vi 互換のテキストエディタです。 どのような種類のプレインテキストでも
       編集できます。 特に、プログラムの編集に力を発揮します。


Additional info:

In the progress of building the vim-common rpm packages, there is a process of converting the document files included in the Vim upstream source code into UTF-8.
showing below lines of 513 to 521 in vim.spec.

 513 ( cd %{buildroot}/%{_mandir}
 514   for i in `find ??/ -type f`; do
 515     if [[ "`file $i`" == *UTF-8\ Unicode\ text* ]]; then
 516       continue
 517     fi
 518     bi=`basename $i`
 519     iconv -f latin1 -t UTF8 $i > %{buildroot}/$bi
 520     mv -f %{buildroot}/$bi $i
 521   done
 522 )


In executing file command with 'runtime/doc/vim-*.UTF-8.1' included by Vim source, I got the following result.

[iwatam@FedoraPi01 vim81]$ file  runtime/doc/vim-*.UTF-8.1
runtime/doc/vim-de.UTF-8.1: troff or preprocessor input, UTF-8 Unicode text
runtime/doc/vim-fr.UTF-8.1: troff or preprocessor input, UTF-8 Unicode text
runtime/doc/vim-it.UTF-8.1: troff or preprocessor input, UTF-8 Unicode text
runtime/doc/vim-ja.UTF-8.1: DIY-Thermocam raw data (Lepton 2.x), scale 58241-43747, spot sensor temperature -0.000000, color scheme 190, minimum point enabled, maximum point enabled, calibration: offset 0.000000, slope 155727473974181888.000000
runtime/doc/vim-pl.UTF-8.1: troff or preprocessor input, UTF-8 Unicode text
runtime/doc/vim-ru.UTF-8.1: troff or preprocessor input, UTF-8 Unicode text
[iwatam@FedoraPi01 vim81]$

Only runtime/doc/vim-ja.UTF-8.1 is determined as 'DIY-Thermocam raw data'.
Probably, this is incorrect.


In line 515 of vim.spec, if file command determines as 'DIY-Thermocam raw data', not UTF-8 text,
Target file is converted as latin1 to UTF-8.
Actually, a part of build log is below.

+ for i in `find ??/ -type f`
++ file ja/man1/vim.1
+ [[ ja/man1/vim.1: DIY-Thermocam raw data (Lepton 2.x), scale 58241-43747, spot sensor temperature -0.000000, color scheme 190, minimum point enabled, maximum point enabled, calibration: offset 0.000000, slope 155727473974181888.000000 == *UTF-8\ Unicode\ text* ]]
++ basename ja/man1/vim.1
+ bi=vim.1
+ iconv -f latin1 -t UTF8 ja/man1/vim.1
+ mv -f /home/iwatam/rpmbuild/BUILDROOT/vim-8.1.020-1.fc28.arm/vim.1 ja/man1/vim.1


Then, I propose that the solution.
File command has a option '--exclude soft' for suppress file type determine with magic file.

[iwatam@FedoraPi01 vim81]$ file --exclude soft runtime/doc/vim-*.UTF-8.1
runtime/doc/vim-de.UTF-8.1: UTF-8 Unicode text
runtime/doc/vim-fr.UTF-8.1: UTF-8 Unicode text
runtime/doc/vim-it.UTF-8.1: UTF-8 Unicode text
runtime/doc/vim-ja.UTF-8.1: UTF-8 Unicode text
runtime/doc/vim-pl.UTF-8.1: UTF-8 Unicode text
runtime/doc/vim-ru.UTF-8.1: UTF-8 Unicode text
[iwatam@FedoraPi01 vim81]$


Diff of vim.spec and a part of option added build log is as follows.
In adding it, It looks like work well.

--- vim.spec.org        2018-05-23 21:12:13.000000000 +0900
+++ vim.spec    2018-05-27 13:31:34.018396545 +0900
@@ -512,7 +512,7 @@
 rm -f %{buildroot}/%{_datadir}/vim/%{vimdir}/tutor/tutor.gr.utf-8~
 ( cd %{buildroot}/%{_mandir}
   for i in `find ??/ -type f`; do
-    if [[ "`file $i`" == *UTF-8\ Unicode\ text* ]]; then
+    if [[ "`file --exclude soft $i`" == *UTF-8\ Unicode\ text* ]]; then
       continue
     fi
     bi=`basename $i`


+ for i in `find ??/ -type f`
++ file --exclude soft ja/man1/vimdiff.1
+ [[ ja/man1/vimdiff.1: UTF-8 Unicode text == *UTF-8\ Unicode\ text* ]]
+ continue

Comment 1 Zdenek Dohnal 2018-05-31 12:33:37 UTC
Hi Iwata-san,

thank you for reporting this issue and a complete analysis! IMHO it was caused by a change in Vim, because 8.0.1630 doesn't have the issue and 8.0.1842 has the issue. Tracking it down...

Comment 2 Ben Cotton 2019-05-02 21:40:28 UTC
This message is a reminder that Fedora 28 is nearing its end of life.
On 2019-May-28 Fedora will stop maintaining and issuing updates for
Fedora 28. It is Fedora's policy to close all bug reports from releases
that are no longer maintained. At that time this bug will be closed as
EOL if it remains open with a Fedora 'version' of '28'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 28 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 3 Ben Cotton 2019-08-13 16:56:03 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 31 development cycle.
Changing version to '31'.

Comment 4 Ben Cotton 2019-08-13 17:34:55 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 31 development cycle.
Changing version to 31.

Comment 5 Zdenek Dohnal 2019-08-19 12:02:52 UTC
Hi Mutsumi,

I'm sorry for no update for long time. 

Are you still able to reproduce the issue? 

I was not able to reproduce on my F29 and I did not fix the bug in Vim as far as I remember.

Comment 6 Zdenek Dohnal 2019-08-19 12:08:53 UTC
Japanese man page is now recognized correctly by 'file' as uncode text, so it fixed the issue.

Comment 7 Red Hat Bugzilla 2023-09-14 04:29:14 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days


Note You need to log in before you can comment on or make changes to this bug.