Bug 1256244

Summary: Transliteration not working on Marathi language
Product: [Fedora] Fedora Reporter: Pravin Satpute <psatpute>
Component: ibus-typing-boosterAssignee: Mike FABIAN <mfabian>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 23CC: anish.developer, harshula, i18n-bugs, mfabian
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 1.7.0-4.fc23 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-09-01 18:30:57 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
m17n-lib-1.7.0-fix-crash-in-marathi-transliteration.patch
none
m17n-lib-1.7.0-fix-crash-in-marathi-transliteration.patch none

Description Pravin Satpute 2015-08-24 07:17:46 UTC
Description of problem:
Transliteration not happening for m17n itrans maps languages.

Version-Release number of selected component (if applicable):
ibus-typing-booster-1.2.11-3.fc23.noarch

How reproducible:
/usr/share/myspell/mr_IN.dic

Steps to Reproduce:
1. Install hunspell-mr
2. Select Marathi Hunspell
3. From Setting select Itrans layout
4. Type any word. say 'praviN' 

Actual results:
Getting in Latin 'praviN'

Expected results:
प्रविण 

Additional info:
looks like issue with libtranslit. Inscript layout working fine.

Comment 1 Mike FABIAN 2015-08-24 12:31:22 UTC
When using the itrans layout and type “praviN”, I get:

    प्रविण् 

Isn’t that right?

I certainly don’t get Latin.

Comment 2 Pravin Satpute 2015-08-24 12:48:01 UTC
yes. that's correct transliteration. I am not getting same. I also verified on other machine as well.

libtranslit-0.0.3-9.fc23.x86_64
hunspell-mr-1.0.0-3.fc23.noarch

Same time it works perfectly with Inscript.

Comment 3 Mike FABIAN 2015-08-24 12:50:42 UTC
I have:

$ rpm -q libtranslit hunspell-mr ibus-typing-booster
libtranslit-0.0.3-8.fc22.x86_64
hunspell-mr-1.0.0-2.fc21.noarch
ibus-typing-booster-1.2.11-1.fc22.noarch
$ 

Trying on F23 now ...

Comment 4 Mike FABIAN 2015-08-24 13:27:57 UTC
On F23, it does *not* work ☹. 

A crash occurs, see: https://bugzilla.redhat.com/show_bug.cgi?id=1256396

Comment 5 Mike FABIAN 2015-08-25 06:13:10 UTC
*** Bug 1256396 has been marked as a duplicate of this bug. ***

Comment 6 Mike FABIAN 2015-08-25 06:17:24 UTC
On Fedora 22:

    $ python3
    Python 3.4.2 (default, Jul  9 2015, 17:24:30) 
    [GCC 5.1.1 20150618 (Red Hat 5.1.1-4)] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> from gi.repository import Translit
    >>> trans=Translit.Transliterator.get('m17n', 'mr-itrans')
    >>> trans.transliterate('pravinN')
    ('प्रविन्ण्', 7)
    >>> 

On Fedora 23:

    $ python3
    Python 3.4.3 (default, Jun 29 2015, 12:16:01) 
    [GCC 5.1.1 20150618 (Red Hat 5.1.1-4)] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> from gi.repository import Translit
    __main__:1: PyGIWarning: Translit was imported without specifying a version first. Use gi.require_version('Translit', '1.0') before import to ensure that the right version gets loaded.
    >>> trans=Translit.Transliterator.get('m17n', 'mr-itrans')
    >>> trans.transliterate('pravinN')
    Segmentation fault (?????)

Comment 7 Mike FABIAN 2015-08-25 12:21:36 UTC
The problem happens no matter whether GObject introspection or
ctypes is used:

[mfabian@Fedora-Workstation-netinst-x86_6 ~]$ cat /etc/fedora-release 
cat /etc/fedora-release 
Fedora release 23 (Twenty Three)
[mfabian@Fedora-Workstation-netinst-x86_6 ~]$ cat transliterator-gi.py 
cat transliterator-gi.py 
import gi
gi.require_version('Translit', '1.0')
from gi.repository import Translit
trans = Translit.Transliterator.get('m17n', 'mr-itrans')
result = trans.transliterate('praviN')
print(result[0])
print(result[0].encode('utf-8'))
[mfabian@Fedora-Workstation-netinst-x86_6 ~]$ python3 transliterator-gi.py 
python3 transliterator-gi.py 
Segmentation fault (core dumped)
[mfabian@Fedora-Workstation-netinst-x86_6 ~]$ cat transliterator-ctypes.py 
cat transliterator-ctypes.py 
import ctypes

class Transliterator(object):
    __libtranslit = ctypes.CDLL("libtranslit.so.0",
                                mode=ctypes.RTLD_GLOBAL)

    __get = __libtranslit.translit_transliterator_get
    __get.argtypes = [ctypes.c_char_p,
                      ctypes.c_char_p]
    __get.restype = ctypes.c_void_p

    __transliterate = __libtranslit.translit_transliterator_transliterate
    __transliterate.argtypes = [ctypes.c_void_p,
                                ctypes.c_char_p,
                                ctypes.c_void_p]
    __transliterate.restype = ctypes.c_char_p

    def __init__(self, trans):
        self.__trans = trans

    @staticmethod
    def get(backend, name):
        return Transliterator(Transliterator.__get(
            ctypes.c_char_p(backend.encode('utf-8')),
            ctypes.c_char_p(name.encode('utf-8'))))

    def transliterate(self, _input):
        endpos = ctypes.c_ulong()
        output = Transliterator.__transliterate(
            self.__trans,
            ctypes.c_char_p(_input.encode('utf-8')),
            ctypes.byref(endpos))
        return (output.decode('utf-8'), endpos.value)

trans = Transliterator.get('m17n', 'mr-itrans')
result = trans.transliterate('praviN')
print(result[0])
print(result[0].encode('utf-8'))

[mfabian@Fedora-Workstation-netinst-x86_6 ~]$ python3 transliterator-ctypes.py 
<-netinst-x86_6 ~]$ python3 transliterator-ctypes.py 
Segmentation fault (core dumped)
[mfabian@Fedora-Workstation-netinst-x86_6 ~]$

On Fedora 22, both GObject Introspection and ctypes work and produce
identical results.

Comment 8 Mike FABIAN 2015-08-26 15:35:06 UTC
Created attachment 1067319 [details]
m17n-lib-1.7.0-fix-crash-in-marathi-transliteration.patch

Comment 9 Mike FABIAN 2015-08-27 13:05:59 UTC
Created attachment 1067739 [details]
m17n-lib-1.7.0-fix-crash-in-marathi-transliteration.patch

First hunk of the patch changed, thanks to Daiki Ueno.

Comment 10 Mike FABIAN 2015-08-27 13:06:55 UTC
Packages for testing with the patch from comment#9 are here: 

https://mfabian.fedorapeople.org/m17n-lib/

Comment 11 Pravin Satpute 2015-08-28 09:23:26 UTC
Working perfect now with updated m17n-libs and ibus-typing-booster

Comment 12 Mike FABIAN 2015-08-31 06:10:49 UTC
My patch seems to be OK according to Kenichi Handa:

https://lists.nongnu.org/archive/html/m17n-list/2015-08/msg00002.html

Comment 13 Fedora Update System 2015-08-31 06:37:12 UTC
m17n-lib-1.7.0-2.fc22 has been submitted as an update to Fedora 22. https://bodhi.fedoraproject.org/updates/FEDORA-2015-14621

Comment 14 Fedora Update System 2015-08-31 06:42:21 UTC
m17n-lib-1.7.0-4.fc23 has been submitted as an update to Fedora 23. https://bodhi.fedoraproject.org/updates/FEDORA-2015-14622

Comment 15 Fedora Update System 2015-09-01 09:58:25 UTC
m17n-lib-1.7.0-2.fc22 has been pushed to the Fedora 22 testing repository. If problems still persist, please make note of it in this bug report.\nIf you want to test the update, you can install it with \n su -c 'yum --enablerepo=updates-testing update m17n-lib'. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-14621

Comment 16 Fedora Update System 2015-09-01 10:04:53 UTC
m17n-lib-1.7.0-4.fc23 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report.\nIf you want to test the update, you can install it with \n su -c 'yum --enablerepo=updates-testing update m17n-lib'. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-14622

Comment 17 Fedora Update System 2015-09-01 18:30:56 UTC
m17n-lib-1.7.0-4.fc23 has been pushed to the Fedora 23 stable repository. If problems still persist, please make note of it in this bug report.

Comment 18 Fedora Update System 2015-10-01 18:51:38 UTC
m17n-lib-1.7.0-2.fc22 has been pushed to the Fedora 22 stable repository. If problems still persist, please make note of it in this bug report.