Hide Forgot
Description of problem: I tried to use hu-rovas-post.mim with the hu-HU engine of ibus-typing-booster Version-Release number of selected component: ibus-typing-booster-1.5.0-1.fc24 Additional info: reporter: libreport-2.7.2 backtrace_rating: 4 cmdline: /usr/bin/python3 /usr/share/ibus-typing-booster/engine/main.py --profile --ibus crash_function: SuggestMgr::leftcommonsubstring(char*, char const*) executable: /usr/bin/python3.5 global_pid: 9190 kernel: 4.6.4-301.fc24.x86_64 pkg_fingerprint: 73BD E983 81B4 6521 pkg_vendor: Fedora Project runlevel: N 5 type: CCpp uid: 10030 Truncated backtrace: Thread no. 0 (10 frames) #0 SuggestMgr::leftcommonsubstring(char*, char const*) at suggestmgr.cxx:1859 #1 SuggestMgr::ngsuggest(char**, char*, int, HashMgr**, int) at suggestmgr.cxx:1106 #2 Hunspell::suggest(char***, char const*) at hunspell.cxx:889 #3 MySpellChecker::suggestWord(char const*, unsigned long, unsigned long*) at myspell_checker.cpp:197 #4 enchant_dict_suggest at enchant.c:943 #5 ffi_call_unix64 at ../src/x86/unix64.S:76 #6 ffi_call at ../src/x86/ffi64.c:525 #7 _ctypes_callproc at /usr/src/debug/Python-3.5.1/Modules/_ctypes/callproc.c:811 #9 PyCFuncPtr_call at /usr/src/debug/Python-3.5.1/Modules/_ctypes/_ctypes.c:3869 #10 PyObject_Call at /usr/src/debug/Python-3.5.1/Objects/abstract.c:2165
Created attachment 1192545 [details] File: backtrace
Created attachment 1192546 [details] File: cgroup
Created attachment 1192547 [details] File: core_backtrace
Created attachment 1192548 [details] File: dso_list
Created attachment 1192549 [details] File: environ
Created attachment 1192550 [details] File: exploitable
Created attachment 1192551 [details] File: limits
Created attachment 1192552 [details] File: maps
Created attachment 1192553 [details] File: mountinfo
Created attachment 1192554 [details] File: namespaces
Created attachment 1192555 [details] File: open_fds
Created attachment 1192556 [details] File: proc_pid_status
Created attachment 1192557 [details] File: var_log_messages
Created attachment 1192563 [details] python-enchant-crash.py It crashes, because python3-enchant crashes: $ python3 python-enchant-crash.py ['Budapest', 'Budapesti', 'Budapesté', 'Budapestű'] [b'Budapest', b'Budapesti', b'Budapest\xc3\xa9', b'Budapest\xc5\xb1'] This UTF-8 encoding can't convert to UTF-16: 𐲂𐳪𐳇𐳀𐳠𐳉𐳤𐳦 This UTF-8 encoding can't convert to UTF-16: 𐲂𐳪𐳇𐳀𐳠𐳉𐳤𐳦 This UTF-8 encoding can't convert to UTF-16: 𐲂𐳪𐳇𐳀𐳠𐳉𐳤𐳦 Segmentation fault (コアダンプ) mfabian@ari:~ $
Created attachment 1192564 [details] hunspell-conversion-problem.txt python3-enchant probably crashes because of this problem in hunspell: $ hunspell -d hu_HU -i utf-8 -l hunspell-conversion-problem.txt hBudapxst This UTF-8 encoding can't convert to UTF-16: 😇 This UTF-8 encoding can't convert to UTF-16: 😇 This UTF-8 encoding can't convert to UTF-16: 𐳠 This UTF-8 encoding can't convert to UTF-16: 𐳠 mfabian@ari:~ $ Of course the file converts to UTF-16 just fine: $ iconv -f utf-8 -t utf-16 < hunspell-conversion-problem.txt | iconv -f utf-16 -t utf-8 Budapxst 😇 𐳠 mfabian@ari:~ $ It looks like hunspell has problems with characters above the BMP (Basic Multilingual Plane).
It works on current rawhide with hunspell-1.4.1: [mfabian@Fedora-Workstation-netinst-x86_6 ~]$ cat /etc/fedora-release Fedora release 26 (Rawhide) [mfabian@Fedora-Workstation-netinst-x86_6 ~]$ python3 python-enchant-crash.py ['Budapest', 'Budapesti', 'Budapesté', 'Budapestű'] [b'Budapest', b'Budapesti', b'Budapest\xc3\xa9', b'Budapest\xc5\xb1'] [] [] [mfabian@Fedora-Workstation-netinst-x86_6 ~]$ rpm -q hunspell hunspell-1.4.1-1.fc25.x86_64 [mfabian@Fedora-Workstation-netinst-x86_6 ~]$ hunspell -d hu_HU -i utf-8 -l hunspell-conversion-problem.txt Budapxst [mfabian@Fedora-Workstation-netinst-x86_6 ~]$
Can the fix be backported to f24?
Good to know the big rework of stuff in hunspell had a practical worthwhile effect. I'll have to bisect to find when it started working to see what exactly was the cause to see if its backportable in isolation.
hunspell-1.3.3-10.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2016-1a8b18ee44
hunspell-1.3.3-10.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-1a8b18ee44
hunspell-1.3.3-10.fc24 has been pushed to the Fedora 24 stable repository. If problems still persist, please make note of it in this bug report.