Bug 1484497
| Summary: | UnicodeDecodeError in localeconv() makes test_float fail in Koji | ||
|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | Petr Viktorin (pviktori) <pviktori> |
| Component: | python3 | Assignee: | Charalampos Stratakis <cstratak> |
| Status: | CLOSED ERRATA | QA Contact: | Fedora Extras Quality Assurance <extras-qa> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | rawhide | CC: | bkabrda, cstratak, ishcherb, mcyprian, mhroncok, pviktori, rkuska, tomspur, torsava |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | python3-3.6.4-6.fc27 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2018-01-26 18:10:04 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
I was able to reproduce this with the self-contained SPEC file on rawhide, but not on f27 or f26. rawhide: https://koji.fedoraproject.org/koji/taskinfo?taskID=21425550 (failed) f27: https://koji.fedoraproject.org/koji/taskinfo?taskID=21425571 (completed) f26: https://koji.fedoraproject.org/koji/taskinfo?taskID=21425582 (completed) Package diff: -bash.x86_644.4.12-9 +bash.x86_644.4.12-10 -cracklib.x86_642.9.6-7 +cracklib.x86_642.9.6-9 -crypto-policies.noarch20170816-1.git2618a6c +crypto-policies.noarch20170823-1.git8d18c27 -curl.x86_647.55.1-2 -libcurl.x86_647.55.1-2 +curl.x86_647.55.1-3 +libcurl.x86_647.55.1-3 -file-libs.x86_645.31-9 +file-libs.x86_645.31-10 -file.x86_645.31-9 +file.x86_645.31-10 -glibc-all-langpacks.x86_642.26-4 -glibc-common.x86_642.26-4 -glibc-devel.x86_642.26-4 -glibc-headers.x86_642.26-4 -glibc.x86_642.26-4 +glibc-all-langpacks.x86_642.26.90-9 +glibc-common.x86_642.26.90-9 +glibc-devel.x86_642.26.90-9 +glibc-headers.x86_642.26.90-9 +glibc.x86_642.26.90-9 -gnutls.x86_643.5.15-1 +gnutls.x86_643.6.0-1 -kernel-headers.x86_644.13.0-0.rc5.git1.1 +kernel-headers.x86_644.13.0-0.rc6.git1.1 -krb5-libs.x86_641.15.1-22 +krb5-libs.x86_641.15.1-24 -libcap-ng.x86_640.7.8-5 +libcap-ng.x86_640.7.8-7 -libcrypt-nss.x86_642.26-4 +libcrypt-nss.x86_642.26.90-9 -libselinux.x86_642.7-1 -libsemanage.x86_642.7-1 +libselinux.x86_642.7-3 +libsemanage.x86_642.7-3 -libxml2.x86_642.9.4-4 +libxml2.x86_642.9.4-5 -setup.noarch2.10.5-3 +setup.noarch2.10.7-1 I bet the change in glibc is relevant. I can verify that the issue is with the latest glibc and more specifically with the 2.26.90 version. My tests compared version 2.26-15.fc27.x86_64 to 2.26.90-3.fc28 [0]. The easiest way to reproduce it, is to initialize a F27 chroot or VM, run the test (it will pass on this step) and then upgrade the five glibc* packages that come with the base install to those from [0] and rerun the test (it will fail here). dnf will complain during upgrading so it's better to wget the rpm's and then install them through 'rpm -Uvh <rpm's>'. [0] https://koji.fedoraproject.org/koji/buildinfo?buildID=956358 A PR has been submitted upstream which fixes the issue [0]. Waiting on it to be merged before backporting it. [0] https://github.com/python/cpython/pull/4174 python3-3.6.4-6.fc27 has been submitted as an update to Fedora 27. https://bodhi.fedoraproject.org/updates/FEDORA-2018-ff1b193de3 python3-3.6.4-6.fc27 has been pushed to the Fedora 27 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-ff1b193de3 python3-3.6.4-6.fc27 has been pushed to the Fedora 27 stable repository. If problems still persist, please make note of it in this bug report. |
I can reproduce this bug only in Koji, and in Mock if the buildroot is cleaned immediately before. I don't know how to reproduce it on a system/chroot I can SSH into. Under certain circumstances, when LC_NUMERIC is fr_FR.ISO8859-1 but LC_ALL is C.UTF-8, locale.localeconv() fails with UnicodeDecodeError: 'locale' codec can't decode byte 0xa0 in position 0: Invalid or incomplete multibyte or wide character Apparently, the thousands separator (or something else) in the lconv is "\xa0" (unbreakable space in fr_FR.ISO8859-1), and it's being decoded with UTF-8. This is tripped by Python's test suite, namely test_float.GeneralFloatCases.test_float_with_comma One failed Koji build is here: https://koji.fedoraproject.org/koji/taskinfo?taskID=21419062 A smaller reproducer is here (self-contained SPEC file): https://gist.github.com/encukou/70b3d3f9ef3e29ac1dbc23a5f7bd6431