Bug 1457786 - rpmlint spams with "Python runtime initialized with LC_CTYPE=C"
Summary: rpmlint spams with "Python runtime initialized with LC_CTYPE=C"
Keywords:
Status: CLOSED DUPLICATE of bug 1436345
Alias: None
Product: Fedora
Classification: Fedora
Component: rpmlint
Version: 26
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Tom "spot" Callaway
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-06-01 10:07 UTC by Miro Hrončok
Modified: 2017-06-01 15:50 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-06-01 15:14:26 UTC
Type: Bug


Attachments (Terms of Use)

Description Miro Hrončok 2017-06-01 10:07:41 UTC
Description of problem:
rpmlint shouts a lot with:

E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.

It's outputed several times and the detection does not work properly.

Version-Release number of selected component (if applicable):
rpmlint-1.9-9.fc26.noarch


How reproducible:
Easy


Steps to Reproduce:
1. fedpkg clone ipython && cd ipython && git checkout cc7cf5a
2. rpmlint ./ipython.spec

Actual results:
./ipython.spec: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
./ipython.spec: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
./ipython.spec: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
./ipython.spec: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
./ipython.spec: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
./ipython.spec: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
./ipython.spec: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
./ipython.spec: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
./ipython.spec: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
./ipython.spec: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
./ipython.spec: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
./ipython.spec: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
./ipython.spec: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
./ipython.spec: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
./ipython.spec: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
./ipython.spec: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
./ipython.spec: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
./ipython.spec: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
./ipython.spec: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
./ipython.spec: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
./ipython.spec: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
0 packages and 1 specfiles checked; 21 errors, 0 warnings.

Expected results:
./ipython.spec: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
0 packages and 1 specfiles checked; 1 error, 0 warnings.

Also, rpmlint should say on what line this happens to ease fixing.

Additional info:
When I add export LC_CTYPE=C.UTF-8 (or LANG) to start %build, %install and %check:

    diff --git a/ipython.spec b/ipython.spec
    index c354c01..5978228 100644
    --- a/ipython.spec
    +++ b/ipython.spec
    @@ -160,6 +160,7 @@ find . -name '*.py' -print0 | xargs -0 sed -i '1s|^#!python|#!%{__python3}|'
     
     
     %build
    +export LC_CTYPE=C.UTF-8
     %py3_build
     
     
    @@ -173,6 +174,7 @@ popd
     
     
     %install
    +export LC_CTYPE=C.UTF-8
     %py3_install
     
     # move the manpage to ipython3
    @@ -181,6 +183,7 @@ mv %{buildroot}%{_mandir}/man1/ipython{,3}.1
     
     %if %{with check}
     %check
    +export LC_CTYPE=C.UTF-8
     %global test_groups extensions lib testing terminal utils nbformat core autoreload
     # the following group seems to block on python3.4
     #kernel kernel.inprocess

I still get those 21 errors.

Something is wrong with this check.

Comment 1 Ville Skyttä 2017-06-01 14:27:52 UTC
What's your environment like? "env | grep -E 'LANG|LC_'" output?

The specfile modifications you made make no difference as the errors come from somewhere else entirely. Actually they come from "rpm --specfile" command invoked by rpmlint, which is why there are no line numbers available; it doesn't give any in this case.

Anyway, in a mock rawhide chroot I can reproduce but get only one such error, not 21, for the posted case. And I need to manually set LC_ALL=C, otherwise I get another related message (the one from bug 1436345).

I have ideas what to do about this upstream, but would like a complete reproducer using docker that could be used in CI to verify behavior and test for regressions, see bug 1436345 comment 3 and 4.

Comment 2 Miro Hrončok 2017-06-01 14:40:07 UTC
(In reply to Ville Skyttä from comment #1)
> What's your environment like? "env | grep -E 'LANG|LC_'" output?

LANG=cs_CZ.utf8

> The specfile modifications you made make no difference as the errors come
> from somewhere else entirely. Actually they come from "rpm --specfile"
> command invoked by rpmlint, which is why there are no line numbers
> available; it doesn't give any in this case.


> Anyway, in a mock rawhide chroot I can reproduce but get only one such
> error, not 21, for the posted case. And I need to manually set LC_ALL=C,
> otherwise I get another related message (the one from bug 1436345).
> 
> I have ideas what to do about this upstream, but would like a complete
> reproducer using docker that could be used in CI to verify behavior and test
> for regressions, see bug 1436345 comment 3 and 4.

Will look.

Comment 3 Miro Hrončok 2017-06-01 14:48:14 UTC
BTW

$ rpm --specfile ./ipython.spec 
ipython-6.1.0-1.fc26.noarch
python3-ipython-6.1.0-1.fc26.noarch
python3-ipython-sphinx-6.1.0-1.fc26.noarch
python3-ipython-tests-6.1.0-1.fc26.noarch
python3-ipython-doc-6.1.0-1.fc26.noarch

$ LANG=C rpm --specfile ./ipython.spec 
Python detected LC_CTYPE=C: LC_CTYPE & LANG coerced to C.UTF-8 (set another locale or PYTHONCOERCECLOCALE=0 to disable this locale coercion behavior).
Python detected LC_CTYPE=C: LC_CTYPE & LANG coerced to C.UTF-8 (set another locale or PYTHONCOERCECLOCALE=0 to disable this locale coercion behavior).
Python detected LC_CTYPE=C: LC_CTYPE & LANG coerced to C.UTF-8 (set another locale or PYTHONCOERCECLOCALE=0 to disable this locale coercion behavior).
Python detected LC_CTYPE=C: LC_CTYPE & LANG coerced to C.UTF-8 (set another locale or PYTHONCOERCECLOCALE=0 to disable this locale coercion behavior).
Python detected LC_CTYPE=C: LC_CTYPE & LANG coerced to C.UTF-8 (set another locale or PYTHONCOERCECLOCALE=0 to disable this locale coercion behavior).
Python detected LC_CTYPE=C: LC_CTYPE & LANG coerced to C.UTF-8 (set another locale or PYTHONCOERCECLOCALE=0 to disable this locale coercion behavior).
Python detected LC_CTYPE=C: LC_CTYPE & LANG coerced to C.UTF-8 (set another locale or PYTHONCOERCECLOCALE=0 to disable this locale coercion behavior).
Python detected LC_CTYPE=C: LC_CTYPE & LANG coerced to C.UTF-8 (set another locale or PYTHONCOERCECLOCALE=0 to disable this locale coercion behavior).
Python detected LC_CTYPE=C: LC_CTYPE & LANG coerced to C.UTF-8 (set another locale or PYTHONCOERCECLOCALE=0 to disable this locale coercion behavior).
Python detected LC_CTYPE=C: LC_CTYPE & LANG coerced to C.UTF-8 (set another locale or PYTHONCOERCECLOCALE=0 to disable this locale coercion behavior).
Python detected LC_CTYPE=C: LC_CTYPE & LANG coerced to C.UTF-8 (set another locale or PYTHONCOERCECLOCALE=0 to disable this locale coercion behavior).
Python detected LC_CTYPE=C: LC_CTYPE & LANG coerced to C.UTF-8 (set another locale or PYTHONCOERCECLOCALE=0 to disable this locale coercion behavior).
Python detected LC_CTYPE=C: LC_CTYPE & LANG coerced to C.UTF-8 (set another locale or PYTHONCOERCECLOCALE=0 to disable this locale coercion behavior).
Python detected LC_CTYPE=C: LC_CTYPE & LANG coerced to C.UTF-8 (set another locale or PYTHONCOERCECLOCALE=0 to disable this locale coercion behavior).
Python detected LC_CTYPE=C: LC_CTYPE & LANG coerced to C.UTF-8 (set another locale or PYTHONCOERCECLOCALE=0 to disable this locale coercion behavior).
Python detected LC_CTYPE=C: LC_CTYPE & LANG coerced to C.UTF-8 (set another locale or PYTHONCOERCECLOCALE=0 to disable this locale coercion behavior).
Python detected LC_CTYPE=C: LC_CTYPE & LANG coerced to C.UTF-8 (set another locale or PYTHONCOERCECLOCALE=0 to disable this locale coercion behavior).
Python detected LC_CTYPE=C: LC_CTYPE & LANG coerced to C.UTF-8 (set another locale or PYTHONCOERCECLOCALE=0 to disable this locale coercion behavior).
Python detected LC_CTYPE=C: LC_CTYPE & LANG coerced to C.UTF-8 (set another locale or PYTHONCOERCECLOCALE=0 to disable this locale coercion behavior).
Python detected LC_CTYPE=C: LC_CTYPE & LANG coerced to C.UTF-8 (set another locale or PYTHONCOERCECLOCALE=0 to disable this locale coercion behavior).
Python detected LC_CTYPE=C: LC_CTYPE & LANG coerced to C.UTF-8 (set another locale or PYTHONCOERCECLOCALE=0 to disable this locale coercion behavior).
ipython-6.1.0-1.fc26.noarch
python3-ipython-6.1.0-1.fc26.noarch
python3-ipython-sphinx-6.1.0-1.fc26.noarch
python3-ipython-tests-6.1.0-1.fc26.noarch
python3-ipython-doc-6.1.0-1.fc26.noarch

(I cannot reproduce this in mock without explicitly setting LANG=C and that also just prints this once.)

Comment 4 Miro Hrončok 2017-06-01 14:52:41 UTC
/usr/share/rpmlint/SpecCheck.py:545

        out = Pkg.getstatusoutput(('env', 'LC_ALL=C', 'rpm', '-q',
                                   '--qf=', '--specfile', self._spec_file))

Could this be changed to 'LC_ALL=C.UTF-8'? It fixes the problem for me.

Comment 5 Miro Hrončok 2017-06-01 15:12:05 UTC
The difference between the messages:

$ LC_ALL=C rpm -q --qf= --specfile /home/churchyard/rpmbuild/fedora-scm/ipython/ipython.spec
Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
...

[~]$ LANG=C rpm -q --qf= --specfile /home/churchyard/rpmbuild/fedora-scm/ipython/ipython.spec
Python detected LC_CTYPE=C: LC_CTYPE & LANG coerced to C.UTF-8 (set another locale or PYTHONCOERCECLOCALE=0 to disable this locale coercion behavior).
...

I believe bug 1436345 and this one are duplicates.

Comment 6 Miro Hrončok 2017-06-01 15:14:26 UTC

*** This bug has been marked as a duplicate of bug 1436345 ***

Comment 7 Ville Skyttä 2017-06-01 15:32:32 UTC
(In reply to Miro Hrončok from comment #3)
> BTW
> 
> $ rpm --specfile ./ipython.spec 
> ipython-6.1.0-1.fc26.noarch
> python3-ipython-6.1.0-1.fc26.noarch
> python3-ipython-sphinx-6.1.0-1.fc26.noarch
> python3-ipython-tests-6.1.0-1.fc26.noarch
> python3-ipython-doc-6.1.0-1.fc26.noarch
> 
> $ LANG=C rpm --specfile ./ipython.spec 
> Python detected LC_CTYPE=C: LC_CTYPE & LANG coerced to C.UTF-8 (set another
> locale or PYTHONCOERCECLOCALE=0 to disable this locale coercion behavior).

I suppose this means that rpm itself needs a "fix" as well.

Comment 8 Miro Hrončok 2017-06-01 15:41:21 UTC
LANG=C is bad. I'd very much like to see a day when we build packages with LANG=C.UTF-8. In the meantime, we might want to "fix" all python3 macros to set LANG. Will discuss that on Python Devel ML.

Comment 9 Ville Skyttä 2017-06-01 15:44:30 UTC
(In reply to Miro Hrončok from comment #4)
> /usr/share/rpmlint/SpecCheck.py:545
> 
>         out = Pkg.getstatusoutput(('env', 'LC_ALL=C', 'rpm', '-q',
>                                    '--qf=', '--specfile', self._spec_file))
> 
> Could this be changed to 'LC_ALL=C.UTF-8'? It fixes the problem for me.

I think that could cause breakage in non-UTF-8 setups (e.g. if the command outputs non-ASCII UTF-8, and the read routines in Pkg.getstatusoutput expect e.g. ISO-8859-1). And besides, there are a lot more potential places for this breakage in rpmlint than just this one, basically everywhere where an executable is invoked. So it needs a more centralized fix (I'm on one).

Comment 10 Ville Skyttä 2017-06-01 15:50:53 UTC
(In reply to Miro Hrončok from comment #8)
> LANG=C is bad.

I'd rather say that Python's default second guessing and deliberate incorrect^Htradition deviant handling of LANG=C, essentially treating it as an user error, is bad. But of course, if ones doesn't actually intend LANG=C, then one shouldn't use it but something else.

> In the meantime, we might want to "fix" all python3 macros to
> set LANG. Will discuss that on Python Devel ML.

Setting PYTHONCOERCECLOCALE=0 could be a better choice, keeping e.g. ISO-8859-1 environments in mind. That's what I'm looking into doing first in rpmlint.


Note You need to log in before you can comment on or make changes to this bug.