Bug 1436345 - false positives if LC_CTYPE=C
Summary: false positives if LC_CTYPE=C
Keywords:
Status: CLOSED EOL
Alias: None
Product: Fedora
Classification: Fedora
Component: rpmlint
Version: 26
Hardware: All
OS: Linux
unspecified
low
Target Milestone: ---
Assignee: Tom "spot" Callaway
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 1457786 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-03-27 17:08 UTC by Björn 'besser82' Esser
Modified: 2018-05-29 11:33 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-05-29 11:33:15 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
flat file containing results of running the test in this bug report (4.01 KB, text/plain)
2017-08-14 17:26 UTC, George R. Goffe
no flags Details

Description Björn 'besser82' Esser 2017-03-27 17:08:37 UTC
Description of problem:

  see `Actual results`.


Version-Release number of selected component (if applicable):

  rpmlint-1.9-8.fc26


How reproducible:

  100%


Steps to Reproduce:

  1. export LC_CTYPE=C
  2. Run rpmlint to check a src.rpm


Actual results:

  Several appearances of:
    `E: specfile-error Python detected LC_CTYPE=C: LC_ALL & LANG coerced
     to C.UTF-8 (set another locale or PYTHONCOERCECLOCALE=0 to disable
     this locale coercion behaviour).`

  Those are false positives.


Expected results:

  No report of mentioned false positive.

Comment 1 Ville Skyttä 2017-04-07 06:49:53 UTC
I cannot reproduce as described, all I see is one "Python detected LC_CTYPE=C ..." message at the start, not several instances and the one in specfile-error. For example, in a current mock chroot:

<mock-chroot> sh-4.4# curl -sO http://dl.fedoraproject.org/pub/fedora/linux/development/rawhide/Everything/source/tree/Packages/r/rsvndump-0.6-9.fc26.src.rpm
<mock-chroot> sh-4.4# LC_CTYPE=C rpmlint ./rsvndump-0.6-9.fc26.src.rpm 
Python detected LC_CTYPE=C: LC_ALL & LANG coerced to C.UTF-8 (set another locale or PYTHONCOERCECLOCALE=0 to disable this locale coercion behaviour).
rsvndump.src: W: spelling-error %description -l en_US svnadmin -> sysadmin
rsvndump.src: E: unknown-key RSA#f5282ee4 (MD5
1 packages and 0 specfiles checked; 1 errors, 1 warnings.
<mock-chroot> sh-4.4# 

Reading https://www.python.org/dev/peps/pep-0538/ tells me that a better thing than setting LC_CTYPE=C (or LC_ALL=C) for Python programs would be to set them to C.UTF-8 instead, so I suggest you do that.

rpmlint would probably be affected as well and output the messages you describe if it invoked any python3 programs from within it, as it does typically pass LC_ALL=C to things it invokes. For some reason I'm not seeing any of the messages myself though, and would not like to go change things without being able to see the effects. So I guess we'll need a more exact, specific reproducer than just "Run rpmlint to check a src.rpm" -- full details which exact src.rpm was tested, all locale settings in effect.

(Note that to test with current rawhide, you'll probably need the patch for the upstream change mentioned in bug 1439941.)

Comment 2 Björn 'besser82' Esser 2017-04-07 09:33:34 UTC
It reproduces 100% with the srpm from this build [1].  The error shows up, even when LANG / LC_ALL is set to any UTF-8 locale.


```
$ rpmlint ./python-pystray-0.14.3-1.fc27.src.rpm 
python-pystray.src: E: specfile-error Python detected LC_CTYPE=C: LC_ALL & LANG coerced to C.UTF-8 (set another locale or PYTHONCOERCECLOCALE=0 to disable this locale coercion behaviour).
python-pystray.src: E: specfile-error Python detected LC_CTYPE=C: LC_ALL & LANG coerced to C.UTF-8 (set another locale or PYTHONCOERCECLOCALE=0 to disable this locale coercion behaviour).
python-pystray.src: E: specfile-error Python detected LC_CTYPE=C: LC_ALL & LANG coerced to C.UTF-8 (set another locale or PYTHONCOERCECLOCALE=0 to disable this locale coercion behaviour).
python-pystray.src: E: specfile-error Python detected LC_CTYPE=C: LC_ALL & LANG coerced to C.UTF-8 (set another locale or PYTHONCOERCECLOCALE=0 to disable this locale coercion behaviour).
python-pystray.src: E: specfile-error Python detected LC_CTYPE=C: LC_ALL & LANG coerced to C.UTF-8 (set another locale or PYTHONCOERCECLOCALE=0 to disable this locale coercion behaviour).
python-pystray.src: E: specfile-error Python detected LC_CTYPE=C: LC_ALL & LANG coerced to C.UTF-8 (set another locale or PYTHONCOERCECLOCALE=0 to disable this locale coercion behaviour).
python-pystray.src: E: specfile-error Python detected LC_CTYPE=C: LC_ALL & LANG coerced to C.UTF-8 (set another locale or PYTHONCOERCECLOCALE=0 to disable this locale coercion behaviour).
1 packages and 0 specfiles checked; 7 errors, 0 warnings.
```


[1]  https://koji.fedoraproject.org/koji/buildinfo?buildID=874164

Comment 3 Ville Skyttä 2017-04-07 11:10:55 UTC
The mentioned srpm still doesn't reproduce for me in a mock rawhide chroot, behaves the same way as my sample in comment 1. The travis CI test cases ran on rawhide container[1] don't reproduce them either with the data included with rpmlint. Please try to extract a complete, specific reproducer using mock rawhide chroot or docker.

Here's my example with docker that doesn't reproduce it here:

# docker pull fedora:rawhide
# docker run -it --rm fedora:rawhide /bin/bash
# dnf install https://kojipkgs.fedoraproject.org//packages/rpmlint/1.9/9.fc27/noarch/rpmlint-1.9-9.fc27.noarch.rpm
# curl -sO https://kojipkgs.fedoraproject.org//packages/python-pystray/0.14.3/1.fc27/src/python-pystray-0.14.3-1.fc27.src.rpm
# rpmlint ./python-pystray-0.14.3-1.fc27.src.rpm
# LC_CTYPE=C ./python-pystray-0.14.3-1.fc27.src.rpm


[1] https://travis-ci.org/rpm-software-management/rpmlint/jobs/219571945

Comment 4 Ville Skyttä 2017-04-07 11:11:47 UTC
(In reply to Ville Skyttä from comment #3)
> # LC_CTYPE=C ./python-pystray-0.14.3-1.fc27.src.rpm

(This should have been "LC_CTYPE=C rpmlint ...")

Comment 5 Miro Hrončok 2017-06-01 15:14:26 UTC
*** Bug 1457786 has been marked as a duplicate of this bug. ***

Comment 6 Miro Hrončok 2017-06-01 15:18:37 UTC
BTW with the mentioned file i get 7 errors as well:

$ rpmlint ./python-pystray-0.14.3-1.fc27.src.rpm 
python-pystray.src: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
python-pystray.src: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
python-pystray.src: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
python-pystray.src: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
python-pystray.src: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
python-pystray.src: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
python-pystray.src: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
1 packages and 0 specfiles checked; 7 errors, 0 warnings.

Comment 7 Miro Hrončok 2017-06-01 15:24:57 UTC
Why I cannot reproduce this in mock:

In mock, it does not invoke Python.

<mock-chroot> sh-4.4# chmod -x /usr/bin/python3
<mock-chroot> sh-4.4# LC_ALL=C rpm -q --qf= --specfile /ipython.spec 
<mock-chroot> sh-4.4#

Locally, it does:

$ sudo chmod -x /usr/bin/python3
[sudo] heslo pro churchyard: 
[~]$ LC_ALL=C rpm -q --qf= --specfile /home/churchyard/rpmbuild/fedora-scm/ipython/ipython.spec
sh: /usr/bin/python3: Permission denied
sh: /usr/bin/python3: Permission denied
sh: /usr/bin/python3: Permission denied
sh: /usr/bin/python3: Permission denied
sh: /usr/bin/python3: Permission denied
sh: /usr/bin/python3: Permission denied
sh: /usr/bin/python3: Permission denied
sh: /usr/bin/python3: Permission denied
sh: /usr/bin/python3: Permission denied
sh: /usr/bin/python3: Permission denied
sh: /usr/bin/python3: Permission denied
sh: /usr/bin/python3: Permission denied
sh: /usr/bin/python3: Permission denied
sh: /usr/bin/python3: Permission denied
sh: /usr/bin/python3: Permission denied
sh: /usr/bin/python3: Permission denied
sh: /usr/bin/python3: Permission denied
sh: /usr/bin/python3: Permission denied
sh: /usr/bin/python3: Permission denied
sh: /usr/bin/python3: Permission denied
sh: /usr/bin/python3: Permission denied

Possibly evaluating macros?

Comment 8 Miro Hrončok 2017-06-01 15:26:38 UTC
I have a mock reproducer, just install python3-devel.

Comment 9 Miro Hrončok 2017-06-01 15:37:05 UTC
Docker reproducer:

# docker run -it --rm fedora:rawhide /bin/bash
# dnf install -y https://kojipkgs.fedoraproject.org//packages/rpmlint/1.9/9.fc27/noarch/rpmlint-1.9-9.fc27.noarch.rpm python2 python3-devel fedora-packager
# curl -sO https://kojipkgs.fedoraproject.org//packages/python-pystray/0.14.3/1.fc27/src/python-pystray-0.14.3-1.fc27.src.rpm
# LANG=C.UTF-8 rpmlint ./python-pystray-0.14.3-1.fc27.src.rpm
python-pystray.src: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
python-pystray.src: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
python-pystray.src: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
python-pystray.src: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
python-pystray.src: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
python-pystray.src: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
python-pystray.src: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
1 packages and 0 specfiles checked; 7 errors, 0 warnings.


Installing fedora-packager might be overkill but i don't feel like bisecting it's requires to see what brings it.

The message comes form evaluating the specfile's macros that calls python3. I believe the fix is to call `rpm --specfile` with LC_ALL=C.UTF-8, not C.

Comment 10 Ville Skyttä 2017-06-01 15:59:21 UTC
(In reply to Miro Hrončok from comment #8)
> I have a mock reproducer, just install python3-devel.

Nice catch, thanks!

(In reply to Miro Hrončok from comment #9)
> The message comes form evaluating the specfile's macros that calls python3.
> I believe the fix is to call `rpm --specfile` with LC_ALL=C.UTF-8, not C.

But it doesn't necessarily end at all to that particular call, nor is it clear that forcing UTF-8 like this would be a desirable fix as discussed in bug 1457786 comment 9.

Comment 11 Todd Zullinger 2017-06-01 16:03:25 UTC
(In reply to Miro Hrončok from comment #9)
> Installing fedora-packager might be overkill but i don't feel like bisecting
> it's requires to see what brings it.

FWIW, rpm-build looks like all you need from the fedora-packager deps to reproduce.

# docker run -it --rm fedora:rawhide /bin/bash
# dnf install -y https://kojipkgs.fedoraproject.org//packages/rpmlint/1.9/9.fc27/noarch/rpmlint-1.9-9.fc27.noarch.rpm python2 python3-devel rpm-build
# # curl -sO https://kojipkgs.fedoraproject.org//packages/python-pystray/0.14.3/1.fc27/src/python-pystray-0.14.3-1.fc27.src.rpm
# LANG=C.UTF-8 rpmlint ./python-pystray-0.14.3-1.fc27.src.rpm
python-pystray.src: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
python-pystray.src: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
python-pystray.src: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
python-pystray.src: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
python-pystray.src: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
python-pystray.src: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
python-pystray.src: E: specfile-error Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8, C.utf8, or UTF-8 (if available) as alternative Unicode-compatible locales is recommended.
1 packages and 0 specfiles checked; 7 errors, 0 warnings.

Comment 12 George R. Goffe 2017-06-16 08:24:54 UTC
Hi,

I'm getting this message just one time... right after login as root. I put the setting with export in the root .bash_profile but am still seeing this message. The message comes out reliably so if you want info, just ask.

System is freshly up to date from the Fedora 27 (rawhide) x86_64 repository.

George...

Comment 13 George R. Goffe 2017-08-14 17:26:20 UTC
Created attachment 1313291 [details]
flat file containing results of running the test in this bug report

I took the "script" from this bug report and pasted it into a file and executed it. Here's what I got AND the test script that I had copied.

Need more info? Just ask.

George..

Comment 14 Fedora End Of Life 2018-05-03 09:04:12 UTC
This message is a reminder that Fedora 26 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 26. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '26'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 26 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged  change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

Comment 15 Fedora End Of Life 2018-05-29 11:33:15 UTC
Fedora 26 changed to end-of-life (EOL) status on 2018-05-29. Fedora 26
is no longer maintained, which means that it will not receive any
further security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.


Note You need to log in before you can comment on or make changes to this bug.