Description of problem: The same code when redirected from STDOUT to file can result in traceback, which is strange. Version-Release number of selected component (if applicable): python-2.7.12-6.fc24.x86_64 How reproducible: Always Steps to Reproduce: 1.cat > test.py <<:EOF #!/bin/python url = 'https://github.com/EttusResearch/uhd/releases' import requests text = requests.get(url).text print(text) :EOF 2. python test.py 3. python test.py > log.txt Actual results: 2. content 3. Traceback (most recent call last): File "./test0.py", line 10, in <module> print text UnicodeEncodeError: 'ascii' codec can't encode character u'\xb7' in position 1016: ordinal not in range(128) Expected results: 2. content 3. content Additional info: Redirection to file shouldn't result in different behavior, nor in traceback.
Smaller reproducer: 1.cat > test.py <<:EOF print(u'\xb7') :EOF 2. python test.py 3. python test.py | head Actual results: 2. · 3. Traceback (most recent call last): File "test.py", line 1, in <module> print(u'\xb7') UnicodeEncodeError: 'ascii' codec can't encode character u'\xb7' in position 0: ordinal not in range(128) Expected results: 2. · 3. · But I'm afraid this is a "feature" of Python 2. See http://stackoverflow.com/questions/19696652/piping-output-causes-python-program-to-fail
The issue is that in CPython 2 the encoding of stdout changes from utf-8 to ascii if stdout is not a tty. I agree that it's counter-intuitive. FWIW, I proposed changing this in Fedora 13: https://fedoraproject.org/wiki/Features/PythonEncodingUsesSystemLocale but I was asked to not change this by the upstream Python development community, to ensure consistent behavior with every other CPython 2 implementation. See that link for more information, and some workarounds. Closing this out as WONTFIX.