Bug 1484993

Summary: %py_byte_compile broken with Python 2: compile() got an unexpected keyword argument 'optimize'
Product: [Fedora] Fedora Reporter: Rolf Fokkens <rolf>
Component: python3Assignee: Charalampos Stratakis <cstratak>
Status: CLOSED DUPLICATE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 27CC: bkabrda, cstratak, dmalcolm, ishcherb, jdisnard, lzachar, mcyprian, mebrown, mhroncok, msimacek, msuchy, praiskup, pviktori, rkuska, terje.rosten, tomspur, torsava
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-01-05 11:11:05 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1523906    

Description Rolf Fokkens 2017-08-24 19:16:23 UTC
Description of problem:
Since a while rewhide and F27 exhibit these error messages during mock build for a certain package:
+ bytecode_compilation_path=/builddir/build/BUILDROOT/th-tools-0.2-50.8.9c61ed1_git.fc27.x86_64/usr/lib/th-tools
+ find /builddir/build/BUILDROOT/th-tools-0.2-50.8.9c61ed1_git.fc27.x86_64/usr/lib/th-tools -type f -a -name '*.py' -print0
+ xargs -0 /usr/bin/python2 -O -c 'import py_compile, sys; [py_compile.compile(f, dfile=f.partition("$RPM_BUILD_ROOT")[2], optimize=opt) for opt in range(2) for f in sys.argv[1:]]'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
TypeError: compile() got an unexpected keyword argument 'optimize'
+ :

Version-Release number of selected component (if applicable):
mock-1.4.4-1.el7

How reproducible:
100%

Steps to Reproduce:
1. create a src.rpm with python2 parts
2. run mock

Actual results:
TypeError

Expected results:
Successful build

Additional info:
This may be cause by issues in mock, fedora 27, specific details of the package to build er .. The issue seems to be related to a specific RPM macro though.

Comment 1 Miroslav Suchý 2017-12-22 08:52:13 UTC
This macro comes from:
  /usr/lib/rpm/macros.d/macros.pybytecompile3.6
which is owned by
  python3-devel-3.6.3-2.fc27.x86_64

It calls py_compile.compile() with argument 'optimize' which seems does not exists on python2
  https://docs.python.org/2/library/py_compile.html#py_compile.compile
It exist only in python3.

This is clearly python issue.

Comment 2 Miro Hrončok 2017-12-22 12:04:49 UTC
Note that according to documentation [1], the %py_byte_compile macro indeed should work with both Pythons.


[1] https://fedoraproject.org/wiki/Packaging:Python_Appendix#Manual_byte_compilation

Comment 3 Miro Hrončok 2017-12-22 12:10:37 UTC
The macro was updated when going from Python 3.4 to 3.5 (i.e. Fedora 23 to Fedora 24).

https://src.fedoraproject.org/rpms/python3/blob/f23/f/macros.pybytecompile3.4
https://src.fedoraproject.org/rpms/python3/blob/f24/f/macros.pybytecompile3.5

If I remember correctly, the old version didn't work with Python 3.5, so an update was done, while not being Python 2 compatible, this was probably unnoticed ever since, because the situation described in the guidelines (linked as [1] in my previous) is not very common.

We'll definitively need to fix this somehow. In the meantime, if you share the entire spec, I might be able to provide a workaround for you.

Comment 4 Terje Røsten 2018-01-04 20:01:07 UTC
I need this since recoll started to ship filters written in both Python 2 and 3.

Comment 5 Terje Røsten 2018-01-04 20:05:24 UTC
Spec file snippet:

# Turn off the brp-python-bytecompile script
%global __os_install_post %(echo '%{__os_install_post}' | sed -e 's!/usr/lib[^[:space:]]*/brp-python-bytecompile[[:space:]].*$!!g')


[snip]

# Mix of Python 2 and 3. Needs special care
%py_byte_compile %{__python2} %{buildroot}%{python2_sitearch}/recoll

for py in %{buildroot}%{_datadir}/%{name}/filters/*; do
    if [ "$(basename $py)" = "recoll-we-move-files.py" ]; then
	%py_byte_compile %{__python3} $py
    else
	%py_byte_compile %{__python2} $py
    fi
done

BTW: extending logic in brp-python-bytecompile to be a bit smarter should not be
that hard? At least looking at shebang "#! /usr/bin/pythonX" should be possible?

Comment 6 Miro Hrončok 2018-01-05 11:11:05 UTC
Marking this one as a duplicate, because the second one better summarizes what's wrong.

*** This bug has been marked as a duplicate of bug 1531253 ***