Bug 574437 - Python3 build macros are broken
Summary: Python3 build macros are broken
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: python3
Version: rawhide
Hardware: All
OS: Linux
low
medium
Target Milestone: ---
Assignee: Charalampos Stratakis
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On: 976651 1222941 1222944
Blocks:
TreeView+ depends on / blocked
 
Reported: 2010-03-17 14:28 UTC by Jonathan Underwood
Modified: 2016-02-01 11:05 UTC (History)
5 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2016-02-01 11:05:55 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)

Description Jonathan Underwood 2010-03-17 14:28:56 UTC
Description of problem:
I am trying to fix a problem with building the Emacs package in F-13 and devel. 

Package building is broken presently because there are two scripts /usr/share/emacs/23.1/etc/emacs{2,3}.py which are for python 2.6 and 3 and the brp-python-bytecompile script tries to compile both with python 2.6 and fails with emacs3.py.

So, I followed the instructions here: 

http://fedoraproject.org/wiki/Packaging/Python#Bytecompiling_with_the_correct_python_version

to disable the brp-python-bytecompile script and byte compile by hand. In other words, I added the following to the spec file:

BuildRequires: python2-devel python3-devel
%global __os_install_post %(echo '%{__os_install_post}' | sed -e 's!/usr/lib[^[:space:]]*/brp-python-bytecompile!!g')

...

%{py_byte_compile} %{__python} %{buildroot}%{_datadir}/%{name}/%{version}/etc/emacs.py
%{py_byte_compile} %{__python} %{buildroot}%{_datadir}/%{name}/%{version}/etc/emacs2.py
%{py_byte_compile} %{__python3} %{buildroot}%{_datadir}/%{name}/%{version}/etc/emacs3.py


However, package building bails with these errors:

+ python_binary=%1
+ bytecode_compilation_path=%2
+ xargs -0 %1 -c 'import py_compile, sys; [py_compile.compile(f, dfile=f.partition("$RPM_BUILD_ROOT")[2]) for f in sys.argv[1:]]'
+ find %2 -type f -a -name '*.py' -print0
find: `%2': No such file or directory
xargs: %1: Permission denied
+ :
+ find %2 -type f -a -name '*.py' -print0
+ xargs -0 %1 -O -c 'import py_compile, sys; [py_compile.compile(f, dfile=f.partition("$RPM_BUILD_ROOT")[2]) for f in sys.argv[1:]]'
find: `%2': No such file or directory
xargs: %1: Permission denied
+ :
+ /usr/bin/python /builddir/build/BUILDROOT/emacs-23.1-25.fc14.x86_64/usr/share/emacs/23.1/etc/emacs.py
+ python_binary=%1
+ bytecode_compilation_path=%2
+ find %2 -type f -a -name '*.py' -print0
+ xargs -0 %1 -c 'import py_compile, sys; [py_compile.compile(f, dfile=f.partition("$RPM_BUILD_ROOT")[2]) for f in sys.argv[1:]]'
find: `%2': No such file or directory
xargs: %1: Permission denied
+ 
+ find %2 -type f -a -name '*.py' -print0
+ xargs -0 %1 -O -c 'import py_compile, sys; [py_compile.compile(f, dfile=f.partition("$RPM_BUILD_ROOT")[2]) for f in sys.argv[1:]]'
find: `%2': No such file or directory
xargs: %1: Permission denied
+ 
+ /usr/bin/python /builddir/build/BUILDROOT/emacs-23.1-25.fc14.x86_64/usr/share/emacs/23.1/etc/emacs2.py
+ python_binary=%1
+ bytecode_compilation_path=%2
+ find %2 -type f -a -name '*.py' -print0
+ xargs -0 %1 -c 'import py_compile, sys; [py_compile.compile(f, dfile=f.partition("$RPM_BUILD_ROOT")[2]) for f in sys.argv[1:]]'
find: `%2': No such file or directory
xargs: %1: Permission denied
+ :
+ find %2 -type f -a -name '*.py' -print0
+ xargs -0 %1 -O -c 'import py_compile, sys; [py_compile.compile(f, dfile=f.partition("$RPM_BUILD_ROOT")[2]) for f in sys.argv[1:]]'
find: `%2': No such file or directory
xargs: %1: Permission denied
+ 
+ /usr/bin/python3 /builddir/build/BUILDROOT/emacs-23.1-25.fc14.x86_64/usr/share/emacs/23.1/etc/emacs3.py

....

+ /usr/lib/rpm/redhat/brp-strip-comment-note /usr/bin/strip /usr/bin/objdump
+ /usr/lib/rpm/redhat/brp-strip-static-archive /usr/bin/strip
+ /usr/bin/python 1
/usr/bin/python: can't open file '1': [Errno 2] No such file or directory
error: Bad exit status from /var/tmp/rpm-tmp.75sByS (%install)
RPM build errors:
    Bad exit status from /var/tmp/rpm-tmp.75sByS (%install)
Child returncode was: 1
EXCEPTION: Command failed. See logs for output.
 # ['bash', '--login', '-c', 'rpmbuild -bb --target x86_64 --nodeps builddir/build/SPECS/emacs.spec']
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/mock/trace_decorator.py", line 70, in trace
    result = func(*args, **kw)
  File "/usr/lib/python2.6/site-packages/mock/util.py", line 324, in do
    raise mock.exception.Error, ("Command failed. See logs for output.\n # %s" % (command,), child.returncode)
Error: Command failed. See logs for output.


So it looks to me like these macros are broken.

Comment 1 Jonathan Underwood 2010-03-17 14:35:36 UTC
The spec file with the python compilation bits can be found here:

Comment 2 Jonathan Underwood 2010-03-17 14:36:35 UTC
Gah, URL failed.

http://jgu.fedorapeople.org/emacs.spec

Comment 3 Dave Malcolm 2010-03-19 00:18:49 UTC
Thanks for filing this bug report, sorry I didn't check back sooner.

Notes to self:

I see build failures here:
http://koji.fedoraproject.org/koji/taskinfo?taskID=2053382
and:
http://koji.fedoraproject.org/koji/taskinfo?taskID=2053363
but these look like an earlier iteration where:
  + /usr/lib/rpm/brp-python-bytecompile /usr/bin/python 1
is called, and it fails on the python 3 due to language syntax changes.

Looks like the quoting may be wrong in /etc/rpm/macros.pybytecompile

See http://rpm.org/wiki/PackagerDocs/Macros ; do configuration macros actually support parameters?

(CCing Toshio: did this code used to work in the past, or did we mess up when we added this back in:
   * Mon Jan 25 2010 David Malcolm <dmalcolm> - 3.1.1-21
)

Comment 4 Toshio Ernie Kuratomi 2010-03-19 02:00:42 UTC
can you try using this instead::

  %py_byte_compile %{__python} %{buildroot}%{_datadir}/mypackage/foo
  %py_byte_compile %{__python3} %{buildroot}%{_datadir}/mypackage/bar

ie: leave off the {} in the %py_byte_compile macro.

If that works, I need to change the Guidelines to not have {} and write an admonition to leave them off.  The reason I think that form works is here::

  http://rpm.org/wiki/PackagerDocs/Macros#UsingaMacro

Comment 5 Jonathan Underwood 2010-03-19 11:54:23 UTC
(In reply to comment #4)
> can you try using this instead::
> 
>   %py_byte_compile %{__python} %{buildroot}%{_datadir}/mypackage/foo
>   %py_byte_compile %{__python3} %{buildroot}%{_datadir}/mypackage/bar
> 
> ie: leave off the {} in the %py_byte_compile macro.

OK, that seems to work fine and the python files are byte compiled. However, the build still bails with 

+ /usr/bin/python 1
/usr/bin/python: can't open file '1': [Errno 2] No such file or directory
error: Bad exit status from /var/tmp/rpm-tmp.QAKZGo (%install)

Comment 6 Toshio Ernie Kuratomi 2010-03-19 15:07:50 UTC
Ugh.  We really messed up the recipes we included in the Guidelines.  Here's what's happening there::

  %global __os_install_post %(echo '%{__os_install_post}' | sed -e
's!/usr/lib[^[:space:]]*/brp-python-bytecompile!!g')

This is stripping out brp-python-bytecompile from __os_install_post.  However, it is leaving the arguments to brp-python-bytecompile in.  In F-13 and F-14 the arguments are "/usr/bin/python 1" which is leading to that error.

I think this will work::

  %global __os_install_post %(echo '%{__os_install_post}' | sed -e
's!/usr/lib[^[:space:]]*/brp-python-bytecompile[[:space:]].*$!!g')

but I want to see the build.log to confirm that we don't cut out commands after the bytecompile with that syntax.  If that fails we can do this instead:

  %global __os_install_post %(echo '%{__os_install_post}' | sed -e
's!/usr/lib[^[:space:]]*/brp-python-bytecompile!/bin/true!g')

which I know will work but is a bit hackish.

Comment 7 Toshio Ernie Kuratomi 2010-03-19 15:16:01 UTC
Just checked::

  %global __os_install_post %(echo '%{__os_install_post}' | sed -e
's!/usr/lib[^[:space:]]*/brp-python-bytecompile[[:space:]].*$!!g')

Looks good.  I'm updating the Guidelines; give it a try.

Comment 8 Jonathan Underwood 2010-03-19 15:28:06 UTC
Yup, that works for me too - I was just waiting for a mock build to finish to confirm :) Thanks!

Comment 9 Jonathan Underwood 2010-03-19 15:34:39 UTC
As an aside, would it not be nicer to hide this:

%global __os_install_post %(echo '%{__os_install_post}' | sed -e
's!/usr/lib[^[:space:]]*/brp-python-bytecompile[[:space:]].*$!!g')

behind a macro eg %py_auto_byte_compile_off or some such?

Comment 10 Toshio Ernie Kuratomi 2010-03-29 23:48:13 UTC
dmalcolm is looking into putting this into a macro.

Comment 11 Bug Zapper 2010-11-03 19:24:08 UTC
This message is a reminder that Fedora 12 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 12.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '12'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 12's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 12 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 12 Toshio Ernie Kuratomi 2010-11-03 20:43:41 UTC
dmalcolm ping?  No preference from me as to whether a macro is created; just want to see the bug closed eventually :-)

Comment 13 Fedora Admin XMLRPC Client 2013-05-10 04:58:06 UTC
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Comment 14 Toshio Ernie Kuratomi 2013-05-24 17:20:05 UTC
bkbrda ping?

Closing this is an easyfix.  We just need to make a decision on the request here:

(In reply to Jonathan Underwood from comment #9)
> As an aside, would it not be nicer to hide this:
> 
> %global __os_install_post %(echo '%{__os_install_post}' | sed -e
> 's!/usr/lib[^[:space:]]*/brp-python-bytecompile[[:space:]].*$!!g')
> 
> behind a macro eg %py_auto_byte_compile_off or some such?

Comment 15 Bohuslav "Slavek" Kabrda 2013-05-27 06:27:48 UTC
Sorry, I must have missed this. Macroing this seems fine to me, although I would probably prefer having something like

%{!?py_auto_byte_compile_off:<current bytecompilation stuff>}

in %__os_install_post, so that we could override default bytecompilation without sedding a specific string in python macros (if that would change in RPM, we would have to change ours, too).

Thoughts?

Comment 16 Toshio Ernie Kuratomi 2013-05-28 17:30:06 UTC
If it works, that sounds good to me.  I'd reverse the meaning of the switch though -- py_auto_byte_compile => when on (default), it enables byte compilation.  When set to %{nil} it disables auto byte compilation.  This would mirror what is done for debug packages:

"If you wish to disable generation of the useless debuginfo package [...] use %global debug_package %{nil} in the specfile, and be sure to add a comment next to it explaining why it was done." -- http://fedoraproject.org/wiki/Packaging:Debuginfo#Useless_or_incomplete_debuginfo_packages_due_to_other_reasons

Comment 17 Bohuslav "Slavek" Kabrda 2013-06-06 20:05:45 UTC
(In reply to Toshio Ernie Kuratomi from comment #16)
> If it works, that sounds good to me.  I'd reverse the meaning of the switch
> though -- py_auto_byte_compile => when on (default), it enables byte
> compilation.  When set to %{nil} it disables auto byte compilation.  This
> would mirror what is done for debug packages:
> 
> "If you wish to disable generation of the useless debuginfo package [...]
> use %global debug_package %{nil} in the specfile, and be sure to add a
> comment next to it explaining why it was done." --
> http://fedoraproject.org/wiki/Packaging:
> Debuginfo#Useless_or_incomplete_debuginfo_packages_due_to_other_reasons

Good point, so this would mean adding


%py_auto_byte_compile 1

%__os_install_post \
<snip>
%{?py_auto_byte_compile:<current bytecompilation stuff>} \
<snip>


If this is what you mean then I think we're in agreement and I can propose the patch for macros file :)

Comment 18 Toshio Kuratomi 2013-06-07 15:56:33 UTC
Yep.  +1

Comment 19 Bohuslav "Slavek" Kabrda 2013-06-21 06:10:19 UTC
See bug 976651 for the proposal to add this to redhat-rpm-config.

Comment 20 Jonathan Underwood 2014-07-25 09:22:53 UTC
Not sure why this is showing as NEEDINFO from me. Clearing.

Comment 21 Fedora Admin XMLRPC Client 2015-05-12 12:01:30 UTC
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Comment 22 Jonathan Underwood 2015-05-12 22:10:59 UTC
Wow, is this really not fixed yet? :)

Comment 23 Matej Stuchlik 2015-05-13 16:38:41 UTC
(In reply to Jonathan Underwood from comment #22)
> Wow, is this really not fixed yet? :)

Pinged bug 976651 :)

Comment 24 Fedora Admin XMLRPC Client 2016-01-29 13:06:58 UTC
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Comment 25 Jonathan Underwood 2016-02-01 11:05:55 UTC
OK, I am going to close this (nearly 6 years old) bug, as it looks like the work has been done in 976651.


Note You need to log in before you can comment on or make changes to this bug.