Bug 1721147 - %__brp_python_bytecompile uses %__python macro
Summary: %__brp_python_bytecompile uses %__python macro
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: redhat-rpm-config
Version: 32
Hardware: All
OS: Linux
unspecified
unspecified
Target Milestone: ---
Assignee: Lumír Balhar
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-06-17 13:12 UTC by Orion Poplawski
Modified: 2020-06-16 05:30 UTC (History)
14 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-06-16 05:30:33 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Orion Poplawski 2019-06-17 13:12:28 UTC
Description of problem:

+ /usr/lib/rpm/brp-strip-static-archive /usr/bin/strip
BUILDSTDERR: /var/tmp/rpm-tmp.GIHOBr: line 86: unexpected EOF while looking for matching `"'
BUILDSTDERR: error: Bad exit status from /var/tmp/rpm-tmp.GIHOBr (%install)
mkdir (/tmp/oct-kpkHBG)
untar (/builddir/build/BUILD/octave-general-2.1.0/build/general-2.1.0-x86_64-redhat-linux-gnu-api-v53.tar.gz, /tmp/oct-kpkHBG)
copyfile /tmp/oct-kpkHBG/general-2.1.0/src/SHA1.oct /tmp/oct-kpkHBG/general-2.1.0/src/mark_for_deletion.oct /tmp/oct-kpkHBG/general-2.1.0/src/packfields.oct /tmp/oct-kpkHBG/general-2.1.0/src/unpackfields.oct /tmp/oct-kpkHBG/general-2.1.0/inst/x86_64-redhat-linux-gnu-api-v53
For information about changes from previous versions of the general package, run 'news general'.
Did not find a .metainfo.xml appdata file
explicitly decompress any DWARF compressed ELF sections in /builddir/build/BUILDROOT/octave-general-2.1.0-3.fc31.x86_64/usr/lib64/octave/packages/general-2.1.0/x86_64-redhat-linux-gnu-api-v53/SHA1.oct
explicitly decompress any DWARF compressed ELF sections in /builddir/build/BUILDROOT/octave-general-2.1.0-3.fc31.x86_64/usr/lib64/octave/packages/general-2.1.0/x86_64-redhat-linux-gnu-api-v53/unpackfields.oct
explicitly decompress any DWARF compressed ELF sections in /builddir/build/BUILDROOT/octave-general-2.1.0-3.fc31.x86_64/usr/lib64/octave/packages/general-2.1.0/x86_64-redhat-linux-gnu-api-v53/mark_for_deletion.oct
explicitly decompress any DWARF compressed ELF sections in /builddir/build/BUILDROOT/octave-general-2.1.0-3.fc31.x86_64/usr/lib64/octave/packages/general-2.1.0/x86_64-redhat-linux-gnu-api-v53/packfields.oct
extracting debug info from /builddir/build/BUILDROOT/octave-general-2.1.0-3.fc31.x86_64/usr/lib64/octave/packages/general-2.1.0/x86_64-redhat-linux-gnu-api-v53/SHA1.oct
extracting debug info from /builddir/build/BUILDROOT/octave-general-2.1.0-3.fc31.x86_64/usr/lib64/octave/packages/general-2.1.0/x86_64-redhat-linux-gnu-api-v53/unpackfields.oct
extracting debug info from /builddir/build/BUILDROOT/octave-general-2.1.0-3.fc31.x86_64/usr/lib64/octave/packages/general-2.1.0/x86_64-redhat-linux-gnu-api-v53/mark_for_deletion.oct
extracting debug info from /builddir/build/BUILDROOT/octave-general-2.1.0-3.fc31.x86_64/usr/lib64/octave/packages/general-2.1.0/x86_64-redhat-linux-gnu-api-v53/packfields.oct
original debug info size: 9740kB, size after compression: 6920kB
/usr/lib/rpm/sepdebugcrcfix: Updated 4 CRC32s, 0 CRC32s did match.
RPM build errors:
BUILDSTDERR:     attempt to use unversioned python, define %__python to /usr/bin/python2 or /usr/bin/python3 explicitly
BUILDSTDERR:     Bad exit status from /var/tmp/rpm-tmp.GIHOBr (%install)
Child return code was: 1
EXCEPTION: [Error()]
Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/mockbuild/trace_decorator.py", line 96, in trace
    result = func(*args, **kw)
  File "/usr/lib/python3.7/site-packages/mockbuild/util.py", line 736, in do_with_status
    raise exception.Error("Command failed: \n # %s\n%s" % (command, output), child.returncode)
mockbuild.exception.Error: Command failed: 
 # bash --login -c /usr/bin/rpmbuild -bb --target x86_64 --nodeps /builddir/build/SPECS/octave-general.spec


Version-Release number of selected component (if applicable):
rpm-4.14.90-0.git14653.15.fc31

Comment 1 Orion Poplawski 2019-06-17 15:23:26 UTC
Looks like this was caused by a change to python-rpm-macros to drop %{__python}, but it looks like some coordination with rpm may be required to handle changes to the python macros.  See #1721138

Comment 2 Panu Matilainen 2019-06-24 13:29:38 UTC
Yup, this is caused by redhat-rpm-config using "%{__python}" in brp-pytohn-bytecompile invocation, and when %__python is defined to an error the line terminates unexpectedly and causes the next thing in line to fail.

This has been fixed for now by python-rpm-macros defining %__python as /usr/bin/python again, but that will go away eventually so redhat-rpm-config needs to change.

Comment 3 Panu Matilainen 2019-06-27 10:04:46 UTC
Miro, any thoughts on what to do about this? 

In RHEL the %__python use was just axed away:
-%__brp_python_bytecompile /usr/lib/rpm/brp-python-bytecompile %{__python} %{?_python_bytecompile_errors_terminate_build}
+%__brp_python_bytecompile /usr/lib/rpm/brp-python-bytecompile %{?_python_bytecompile_errors_terminate_build}

...which isn't right at all when the script itself wasn't updated accordingly (I'll file a separate RHEL bug on it)

Comment 4 Miro Hrončok 2019-06-27 10:18:37 UTC
We have ~200 packages that define %_python_bytecompile_extra to 1:

audio-convert-mod
audit-viewer
bibus
bleachbit
blueberry
bro
bugzilla
calamares
cas
ceph
chromium
cinnamon-screensaver
comix
cyphesis
darkserver
dracut-modules-olpc
edk2
encuentro
eog-plugins
exaile
fedocal
firmware-tools
fish
fleet-commander-admin
fleet-commander-client
flumotion
font-manager
freecad
fwfstab
gaupol
gazebo
gdb
gdesklets
gedit-latex
gedit-plugins
gedit
git-cola
glusterfs
gnome-activity-journal
gnome-code-assistance
gnome-devel-docs
gnome-python2
gnome-transliteration
golly
graphite-web
grass
gtk-doc
gtranslator
ibus-anthy
ibus-hangul
ibus-libpinyin
ibus-libzhuyin
ibus-pinyin
ibus
ibus-table
ibus-typing-booster
ibus-uniemoji
inkscape-sozi
inkscape-table
insight
kajongg
kdevelop-python
klatexformula
kupfer
lcgdm
libgda
libopensync-plugin-python
libpeas
libreoffice
libsmbios
libunity
lilypond
lirc
lyx
mate-menu
meta-test-family
metromap
mingw-glib2
mingw-qpid-cpp
mirrormanager2
modularity-testing-framework
odcs
olpc-os-builder
ompl
onboard
openlayers
otf2
pagure-dist-git
pagure
paraview
passenger
pcs
php-opencloud-openstack
playonlinux
pygobject2
pygtk2
python3-cherrypy
python-cherrypy
python-feedparser
python-flask-silk
python-genshi
python-pycurl
python-reportlab
python-scandir
python-tahrir-api
python-testscenarios
python-webtest
qpid-dispatch
qtools
rhythmbox
sems
sigul
soletta
solfege
soundconverter
sugar-abacus
sugar-browse
sugar-calculator
sugar-castle
sugar-chat
sugar-clock
sugar-colordeducto
sugar-connect
sugar-countries
sugar-deducto
sugar-distance
sugar-finance
sugar-flip
sugar-flipsticks
sugar-fototoon
sugar-fractionbounce
sugar-getiabooks
sugar-hello-world
sugar-imageviewer
sugar-implode
sugar-infoslicer
sugar-jukebox
sugar-kuku
sugar-labyrinth
sugar-locosugar
sugar-log
sugar-maze
sugar-measure
sugar-memorize
sugar-moon
sugar-nutrition
sugar-paint
sugar-physics
sugar-pippy
sugar-playgo
sugar-portfolio
sugar-pukllanapac
sugar-read
sugar-recall
sugar-record
sugar-ruler
sugar-speak
sugar
sugar-srilanka
sugar-starchart
sugar-stopwatch
sugar-story
sugar-terminal
sugar-turtleart
sugar-typing-turtle
sugar-view-slides
sugar-visualmatch
sugar-words
sugar-write
sugar-xoeditor
sugar-xoirc
sugar-xomail
sugar-yupana
swatchbooker
synce-sync-engine
synfigstudio
system-config-bind
system-config-firewall
system-config-kdump
system-config-repo
system-config-users
system-switch-displaymanager
system-switch-java
system-switch-mail
texlive
totem
transmageddon
ufw-kde
variety
virt-manager
vtk
xed
znc

Out of that, 34 define %__python:

blueberry
calamares
edk2
eog-plugins
fish
gaupol
gdb
gedit-plugins
gedit
git-cola
gnome-code-assistance
gtk-doc
ibus-anthy
ibus
ibus-uniemoji
insight
kajongg
kdevelop-python
lirc
mingw-glib2
mirrormanager2
pagure-dist-git
pagure
pcs
python-feedparser
qpid-dispatch
system-config-kdump
system-config-repo
system-switch-mail
totem
transmageddon
ufw-kde
virt-manager
znc


Let's wait for Fedora 32 https://fedoraproject.org/wiki/Changes/RetirePython2 - most of the packages will just finally die. Later, we can say this no longer happens and adapt the script to never do the automagic bytecompilation with %__python.

Comment 5 Panu Matilainen 2019-06-27 11:02:12 UTC
I can live with that, just wanted to point out that as long as it's there, we can't make %__python an %{error:...} like the plan was. But of course, ramping down python2 is a long, long process (even by rpm standards :), I'm in no hurry.

Comment 6 Ben Cotton 2019-08-13 17:00:34 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 31 development cycle.
Changing version to '31'.

Comment 7 Ben Cotton 2019-08-13 19:08:35 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 31 development cycle.
Changing version to 31.

Comment 8 Panu Matilainen 2019-12-03 15:20:58 UTC
Moving back to rawhide.

Comment 9 Ben Cotton 2020-02-11 15:42:02 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 32 development cycle.
Changing version to 32.

Comment 10 Lumír Balhar 2020-05-18 13:00:50 UTC
Currently, ~130 packages define _python_bytecompile_extra to 1:

bugzilla
calamares
ceph
chromium
cinnamon-screensaver
edk2
eog-plugins
fish
fleet-commander-admin
fleet-commander-client
freecad
gaupol
gazebo
gdb
gedit-latex
gedit-plugins
gedit
git-cola
glusterfs
gnome-code-assistance
gnome-devel-docs
grass
gtk-doc
gtranslator
ibus-anthy
ibus-hangul
ibus-libpinyin
ibus-libzhuyin
ibus-pinyin
ibus
ibus-table
ibus-typing-booster
ibus-uniemoji
kajongg
kdevelop-python
klatexformula
libpeas
libreoffice
libsmbios
libunity
lirc
lyx
mate-menu
mingw-glib2
mirrormanager2
odcs
onboard
otf2
paraview
pcs
php-opencloud-openstack
pygobject2
pygtk2
python-cherrypy
python-flask-silk
python-genshi
python-pycurl
python-reportlab
qpid-dispatch
rhythmbox
sems
sigul
soundconverter
sugar-abacus
sugar-browse
sugar-calculator
sugar-castle
sugar-chat
sugar-clock
sugar-colordeducto
sugar-countries
sugar-deducto
sugar-distance
sugar-finance
sugar-flip
sugar-flipsticks
sugar-fototoon
sugar-fractionbounce
sugar-getiabooks
sugar-imageviewer
sugar-implode
sugar-infoslicer
sugar-jukebox
sugar-kuku
sugar-labyrinth
sugar-locosugar
sugar-log
sugar-maze
sugar-measure
sugar-memorize
sugar-moon
sugar-nutrition
sugar-paint
sugar-physics
sugar-pippy
sugar-playgo
sugar-portfolio
sugar-pukllanapac
sugar-read
sugar-recall
sugar-record
sugar-ruler
sugar-speak
sugar
sugar-srilanka
sugar-starchart
sugar-stopwatch
sugar-story
sugar-terminal
sugar-turtleart
sugar-typing-turtle
sugar-view-slides
sugar-visualmatch
sugar-words
sugar-write
sugar-xoeditor
sugar-xoirc
sugar-yupana
synfigstudio
system-config-repo
system-switch-java
system-switch-mail
texlive
totem
transmageddon
ufw-kde
variety
virt-manager
vtk
xed

Out of that, 29 define %__python:

calamares
ceph
edk2
eog-plugins
fish
gaupol
gdb
gedit
gedit-plugins
gnome-code-assistance
grass
gtk-doc
ibus
ibus-anthy
ibus-uniemoji
kajongg
kdevelop-python
lirc
mingw-glib2
mirrormanager2
pcs
qpid-dispatch
system-config-repo
system-switch-mail
totem
transmageddon
ufw-kde
variety
virt-manager

Our plan is to entirely remove the _python_bytecompile_extra functionality and let the packagers to use explicit %py_byte_compile for cases when it's needed.

I'll inspect the packages above more deeply because I am pretty sure not all of them have something to compile by _python_bytecompile_extra. I'll also send an email to the maintainers and to fedora-devel about our plans.

Comment 12 Lumír Balhar 2020-06-08 12:44:41 UTC
Fesco issue: https://pagure.io/fesco/issue/2401

Comment 13 Lumír Balhar 2020-06-16 05:30:33 UTC
The self-contained change has been approved by Fesco. redhat-rpm-config update: https://bodhi.fedoraproject.org/updates/FEDORA-2020-922b21ffde

I'm gonna close this bug and continue in https://bugzilla.redhat.com/show_bug.cgi?id=1847153


Note You need to log in before you can comment on or make changes to this bug.