Bug 1023607 - python3-libs is not using the prebuilt .pyc files
python3-libs is not using the prebuilt .pyc files
Product: Fedora
Classification: Fedora
Component: python3 (Show other bugs)
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Bohuslav "Slavek" Kabrda
Fedora Extras Quality Assurance
Depends On:
  Show dependency treegraph
Reported: 2013-10-25 18:49 EDT by Toshio Ernie Kuratomi
Modified: 2013-11-10 01:22 EST (History)
4 users (show)

See Also:
Fixed In Version: python3-3.3.2-7.fc20
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2013-11-07 23:32:43 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

External Trackers
Tracker ID Priority Status Summary Last Updated
Python 19447 None None None Never

  None (edit)
Description Toshio Ernie Kuratomi 2013-10-25 18:49:42 EDT
Description of problem:

For some reason the prebuilt .pyc files from python3-libs are not being used.  Instead, python3 is recomputing the byte code from the source files.

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:

$ su -
# yum reinstall -y python3-libs
# rpm -V python3-libs
# python3 -c 'import warnings'
# rpm -V python3-libs

Actual results:

..5....T.    /usr/lib64/python3.3/__pycache__/zipfile.cpython-33.pyc

Expected results:

Expect rpm -V to not show any changed files

Additional info:

This does not occur with third party modules (for instance, python3-setuptools) so it seems like some problem with the python3 package's build.
Comment 1 Bohuslav "Slavek" Kabrda 2013-10-29 06:18:36 EDT
Hmm, interesting, thanks for the bug report.
The problem seems to be that the timestamps of pyc files really say that pyc files are older than py files:

[bkabrda@zizalka ~]$ stat /usr/lib64/python3.3/warnings.py 
  File: ‘/usr/lib64/python3.3/warnings.py’
Modify: 2013-08-23 21:01:34.000000000 +0200
Change: 2013-10-29 10:21:03.813749550 +0100

[bkabrda@zizalka ~]$ stat /usr/lib64/python3.3/__pycache__/warnings.cpython-33.pyc
Modify: 2013-08-23 21:01:22.000000000 +0200
Change: 2013-10-29 10:21:01.902757847 +0100

My guess is that problem is somewhere during rpmbuild, where py files may be unintentionally touched. I'll investigate.
Comment 2 Bohuslav "Slavek" Kabrda 2013-10-29 10:19:34 EDT
Ok, the problem seems to be much simpler than that. The py_compile.compile function (used to do the bytecompilation in %install) in 3.3 is protected from all sorts of exceptions except exceptions raised from tokenize.open [1].
py_compile.compile is called on all *.py files, even test files with bad encoding (/builddir/build/BUILDROOT/python3-3.3.2-3.fc21.x86_64/usr/lib64/python3.3/test/bad_coding2.py). This makes tokenize.open raise an error that is not caught. Therefore, all files passed by xargs before bad_coding2.py are recompiled, the others aren't and thus they get to the binary RPM with older timestamps.
I'll create a patch for this and rebuild (hopefully) tomorrow.

[1] http://hg.python.org/cpython/file/321b665d8c3c/Lib/py_compile.py#l111
Comment 3 Fedora Update System 2013-10-30 10:01:08 EDT
python3-3.3.0-4.fc18 has been submitted as an update for Fedora 18.
Comment 4 Fedora Update System 2013-10-30 10:01:32 EDT
python3-3.3.2-7.fc19 has been submitted as an update for Fedora 19.
Comment 5 Fedora Update System 2013-10-30 10:01:48 EDT
python3-3.3.2-7.fc20 has been submitted as an update for Fedora 20.
Comment 6 Bohuslav "Slavek" Kabrda 2013-10-30 10:06:06 EDT
Fixed in the builds mentioned above in stable Fedoras + python3-3.3.2-7.fc21 in rawhide.
With the fix, I also added a script that checks proper timestamps in the %check section and fails if they're wrong, so this should never happen again.
Comment 7 Fedora Update System 2013-10-30 13:10:03 EDT
Package python3-3.3.2-7.fc20:
* should fix your issue,
* was pushed to the Fedora 20 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing python3-3.3.2-7.fc20'
as soon as you are able to.
Please go to the following url:
then log in and leave karma (feedback).
Comment 8 Fedora Update System 2013-11-07 23:32:43 EST
python3-3.3.0-4.fc18 has been pushed to the Fedora 18 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 9 Fedora Update System 2013-11-07 23:33:40 EST
python3-3.3.2-7.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 10 Fedora Update System 2013-11-10 01:22:07 EST
python3-3.3.2-7.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.