Bug 1023607 - python3-libs is not using the prebuilt .pyc files
Summary: python3-libs is not using the prebuilt .pyc files
Alias: None
Product: Fedora
Classification: Fedora
Component: python3
Version: rawhide
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
Assignee: Bohuslav "Slavek" Kabrda
QA Contact: Fedora Extras Quality Assurance
Depends On:
TreeView+ depends on / blocked
Reported: 2013-10-25 22:49 UTC by Toshio Ernie Kuratomi
Modified: 2013-11-10 06:22 UTC (History)
4 users (show)

Fixed In Version: python3-3.3.2-7.fc20
Doc Type: Bug Fix
Doc Text:
Clone Of:
Last Closed: 2013-11-08 04:32:43 UTC
Type: Bug

Attachments (Terms of Use)

System ID Private Priority Status Summary Last Updated
Python 19447 0 None None None Never

Description Toshio Ernie Kuratomi 2013-10-25 22:49:42 UTC
Description of problem:

For some reason the prebuilt .pyc files from python3-libs are not being used.  Instead, python3 is recomputing the byte code from the source files.

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:

$ su -
# yum reinstall -y python3-libs
# rpm -V python3-libs
# python3 -c 'import warnings'
# rpm -V python3-libs

Actual results:

..5....T.    /usr/lib64/python3.3/__pycache__/zipfile.cpython-33.pyc

Expected results:

Expect rpm -V to not show any changed files

Additional info:

This does not occur with third party modules (for instance, python3-setuptools) so it seems like some problem with the python3 package's build.

Comment 1 Bohuslav "Slavek" Kabrda 2013-10-29 10:18:36 UTC
Hmm, interesting, thanks for the bug report.
The problem seems to be that the timestamps of pyc files really say that pyc files are older than py files:

[bkabrda@zizalka ~]$ stat /usr/lib64/python3.3/warnings.py 
  File: ‘/usr/lib64/python3.3/warnings.py’
Modify: 2013-08-23 21:01:34.000000000 +0200
Change: 2013-10-29 10:21:03.813749550 +0100

[bkabrda@zizalka ~]$ stat /usr/lib64/python3.3/__pycache__/warnings.cpython-33.pyc
Modify: 2013-08-23 21:01:22.000000000 +0200
Change: 2013-10-29 10:21:01.902757847 +0100

My guess is that problem is somewhere during rpmbuild, where py files may be unintentionally touched. I'll investigate.

Comment 2 Bohuslav "Slavek" Kabrda 2013-10-29 14:19:34 UTC
Ok, the problem seems to be much simpler than that. The py_compile.compile function (used to do the bytecompilation in %install) in 3.3 is protected from all sorts of exceptions except exceptions raised from tokenize.open [1].
py_compile.compile is called on all *.py files, even test files with bad encoding (/builddir/build/BUILDROOT/python3-3.3.2-3.fc21.x86_64/usr/lib64/python3.3/test/bad_coding2.py). This makes tokenize.open raise an error that is not caught. Therefore, all files passed by xargs before bad_coding2.py are recompiled, the others aren't and thus they get to the binary RPM with older timestamps.
I'll create a patch for this and rebuild (hopefully) tomorrow.

[1] http://hg.python.org/cpython/file/321b665d8c3c/Lib/py_compile.py#l111

Comment 3 Fedora Update System 2013-10-30 14:01:08 UTC
python3-3.3.0-4.fc18 has been submitted as an update for Fedora 18.

Comment 4 Fedora Update System 2013-10-30 14:01:32 UTC
python3-3.3.2-7.fc19 has been submitted as an update for Fedora 19.

Comment 5 Fedora Update System 2013-10-30 14:01:48 UTC
python3-3.3.2-7.fc20 has been submitted as an update for Fedora 20.

Comment 6 Bohuslav "Slavek" Kabrda 2013-10-30 14:06:06 UTC
Fixed in the builds mentioned above in stable Fedoras + python3-3.3.2-7.fc21 in rawhide.
With the fix, I also added a script that checks proper timestamps in the %check section and fails if they're wrong, so this should never happen again.

Comment 7 Fedora Update System 2013-10-30 17:10:03 UTC
Package python3-3.3.2-7.fc20:
* should fix your issue,
* was pushed to the Fedora 20 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing python3-3.3.2-7.fc20'
as soon as you are able to.
Please go to the following url:
then log in and leave karma (feedback).

Comment 8 Fedora Update System 2013-11-08 04:32:43 UTC
python3-3.3.0-4.fc18 has been pushed to the Fedora 18 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 9 Fedora Update System 2013-11-08 04:33:40 UTC
python3-3.3.2-7.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 10 Fedora Update System 2013-11-10 06:22:07 UTC
python3-3.3.2-7.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.