Bug 1023607 - python3-libs is not using the prebuilt .pyc files
Summary: python3-libs is not using the prebuilt .pyc files
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: python3
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Bohuslav "Slavek" Kabrda
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-10-25 22:49 UTC by Toshio Ernie Kuratomi
Modified: 2013-11-10 06:22 UTC (History)
4 users (show)

Fixed In Version: python3-3.3.2-7.fc20
Clone Of:
Environment:
Last Closed: 2013-11-08 04:32:43 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Python 19447 0 None None None Never

Description Toshio Ernie Kuratomi 2013-10-25 22:49:42 UTC
Description of problem:

For some reason the prebuilt .pyc files from python3-libs are not being used.  Instead, python3 is recomputing the byte code from the source files.



Version-Release number of selected component (if applicable):
python3-libs-3.3.2-6.fc19.x86_64

How reproducible:
Everytime

Steps to Reproduce:

$ su -
# yum reinstall -y python3-libs
# rpm -V python3-libs
# python3 -c 'import warnings'
# rpm -V python3-libs


Actual results:

..5....T.    /usr/lib64/python3.3/__pycache__/zipfile.cpython-33.pyc


Expected results:

Expect rpm -V to not show any changed files

Additional info:

This does not occur with third party modules (for instance, python3-setuptools) so it seems like some problem with the python3 package's build.

Comment 1 Bohuslav "Slavek" Kabrda 2013-10-29 10:18:36 UTC
Hmm, interesting, thanks for the bug report.
The problem seems to be that the timestamps of pyc files really say that pyc files are older than py files:


[bkabrda@zizalka ~]$ stat /usr/lib64/python3.3/warnings.py 
  File: ‘/usr/lib64/python3.3/warnings.py’
...
Modify: 2013-08-23 21:01:34.000000000 +0200
Change: 2013-10-29 10:21:03.813749550 +0100
...

[bkabrda@zizalka ~]$ stat /usr/lib64/python3.3/__pycache__/warnings.cpython-33.pyc
...
Modify: 2013-08-23 21:01:22.000000000 +0200
Change: 2013-10-29 10:21:01.902757847 +0100
...


My guess is that problem is somewhere during rpmbuild, where py files may be unintentionally touched. I'll investigate.

Comment 2 Bohuslav "Slavek" Kabrda 2013-10-29 14:19:34 UTC
Ok, the problem seems to be much simpler than that. The py_compile.compile function (used to do the bytecompilation in %install) in 3.3 is protected from all sorts of exceptions except exceptions raised from tokenize.open [1].
py_compile.compile is called on all *.py files, even test files with bad encoding (/builddir/build/BUILDROOT/python3-3.3.2-3.fc21.x86_64/usr/lib64/python3.3/test/bad_coding2.py). This makes tokenize.open raise an error that is not caught. Therefore, all files passed by xargs before bad_coding2.py are recompiled, the others aren't and thus they get to the binary RPM with older timestamps.
I'll create a patch for this and rebuild (hopefully) tomorrow.


[1] http://hg.python.org/cpython/file/321b665d8c3c/Lib/py_compile.py#l111

Comment 3 Fedora Update System 2013-10-30 14:01:08 UTC
python3-3.3.0-4.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/python3-3.3.0-4.fc18

Comment 4 Fedora Update System 2013-10-30 14:01:32 UTC
python3-3.3.2-7.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/python3-3.3.2-7.fc19

Comment 5 Fedora Update System 2013-10-30 14:01:48 UTC
python3-3.3.2-7.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/python3-3.3.2-7.fc20

Comment 6 Bohuslav "Slavek" Kabrda 2013-10-30 14:06:06 UTC
Fixed in the builds mentioned above in stable Fedoras + python3-3.3.2-7.fc21 in rawhide.
With the fix, I also added a script that checks proper timestamps in the %check section and fails if they're wrong, so this should never happen again.

Comment 7 Fedora Update System 2013-10-30 17:10:03 UTC
Package python3-3.3.2-7.fc20:
* should fix your issue,
* was pushed to the Fedora 20 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing python3-3.3.2-7.fc20'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-20308/python3-3.3.2-7.fc20
then log in and leave karma (feedback).

Comment 8 Fedora Update System 2013-11-08 04:32:43 UTC
python3-3.3.0-4.fc18 has been pushed to the Fedora 18 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 9 Fedora Update System 2013-11-08 04:33:40 UTC
python3-3.3.2-7.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 10 Fedora Update System 2013-11-10 06:22:07 UTC
python3-3.3.2-7.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.