Bug 1023607

Summary: python3-libs is not using the prebuilt .pyc files
Product: [Fedora] Fedora Reporter: Toshio Ernie Kuratomi <a.badger>
Component: python3Assignee: Bohuslav "Slavek" Kabrda <bkabrda>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rawhideCC: amcnabb, bkabrda, mstuchli, tomspur
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: python3-3.3.2-7.fc20 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-11-08 04:32:43 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Toshio Ernie Kuratomi 2013-10-25 22:49:42 UTC
Description of problem:

For some reason the prebuilt .pyc files from python3-libs are not being used.  Instead, python3 is recomputing the byte code from the source files.



Version-Release number of selected component (if applicable):
python3-libs-3.3.2-6.fc19.x86_64

How reproducible:
Everytime

Steps to Reproduce:

$ su -
# yum reinstall -y python3-libs
# rpm -V python3-libs
# python3 -c 'import warnings'
# rpm -V python3-libs


Actual results:

..5....T.    /usr/lib64/python3.3/__pycache__/zipfile.cpython-33.pyc


Expected results:

Expect rpm -V to not show any changed files

Additional info:

This does not occur with third party modules (for instance, python3-setuptools) so it seems like some problem with the python3 package's build.

Comment 1 Bohuslav "Slavek" Kabrda 2013-10-29 10:18:36 UTC
Hmm, interesting, thanks for the bug report.
The problem seems to be that the timestamps of pyc files really say that pyc files are older than py files:


[bkabrda@zizalka ~]$ stat /usr/lib64/python3.3/warnings.py 
  File: ‘/usr/lib64/python3.3/warnings.py’
...
Modify: 2013-08-23 21:01:34.000000000 +0200
Change: 2013-10-29 10:21:03.813749550 +0100
...

[bkabrda@zizalka ~]$ stat /usr/lib64/python3.3/__pycache__/warnings.cpython-33.pyc
...
Modify: 2013-08-23 21:01:22.000000000 +0200
Change: 2013-10-29 10:21:01.902757847 +0100
...


My guess is that problem is somewhere during rpmbuild, where py files may be unintentionally touched. I'll investigate.

Comment 2 Bohuslav "Slavek" Kabrda 2013-10-29 14:19:34 UTC
Ok, the problem seems to be much simpler than that. The py_compile.compile function (used to do the bytecompilation in %install) in 3.3 is protected from all sorts of exceptions except exceptions raised from tokenize.open [1].
py_compile.compile is called on all *.py files, even test files with bad encoding (/builddir/build/BUILDROOT/python3-3.3.2-3.fc21.x86_64/usr/lib64/python3.3/test/bad_coding2.py). This makes tokenize.open raise an error that is not caught. Therefore, all files passed by xargs before bad_coding2.py are recompiled, the others aren't and thus they get to the binary RPM with older timestamps.
I'll create a patch for this and rebuild (hopefully) tomorrow.


[1] http://hg.python.org/cpython/file/321b665d8c3c/Lib/py_compile.py#l111

Comment 3 Fedora Update System 2013-10-30 14:01:08 UTC
python3-3.3.0-4.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/python3-3.3.0-4.fc18

Comment 4 Fedora Update System 2013-10-30 14:01:32 UTC
python3-3.3.2-7.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/python3-3.3.2-7.fc19

Comment 5 Fedora Update System 2013-10-30 14:01:48 UTC
python3-3.3.2-7.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/python3-3.3.2-7.fc20

Comment 6 Bohuslav "Slavek" Kabrda 2013-10-30 14:06:06 UTC
Fixed in the builds mentioned above in stable Fedoras + python3-3.3.2-7.fc21 in rawhide.
With the fix, I also added a script that checks proper timestamps in the %check section and fails if they're wrong, so this should never happen again.

Comment 7 Fedora Update System 2013-10-30 17:10:03 UTC
Package python3-3.3.2-7.fc20:
* should fix your issue,
* was pushed to the Fedora 20 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing python3-3.3.2-7.fc20'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-20308/python3-3.3.2-7.fc20
then log in and leave karma (feedback).

Comment 8 Fedora Update System 2013-11-08 04:32:43 UTC
python3-3.3.0-4.fc18 has been pushed to the Fedora 18 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 9 Fedora Update System 2013-11-08 04:33:40 UTC
python3-3.3.2-7.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 10 Fedora Update System 2013-11-10 06:22:07 UTC
python3-3.3.2-7.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.