Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1374667

Summary:	A lot of python-bytecode-inconsistent-mtime
Product:	Red Hat Software Collections	Reporter:	Tomas Orsava <torsava>
Component:	python	Assignee:	Tomas Orsava <torsava>
Status:	CLOSED ERRATA	QA Contact:	Branislav Náter <bnater>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	rh-python35	CC:	bkabrda, bnater, cstratak, extras-qa, kanderso, mhroncok, pviktori, rkuska, tomspur, torsava
Target Milestone:	beta
Target Release:	2.3
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	rh-python35-python-3.5.1-11.el7 rh-python35-python-3.5.1-11.el6	Doc Type:	No Doc Update
Doc Text:	undefined	Story Points:	---
Clone Of:	1373635	Environment:
Last Closed:	2016-11-15 10:00:24 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Tomas Orsava 2016-09-09 11:20:40 UTC

+++ This bug was initially created as a clone of Bug #1373635 +++

Description of problem:
When rpmlint is run on python3-libs, you get a lot of python-bytecode-inconsistent-mtime

See explanation:

$ rpmlint -I python-bytecode-inconsistent-mtime
python-bytecode-inconsistent-mtime:
The timestamp embedded in this python bytecode file isn't equal to the mtime
of the original source file, which will force the interpreter to recompile the
.py source every time, ignoring the saved bytecode.


Version-Release number of selected component (if applicable):
python3-libs-3.5.2-1.fc25

How reproducible: Easy

Steps to Reproduce:
1. rpmlint python3-libs | grep python-bytecode-inconsistent-mtime

Actual results:
Attached

Expected results:
Nothing

Additional info:
Happens since 3.5.0-1
Does not happen on 3.4
Does not happen on 2.7

With my python35 builds, it happens as well, also with python26, but not with python34, python33.

--- Additional comment from Miro Hrončok on 2016-09-06 22:06:12 CEST ---

Also happens with 3.6 from https://copr.fedorainfracloud.org/coprs/cstratak/python-3.6/

--- Additional comment from Charalampos Stratakis on 2016-09-07 20:43:34 CEST ---

It's not exclusive on python3-libs rpm. Every other subpackage that includes py and pyc files has the same warnings.

Either something is fundamentally wrong with the SPEC file (or maybe on how RPM handles some things) or it is a false positive.

Either way it needs investigation.

No idea why this happens on python26, however for Python 3.5 I guess it relates to this [PEP] somehow.

[PEP]: https://www.python.org/dev/peps/pep-0488/

So far I haven't been able to debug the exact phase of where the timestamps change, during the creation of the rpm.

--- Additional comment from Fedora Blocker Bugs Application on 2016-09-08 10:41:02 CEST ---

Proposed as a Blocker for 25-beta by Fedora user churchyard using the blocker tracking app because:

 Upgrade form F23 to F25 is not working with default Workstation install. "For each one of the release-blocking package sets, it must be possible to successfully complete a direct upgrade from fully updated installations of the last two stable Fedora releases with that package set installed. The release-blocking package sets are the minimal set, and the sets for each one of the release-blocking desktops, on all primary Architectures except i686."

--- Additional comment from Miro Hrončok on 2016-09-08 10:47:24 CEST ---

That was a mistake, I had bad bug number in clipboard, sorry about that.

--- Additional comment from Tomas Orsava on 2016-09-08 18:43:33 CEST ---

I have found out what's causing it, it's 3 different commands in the %install section that modify the files:

1. # Fix end-of-line encodings:

2. # Switch all shebangs to refer to the specific Python version.

3. # Remove shebang lines from .py files that aren't executable, and
# remove executability from .py files that don't have a shebang line:


I'm getting close to a fix.

--- Additional comment from Miro Hrončok on 2016-09-08 19:09:24 CEST ---

# Do bytecompilation with the newly installed interpreter

happens after all of those.

--- Additional comment from Charalampos Stratakis on 2016-09-09 10:56:29 CEST ---

Please report any findings here. If someone figures it out I would like the fix to be ideally pushed together with rhbz#1374357

--- Additional comment from Tomas Orsava on 2016-09-09 12:45:37 CEST ---

(In reply to Miro Hrončok from comment #6)
> # Do bytecompilation with the newly installed interpreter
> 
> happens after all of those.

Thank you, Miro, for that suggestion, it saved me a lot of time.

So, here's what was wrong: The aforementioned 3 commands in the %install section were modifying the .py files, thus the .pyc files need to be rebuilt.

Indeed the "# Do bytecompilation with the newly installed interpreter" part comes after and should rebuild all the .pyc files, however, it only rebuilds .pyc files with "optimize=0". It leaves out .pyc files with optimization level 1 and 2, which didn't used to exist, but now they do:

https://www.python.org/dev/peps/pep-0488/

I modified the script to rebuild .pyc files of all optimization levels and the problem is solved.

--- Additional comment from Miro Hrončok on 2016-09-09 13:00:20 CEST ---

In python26 the steps (fix shebangs, bytecompile) wre in bad order, trying to rebuild using the correct order.

--- Additional comment from Miro Hrončok on 2016-09-09 13:15:14 CEST ---

(In reply to Miro Hrončok from comment #9)
> In python26 the steps (fix shebangs, bytecompile) wre in bad order, trying
> to rebuild using the correct order.

Didn't help.

Comment 2 Tomas Orsava 2016-09-09 12:32:22 UTC

This bug was erroneously filed for the metapackage component, correcting my mistake.

Comment 6 Branislav Náter 2016-10-26 13:58:34 UTC

rpmlint version used: rpmlint-1.9-3.fc25

Reproduced on rhscl-2.2 packages:
# for file in *.rpm; do echo $file; rpmlint $file | grep python-bytecode-inconsistent-mtime; done
rh-python35-python-devel-3.5.1-9.el7.x86_64.rpm
rh-python35-python-tkinter-3.5.1-9.el7.x86_64.rpm
rh-python35-python-tkinter.x86_64: E: python-bytecode-inconsistent-mtime /opt/rh/rh-python35/root/usr/lib64/python3.5/tkinter/__pycache__/commondialog.cpython-35.opt-2.pyc 2015-12-06T20:39:09 /opt/rh/rh-python35/root/usr/lib64/python3.5/tkinter/commondialog.py 2016-08-06T06:53:24
<snip>
rh-python35-python-3.5.1-9.el7.x86_64.rpm
rh-python35-python-docutils-0.12-1.el7.noarch.rpm
rh-python35-python-tools-3.5.1-9.el7.x86_64.rpm
rh-python35-python-debug-3.5.1-9.el7.x86_64.rpm
rh-python35-python-debug.x86_64: E: python-bytecode-inconsistent-mtime /opt/rh/rh-python35/root/usr/lib64/python3.5/config-3.5dm/__pycache__/python-config.cpython-35.opt-2.pyc 2016-08-06T06:51:13 /opt/rh/rh-python35/root/usr/lib64/python3.5/config-3.5dm/python-config.py 2016-08-06T06:53:23
rh-python35-python-libs-3.5.1-9.el7.x86_64.rpm
rh-python35-python-libs.x86_64: E: python-bytecode-inconsistent-mtime /opt/rh/rh-python35/root/usr/lib64/python3.5/__pycache__/pickle.cpython-35.opt-2.pyc 2015-12-06T20:39:08 /opt/rh/rh-python35/root/usr/lib64/python3.5/pickle.py 2016-08-06T06:53:20
<snip>
rh-python35-python-virtualenv-13.1.2-1.el7.noarch.rpm
rh-python35-python-debuginfo-3.5.1-9.el7.x86_64.rpm
rh-python35-python-test-3.5.1-9.el7.x86_64.rpm
rh-python35-python-test.x86_64: E: python-bytecode-inconsistent-mtime /opt/rh/rh-python35/root/usr/lib64/python3.5/ctypes/test/__pycache__/test_simplesubclasses.cpython-35.opt-2.pyc 2015-12-06T20:39:07 /opt/rh/rh-python35/root/usr/lib64/python3.5/ctypes/test/test_simplesubclasses.py 2016-08-06T06:53:20
<snip>
#

Verified on all rhscl-2.3 packages:
# for file in *.rpm; do echo $file; rpmlint $file | grep python-bytecode-inconsistent-mtime; done
rh-python35-2.0-2.el6.x86_64.rpm
rh-python35-babel-1.3-2.el6.noarch.rpm
rh-python35-mod_wsgi-4.4.21-1.el6.x86_64.rpm
rh-python35-mod_wsgi-debuginfo-4.4.21-1.el6.x86_64.rpm
rh-python35-numpy-1.10.4-4.el6.x86_64.rpm
rh-python35-numpy-debuginfo-1.10.4-4.el6.x86_64.rpm
rh-python35-numpy-f2py-1.10.4-4.el6.x86_64.rpm
rh-python35-python-3.5.1-11.el6.x86_64.rpm
rh-python35-python-3.5.1-11.el7.x86_64.rpm
rh-python35-python-babel-1.3-2.el6.noarch.rpm
rh-python35-python-bson-3.2.1-2.el6.x86_64.rpm
rh-python35-python-coverage-4.0.3-1.el6.x86_64.rpm
rh-python35-python-coverage-debuginfo-4.0.3-1.el6.x86_64.rpm
rh-python35-python-debug-3.5.1-11.el6.x86_64.rpm
rh-python35-python-debug-3.5.1-11.el7.x86_64.rpm
rh-python35-python-debuginfo-3.5.1-11.el6.x86_64.rpm
rh-python35-python-debuginfo-3.5.1-11.el7.x86_64.rpm
rh-python35-python-devel-3.5.1-11.el6.x86_64.rpm
rh-python35-python-devel-3.5.1-11.el7.x86_64.rpm
rh-python35-python-docutils-0.12-2.el6.noarch.rpm
rh-python35-python-docutils-0.12-2.el7.noarch.rpm
rh-python35-python-jinja2-2.8-2.el6.noarch.rpm
rh-python35-python-libs-3.5.1-11.el6.x86_64.rpm
rh-python35-python-libs-3.5.1-11.el7.x86_64.rpm
rh-python35-python-markupsafe-0.23-1.el6.x86_64.rpm
rh-python35-python-markupsafe-debuginfo-0.23-1.el6.x86_64.rpm
rh-python35-python-nose-1.3.7-2.el6.noarch.rpm
rh-python35-python-nose-docs-1.3.7-2.el6.noarch.rpm
rh-python35-python-pip-7.1.0-2.el6.noarch.rpm
rh-python35-python-psycopg2-2.6.1-1.el6.x86_64.rpm
rh-python35-python-psycopg2-debuginfo-2.6.1-1.el6.x86_64.rpm
rh-python35-python-psycopg2-doc-2.6.1-1.el6.x86_64.rpm
rh-python35-python-pygments-2.1-1.el6.noarch.rpm
rh-python35-python-pymongo-3.2.1-2.el6.x86_64.rpm
rh-python35-python-pymongo-debuginfo-3.2.1-2.el6.x86_64.rpm
rh-python35-python-pymongo-doc-3.2.1-2.el6.x86_64.rpm
rh-python35-python-pymongo-gridfs-3.2.1-2.el6.x86_64.rpm
rh-python35-python-PyMySQL-0.7.5-1.el6.noarch.rpm
rh-python35-python-PyMySQL-0.7.5-1.el7.noarch.rpm
rh-python35-python-setuptools-18.0.1-2.el6.noarch.rpm
rh-python35-python-simplejson-3.6.5-1.el6.x86_64.rpm
rh-python35-python-simplejson-debuginfo-3.6.5-1.el6.x86_64.rpm
rh-python35-python-six-1.10.0-1.el6.noarch.rpm
rh-python35-python-sphinx-1.2.2-2.el6.noarch.rpm
rh-python35-python-sphinx-doc-1.2.2-2.el6.noarch.rpm
rh-python35-python-sqlalchemy-1.0.11-1.el6.x86_64.rpm
rh-python35-python-sqlalchemy-debuginfo-1.0.11-1.el6.x86_64.rpm
rh-python35-python-test-3.5.1-11.el6.x86_64.rpm
rh-python35-python-test-3.5.1-11.el7.x86_64.rpm
rh-python35-python-tkinter-3.5.1-11.el6.x86_64.rpm
rh-python35-python-tkinter-3.5.1-11.el7.x86_64.rpm
rh-python35-python-tools-3.5.1-11.el6.x86_64.rpm
rh-python35-python-tools-3.5.1-11.el7.x86_64.rpm
rh-python35-python-virtualenv-13.1.2-2.el6.noarch.rpm
rh-python35-python-virtualenv-13.1.2-2.el7.noarch.rpm
rh-python35-python-wheel-0.24.0-1.el6.noarch.rpm
rh-python35-PyYAML-3.10-14.el6.x86_64.rpm
rh-python35-PyYAML-debuginfo-3.10-14.el6.x86_64.rpm
rh-python35-runtime-2.0-2.el6.x86_64.rpm
rh-python35-scipy-0.17.0-2.el6.x86_64.rpm
rh-python35-scipy-debuginfo-0.17.0-2.el6.x86_64.rpm
rh-python35-scldevel-2.0-2.el6.x86_64.rpm
#

Comment 8 errata-xmlrpc 2016-11-15 10:00:24 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-2720.html