Bug 1550562

Summary: python-bytecode-inconsistent-mtime gives bogus result with Python 3.7 bytecode
Product: [Fedora] Fedora Reporter: Miro Hrončok <mhroncok>
Component: rpmlintAssignee: Tom "spot" Callaway <tcallawa>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rawhideCC: j, mhroncok, tcallawa, tmz, twoerner
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: rpmlint-1.10-13.fc29 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-06-09 06:56:54 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Miro Hrončok 2018-03-01 13:04:23 UTC
Description of problem: The mtime of all pyc files is 1970-01-01T00:00:00.


Version-Release number of selected component (if applicable):
python37-3.7.0-0.12.b2


Steps to Reproduce:
1. rpmlint ./python37-3.7.0-0.12.b2.fc29.x86_64.rpm  | grep python-bytecode-inconsistent-mtime

Actual results:
python37.x86_64: E: python-bytecode-inconsistent-mtime /usr/lib64/python3.7/__pycache__/__future__.cpython-37.opt-1.pyc 1970-01-01T01:00:00 /usr/lib64/python3.7/__future__.py 2018-02-28T01:49:18
python37.x86_64: E: python-bytecode-inconsistent-mtime /usr/lib64/python3.7/__pycache__/__future__.cpython-37.opt-2.pyc 1970-01-01T01:00:00 /usr/lib64/python3.7/__future__.py 2018-02-28T01:49:18
python37.x86_64: E: python-bytecode-inconsistent-mtime /usr/lib64/python3.7/__pycache__/__future__.cpython-37.pyc 1970-01-01T01:00:00 /usr/lib64/python3.7/__future__.py 2018-02-28T01:49:18
python37.x86_64: E: python-bytecode-inconsistent-mtime /usr/lib64/python3.7/__pycache__/__phello__.foo.cpython-37.opt-1.pyc 1970-01-01T01:00:00 /usr/lib64/python3.7/__phello__.foo.py 2018-02-28T01:49:18
...

Expected results:
nothing


Additional info:
See also Bug 1373635, yet this is different (all the mtimes are 1970-01-01T01:00:00 here).

Comment 1 Miro Hrončok 2018-03-01 13:28:31 UTC
This might or might not be related to https://docs.python.org/dev/whatsnew/3.7.html#hash-based-pycs

Also:

$ LANG=C.utf-8 rpm -qlvp   .../python37-3.7.0-0.12.b2.fc29.x86_64.rpm (shortened)
Feb 28 01:49 /usr/lib64/python3.7/__future__.py
Feb 28 14:27 /usr/lib64/python3.7/__pycache__/__future__.cpython-37.opt-1.pyc
Feb 28 14:27 /usr/lib64/python3.7/__pycache__/__future__.cpython-37.opt-2.pyc
Feb 28 14:27 /usr/lib64/python3.7/__pycache__/__future__.cpython-37.pyc

i.e. not 1970-01-01T01:00:00

Comment 2 Miro Hrončok 2018-03-01 16:00:08 UTC
So this is not the mtime of the pyc file, but rpmlint reads the mtime from withing the pyc file.

From PEP 552:

The pyc header currently consists of 3 32-bit words. We will expand it to 4. The first word will continue to be the magic number, versioning the bytecode and pyc format. The second word, conceptually the new word, will be a bit field. The interpretation of the rest of the header and invalidation behavior of the pyc depends on the contents of the bit field.

So apparently rpmlint reads this second word by:

    pyc_timestamp = py_demarshal_long(chunk[4:8])

And gets zero.

Comment 4 Miro Hrončok 2018-06-08 17:38:13 UTC
Should I keep this open for Fedora 28 and 27?

Comment 5 Tom "spot" Callaway 2018-06-08 18:43:52 UTC
Well, seeing as how I still can't actually build rpmlint in rawhide, I think it would be premature to call this closed.

Comment 6 Miro Hrončok 2018-06-08 19:25:25 UTC
Oh. I failed to notice that it fails. let me guess, flake8?

I'll send you a PR.

Comment 8 Tom "spot" Callaway 2018-06-08 20:52:48 UTC
Okay, thanks. Do we actually need this fix pushed back for F27/F28?

Comment 9 Miro Hrončok 2018-06-09 06:56:54 UTC
The python37 package will scream, but I can live with that.

https://taskotron.fedoraproject.org/artifacts/all/9fb3ddc6-6554-11e8-8feb-525400fc9f92/tests.yml/python37-3.7.0-0.19.b5.fc28.log

It's not really necesery. Thanks.