Bug 1456722 - kvm_stat adds python2 dependency
Summary: kvm_stat adds python2 dependency
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: kernel
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Kernel Maintainer List
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: IoT
TreeView+ depends on / blocked
 
Reported: 2017-05-30 08:32 UTC by Peter Robinson
Modified: 2018-03-23 22:15 UTC (History)
19 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-03-23 22:15:47 UTC


Attachments (Terms of Use)

Description Peter Robinson 2017-05-30 08:32:35 UTC
The new kvm_stat tool should use python3 not python2 so as to not pull a complete new python stack into the distro.

As it's a new dependency of the package I would expect it to use the default python of the distribution which in current Fedora releases is python3

In a minimal build it brings in considerable dependencies:

# dnf install kernel-tools
Last metadata expiration check: 1:19:30 ago on Mon May 29 12:43:28 2017 UTC.
Dependencies resolved.
==============================================================================
 Package                 Arch      Version                  Repository Size
==============================================================================
Installing:
 kernel-tools            armv7hl   4.12.0-0.rc2.git3.1.fc27 rawhide   140 k
Installing dependencies:
 kernel-tools-libs       armv7hl   4.12.0-0.rc2.git3.1.fc27 rawhide    67 k
 pciutils-libs           armv7hl   3.5.4-1.fc26             rawhide    45 k
 python2                 armv7hl   2.7.13-10.fc27           rawhide    98 k
 python2-appdirs         noarch    1.4.0-10.fc26            rawhide    21 k
 python2-libs            armv7hl   2.7.13-10.fc27           rawhide   5.7 M
 python2-packaging       noarch    16.8-5.fc27              rawhide    46 k
 python2-pip             noarch    9.0.1-10.fc27            rawhide   1.8 M
 python2-pyparsing       noarch    2.1.10-3.fc26            rawhide   139 k
 python2-setuptools      noarch    35.0.2-1.fc27            rawhide   445 k
 python2-six             noarch    1.10.0-8.fc26            rawhide    35 k
 
Transaction Summary
==============================================================================
Install  11 Packages

Total download size: 8.5 M
Installed size: 34 M

Comment 1 Justin M. Forbes 2017-07-24 20:39:34 UTC
This is not a kernel issue. kvm_stat was added to the kernel-tools package  (moved from kvm).  The dep is on /usr/bin/python which is correct, it doesn't care what python version you have installed.  If python3 is *the* system python, then it should also provide /usr/bin/python.

rpm -qlp --requires kernel-tools-4.12.3-1.fc26.x86_64.rpm  | grep py
/usr/bin/python


Assigning this to python3 so a suitable solution can be determined.

Comment 2 Charalampos Stratakis 2017-07-24 21:25:21 UTC
(In reply to Justin M. Forbes from comment #1)
> This is not a kernel issue. kvm_stat was added to the kernel-tools package 
> (moved from kvm).  The dep is on /usr/bin/python which is correct, it
> doesn't care what python version you have installed.  If python3 is *the*
> system python, then it should also provide /usr/bin/python.
> 
> rpm -qlp --requires kernel-tools-4.12.3-1.fc26.x86_64.rpm  | grep py
> /usr/bin/python
> 
> 
> Assigning this to python3 so a suitable solution can be determined.

That is wrong actually. Fedora follows the upstream recommendation in regards to /usr/bin/python (see PEP 394), which means that for the time being /usr/bin/python points to python2.

Python 3 is the default system-python we ship (on which also anaconda, dnf, abrt and other system tools depend on), but when it comes to shebangs, they should be explicit, so either /usr/bin/python2 or /usr/bin/python3.

The recommended way to address that is to replace the shebang with a 'sed' command during %prep.

Comment 3 Justin M. Forbes 2017-07-25 15:29:02 UTC
According to the same PEP "python should be used in the shebang line only for scripts that are source compatible with both Python 2 and 3"  An application which is source compatible with both would be correct in using /usr/bin/python.  We basically have a dep problem.

Comment 4 Petr Viktorin 2017-07-25 15:52:09 UTC
Yes, using /usr/bin/python in the shebang will work correctly -- but in Fedora, it'll use the Python 2 stack, and it drags that in as a dependency.

Changing the shebang to /usr/bin/python3 is the easiest way to avoid this. Sorry for that. (Build systems like setuptools can do that change automatically, but with a custom Makefile, you unfortunately need a Fedora-specific patch.)

Comment 5 Miro Hrončok 2017-07-25 15:56:48 UTC
(In reply to Justin M. Forbes from comment #3)
> According to the same PEP "python should be used in the shebang line only
> for scripts that are source compatible with both Python 2 and 3"  An
> application which is source compatible with both would be correct in using
> /usr/bin/python.

Yes. And as an upstream project, using /usr/bin/python to indicate "whatever python" is fine. However that is not the case in Fedora:

From https://fedoraproject.org/wiki/Packaging:Python

"If the executables provide the same functionality independent of whether they are run on top of Python 2 or Python 3, then only the Python 3 version of the executable should be packaged."

=> In this case, we want to package the "Python 3 version of the executable".

"packages in Fedora should not depend on where /usr/bin/python happens to point but instead should call the proper executable for the needed python major version directly, either /usr/bin/python2 or /usr/bin/python3 as appropriate"

=> The Python 3 version of the executable we need to package should have /usr/bin/python3 in shebang.

Note that those are all "shoulds" guidelines, so if there is a good enough reason for kvm_stat to run on Python 2, rules can be broken. However, using Python 2 just because using Python 3 would involve changing the shebang(s) is IMHO not a good enough reason.

Comment 6 Miro Hrončok 2017-07-25 16:00:59 UTC
There is a how to for changing the shebang in spec: http://python-rpm-porting.readthedocs.io/en/latest/applications.html#are-shebangs-dragging-you-down-to-python-2

Comment 7 Fedora Update System 2017-07-28 06:12:10 UTC
kernel-4.12.4-300.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-14ad2c5d17

Comment 8 Miro Hrončok 2017-07-28 07:28:21 UTC
I wonder why adding passive aggressive comments in the specfile is necessary :(

Comment 9 Fedora Update System 2017-07-29 00:54:32 UTC
kernel-4.12.4-300.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-14ad2c5d17

Comment 10 Josh Boyer 2017-07-29 20:18:28 UTC
(In reply to Miro Hrončok from comment #8)
> I wonder why adding passive aggressive comments in the specfile is necessary
> :(

I don't see any passive aggressive comments in the spec...

The only log I see is:

* Tue Jul 25 2017 Justin M. Forbes <jforbes@fedoraproject.org>
- Force python3 for kvm_stat because we can't dep (rhbz 1456722)

which is a technically accurate description of the fix and problem.

Comment 11 Miro Hrončok 2017-07-30 08:54:13 UTC
I was actually referring to:

+# Because the python 3 transition is fail
+Patch124: force-python3-in-kvm_stat.patch

I might have misjudged the tone of that, but comments like this make me feel that every effort we make to move things forward is just bothering others.

Comment 12 Jeremy Cline 2017-09-30 16:11:19 UTC
kvm_stat didn't actually support Python 3 (I assume that's why the patch forcing it to Python 3 was dropped) so I submitted a patch that makes it compatible: https://patchwork.kernel.org/patch/9954979/

Comment 13 Miro Hrončok 2017-10-01 18:31:21 UTC
(In reply to Jeremy Cline from comment #12)
> kvm_stat didn't actually support Python 3 (I assume that's why the patch
> forcing it to Python 3 was dropped) so I submitted a patch that makes it
> compatible: https://patchwork.kernel.org/patch/9954979/

I don't have an account there, so I'll just post a comment here:

-            print ' %9d' % s[k][1],
+            print(' %9d' % s[k][1])

This is not an equivalent change (notice the ,). The equivalent change would be:

+ from __future__ import print_function

...

+            print(' %9d' % s[k][1], end=' ')

Comment 14 Jeremy Cline 2017-10-03 13:29:31 UTC
(In reply to Miro Hrončok from comment #13)
> I don't have an account there, so I'll just post a comment here:
> 
> -            print ' %9d' % s[k][1],
> +            print(' %9d' % s[k][1])
> 
> This is not an equivalent change (notice the ,). The equivalent change would
> be:
> 
> + from __future__ import print_function
> 
> ...
> 
> +            print(' %9d' % s[k][1], end=' ')

Good catch, thanks for taking a look at the patch!

I also discovered there's a few cases where str and bytes are getting mixed so I'm going to need to rework this patch a bit.

Comment 15 Cole Robinson 2018-03-23 22:15:47 UTC
Pretty sure this is fixed, kernel 4.15 shipped with kvm_stat python3 support and all the packaging bits are in place AFAICT


Note You need to log in before you can comment on or make changes to this bug.