Bug 1629435 - python-q: Remove python2-q once python2 is removed
Summary: python-q: Remove python2-q once python2 is removed
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: python-q
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Miro Hrončok
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: PY2REMOVAL F31_PY2REMOVAL
TreeView+ depends on / blocked
 
Reported: 2018-09-16 08:24 UTC by Miro Hrončok
Modified: 2019-11-15 10:54 UTC (History)
9 users (show)

Fixed In Version: python-q-2.6-15.fc32
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-11-15 10:54:06 UTC


Attachments (Terms of Use)
root.log from running mock with --rpmbuild-opts=--without=python2 (60.50 KB, text/plain)
2018-09-18 16:58 UTC, Toshio Kuratomi
no flags Details

Description Miro Hrončok 2018-09-16 08:24:26 UTC
In line with the Mass Python 2 Package Removal [0], the following (sub)packages of python-q were marked for removal:

 * python2-q

According to our query, those (sub)packages only provide a Python 2 importable module. If this is not true, please tell us why, so we can fix our query.

Please remove them from your package.

As said in the change document, if there is no objection in a week, we will remove the package(s) as soon as we get to it. This change might not match your packaging style, so we'd prefer if you did the change. If you need more time, please let us know here.

We hope this doesn't come to you as a surprise. If you want to know our motivation for this, please read the change document [0].

[0] https://fedoraproject.org/wiki/Changes/Mass_Python_2_Package_Removal

Comment 1 Toshio Kuratomi 2018-09-16 14:14:33 UTC
I'd like to keep the python2-q subpackage until the Python 2 package is removed from Fedora.  I have no objection to removing it at that time (although, if a python2.7 compat package were to be maintained, I would like to create a python2.7-q subpackage for that.)  The reason is that q is an essential tool for us to debug problems in Python code.  As our software will have to run on Python2 and Python3 until RHEL7 goes EOL we would like to have it at our disposal on developer workstations as well as our CI servers.

python2-q has no dependencies outside of Python2 itself so it doesn't require that a stack of other packages be maintained to keep it going.

Let me know what I should do with this bug and also what I should expect when you are ready to remove the python2 package from Fedora so I can remove the subpackage for that release.  (Or if you have a conditional that will key off of the presence or absence of Python2 in Fedora that I should modify my spec file to account for automatically).

Comment 2 Miro Hrončok 2018-09-16 19:00:28 UTC
As a developer, who debugs Python code, do you really need to have this RPM packaged on the distro level?

Comment 3 Toshio Kuratomi 2018-09-16 19:44:49 UTC
Do we need it?  Of course not.  End users don't even *need* to have any pure python applications installed, they can be installed with a different package manager just as easily as with rpm.

The real question is does it make our lives easier?  Yes.  Everything from new hire instructions to ansible playbooks installing a dev environment would have to be changed.  And some people just like rpms better than pip installing.

Comment 4 Miro Hrončok 2018-09-16 22:11:55 UTC
According to the change description, this is a package for removal. I understand that removing some RPM packaged python2 modules brings additional challenges.
Once we start having exceptions based on "I'd like to keep this because the module is useful and some people just like rpms better than pip installing" we can as well stop the change right now.

However, that said, this package in particular can be seen as tool rather than a library and (as noted) it has no dependencies outside of Python 2 itself (and hence doesn't block removal of other packages).

I'm torn on this one. What do other change owners think?

Comment 5 Igor Gnatenko 2018-09-17 06:35:55 UTC
I would remove it. It will have to happen sooner or later anyway so I would drop it now and adjust all instructions to use tox or whatsoever. Instead of waiting until it gets dropped and only then do something.

Comment 6 Zbigniew Jędrzejewski-Szmek 2018-09-17 07:28:38 UTC
Or if you want to make it easy for users, add a patch now that spits out a warning every time the tool is run under python2, and remove it after waiting a while. Either way, set a date, sometime not far in the future, where the python2 subpackage will be removed.

Comment 7 Petr Viktorin 2018-09-17 10:04:25 UTC
> I'd like to keep the python2-q subpackage until the Python 2 package is removed from Fedora.

I think this is valid for a debugging tool.

We might actually want to keep a set of packages we want to remove last.
This would mean that as soon as this (and packages like it) is the only thing blocking removal of another py2 package (python2 or python2-setuptools or any other), it gets removed *immediately*, without any more discussion.

Comment 8 Miro Hrončok 2018-09-17 10:41:43 UTC
(In reply to Petr Viktorin from comment #7)
> We might actually want to keep a set of packages we want to remove last.
> This would mean that as soon as this (and packages like it) is the only
> thing blocking removal of another py2 package (python2 or python2-setuptools
> or any other), it gets removed *immediately*, without any more discussion.

I think this can be accomplished with:

(In reply to Toshio Kuratomi from comment #1)
> (Or if you have a conditional that will
> key off of the presence or absence of Python2 in Fedora that I should modify
> my spec file to account for automatically).

Comment 9 Toshio Kuratomi 2018-09-18 14:37:14 UTC
> (Or if you have a conditional that will
> key off of the presence or absence of Python2 in Fedora that I should modify
> my spec file to account for automatically).

I've been thinking about this and I think the following is true:

* Since this will determine BuildRequires, it needs to be a macro that's present in the buildroot.
* The python2 package is no longer in the buildroot
* The python-srpm-macros subpackage is present in the buildroot
* The %{__python} macro is provided by rpm and points to /usr/bin/python

So if that last macro is going to continue to point to /usr/bin/python and /usr/bin/python continues to mean python2, I can use that.... However, I've seen discussion about moving /usr/bin/python to its own package which makes me think people are considering making it point to /usr/bin/python3 instead?  If so, that won't work.

Instead I'd need to have a simple macro added to the buildroot (via the python-srpm-macros package seems to make sense) that will tell the package whether python2 is present or absent on that version of Fedora.  To make the spec file work on other versions of Fedora and EPEL, I could do something like this in my spec file:

%{!?python2_present: %global python2_present 1}
%if 0%{?python2_present}
BuildRequires: python2-devel
%package -n python2-q
[....]
%endif

When you know which Fedora release will drop the Python2 package, simply add this line in the macros provided by the python-srpm-macros package for that release:

%global python2_present 0

And then the python2-q package will automatically stop building there.

Does that sound reasonable?  Due to the backwards compatible nature of the snippet above, I can add that immediately but I need to know that you'll use python2_present as the name of the macro to make it automatically drop the subpackage in the future.

Comment 10 Petr Viktorin 2018-09-18 15:10:15 UTC
Please, don't make this more complicated. A python2_present macro might be nice nice, but to make it generally usable we'd soon need %{python2-six_present} and so on.

My proposal for how to handle this:

This package is now on our short list, to be removed when only packages on this list are blocking some other python2 removal. (In python-q's case, that would be python2 or python2-setuptools, unless you add other dependencies.)
When the time comes, we'll change the package without *any* further warning or discussion, and close this bug.
Please make the change easy for us, preferably by using a "%without python2" bcond or similar macro.

Toshio and Miro, can we agree on that?

Comment 11 Toshio Kuratomi 2018-09-18 16:56:11 UTC
* Done: https://koji.fedoraproject.org/koji/taskinfo?taskID=29751336

* I've also removed the setuptools BuildRequires so this package's presence won't block removal of python2-setuptools.

* Do note that bcond probably isn't sufficient for making this work automatically as BuildRequires are calculated without the benefit of --with/--without.  Perhaps you're thinking that's a missing feature in mock/dnf, though?  (I'll attach a root.log from running "mock rebuild python-q-2.6-8.fc30.src.rpm  --resultdir ~/tmp2 --rpmbuild-opts=--without=python2" to show what's happening in case you want to talk to the mock/dnf maintainers about it.)

* Also note... if I was going to go the %{python2_present} route, I wouldn't create %{python2-six_present} and so forth.  I'd make all the packages that were going to remain until python2 disappeared use the %{python2_present}  macro/.  Then they'd all disappear as one group, at the same time.

Comment 12 Toshio Kuratomi 2018-09-18 16:58:40 UTC
Created attachment 1484439 [details]
root.log from running mock with --rpmbuild-opts=--without=python2

In this root.log, notice that BuildRequires are calculated without consideration of --without=python so mock still installs python2-devel.

Comment 13 Toshio Kuratomi 2018-09-18 17:08:15 UTC
Ah, looking at rpm source, you can probably work around the mock limitation by setting the underlying _without_python2 macro in the python-srpm-macros package rather than passing in rpmbuild_opts so at least you can fallback to something equivalent to what I was proposing.

Comment 14 Miro Hrončok 2018-09-18 17:10:04 UTC
Note that the egg info dir->file update might bite. 

> Due to a known limitation with RPM, it is not possible to replace a directory with any kind of file or symlink, nor is it possible to replace a symlink to a directory with a directory, without RPM producing file conflict errors while trying to install the package.

  https://fedoraproject.org/wiki/Packaging:Directory_Replacement


Also, this does not install python2-devel:

    $ fedpkg mockbuild -N --without python2

I'd like to keep this open until the package actually is removed. We might want to mark it blocked or something, but definitively not CLOSED RAWHIDE.

Comment 15 Toshio Kuratomi 2018-09-18 19:08:38 UTC
Doh!  I forgot that rpm bug affected any file replacing a directory, not just symlinks replacing a directory.  Fixed i nthe latest build.

I see the same results you do for fedpkg mockbuild.  So as long as koji is using fedpkg under the hood I guess you'll be okay.

Comment 16 Miro Hrončok 2018-09-18 19:30:44 UTC
It seems unlikely to me that python2-setuptools would go away any sooner than python2 and your "get rid of setuptools" patch is getting quite complicated, time to reconsider? Nowhere we said: Let's get rid of python2-setuptools from python2-q.

I propose that we discuss first and once we have some agreement, we do stuff. At least it seems better than committing hotfixes on top of hotfixes that were never needed, in the middle of a conversation.

As far as i see it, we are getting to a conclusion...

(In reply to Petr Viktorin from comment #10)
> My proposal for how to handle this:
> 
> This package is now on our short list, to be removed when only packages on
> this list are blocking some other python2 removal. (In python-q's case, that
> would be python2 or python2-setuptools, unless you add other dependencies.)
> When the time comes, we'll change the package without *any* further warning
> or discussion, and close this bug.
> Please make the change easy for us, preferably by using a "%without python2"
> bcond or similar macro.
> 
> Toshio and Miro, can we agree on that?

I agree with that, yet we need to figure out how to track this.

Comment 17 Miro Hrončok 2018-09-19 14:00:07 UTC
As a debugging tool, we now listed this package as an exception. It's going to be removed with python2 (once that happens).

When the time comes, we'll change the package without *any* further warning or discussion, and close this bug.

I've assigned the bug to me and renamed it. You don't need to worry about it any longer.

And you can keep setuptools.

Comment 18 Miro Hrončok 2019-08-26 14:35:38 UTC
For the record we plan to remove python2-q mid November.

Comment 19 Miro Hrončok 2019-11-15 10:47:41 UTC
Doing it now.


Note You need to log in before you can comment on or make changes to this bug.