Bug 1627438 - rhn-client-tools: Remove packages from Fedora 30+: python2-rhn-*
Summary: rhn-client-tools: Remove packages from Fedora 30+: python2-rhn-*
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: rhn-client-tools
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Tomáš Kašpárek
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: PY2REMOVAL
TreeView+ depends on / blocked
 
Reported: 2018-09-10 15:38 UTC by Miro Hrončok
Modified: 2019-02-19 16:27 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-01-23 10:04:02 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)

Description Miro Hrončok 2018-09-10 15:38:48 UTC
In line with the Mass Python 2 Package Removal [0], the following (sub)packages of rhn-client-tools were marked for removal:

 * python2-rhn-setup-gnome

According to our query, those packages only provide a Python 2 importable module. If this is not true, please tell us why, so we can fix our query.

Please remove them from your package.

As said in the change document, if there is no objection in a week, we will remove the package(s) as soon as we get to it. This change might not match your packaging style, so we'd prefer if you did the change. If you need more time, please let us know here.

We hope this doesn't come to you as a surprise. If you want to know our motivation for this, please read the change document [0].

[0] https://fedoraproject.org/wiki/Changes/Mass_Python_2_Package_Removal

Comment 1 Miro Hrončok 2018-11-29 12:24:49 UTC
I intent to remove all of those next week:

python2-rhn-check
python2-rhn-client-tools
python2-rhn-setup
python2-rhn-setup-gnome

This was without a reply for a couple months now, so I guess there are no objections for Python 2 packages removal, but the list was incomplete, so I won't do it right now.

Comment 2 Michael Mráka 2018-12-19 13:18:55 UTC
Fixed in spacewalk git by

commit 665930d401ab5538ba8d4fa8486fb84b1fc9c152
    1627438 - don't build python2 packages on new Fedoras

Comment 3 Miro Hrončok 2018-12-28 16:52:32 UTC
Could we have this in Fedora as well please and not just spacewalk git? Thanks

Comment 4 Michael Mráka 2019-01-10 10:02:52 UTC
Unfortunately I've discovered we need python2 subpackages for server side so can't easily remove them from the package.
So I actually have to revert the above change.

Comment 5 Miro Hrončok 2019-01-10 10:42:53 UTC
Server side of what? What server are we talking about?

Nothing in Fedora requires:

 * python2-rhn-check
 * python2-rhn-client-tools
 * python2-rhn-setup
 * python2-rhn-setup-gnome

Hence IMHO the packages shall be removed from Fedora. I can of course be wrong, but please explain this to me. If we reach consensus, we can close this as WONTFIX, but please don't just do that without providing the information.

Thanks

Comment 6 Michael Mráka 2019-01-10 12:02:28 UTC
python2-rhn-* (as well as python3-rhn-*) packages are client side packages for Spacewalk (and Satellite) server. They are built from rhn-client-tools source rpm which contains python2 and python3 subpackages as well as shared code between server and client. And be able to install server on Fedora we need python2 version of the code.

Comment 7 Miro Hrončok 2019-01-10 12:11:58 UTC
And the server is in what particular package?

Also, what is the reason to install the server using Python 2 on Fedora?

Comment 8 Michael Mráka 2019-01-10 12:19:19 UTC
(In reply to Miro Hrončok from comment #7)
> And the server is in what particular package?

Server isn't packaged in Fedora, it has a separate COPR repo.

> Also, what is the reason to install the server using Python 2 on Fedora?

There's no python3 server version.

Comment 9 Miro Hrončok 2019-01-10 12:32:17 UTC
Would it be viable for you to move the python2 packages to that copr repo together with the server code?

Comment 10 Charalampos Stratakis 2019-01-10 13:13:34 UTC
(In reply to Michael Mráka from comment #8)
> (In reply to Miro Hrončok from comment #7)
> > And the server is in what particular package?
> 
> Server isn't packaged in Fedora, it has a separate COPR repo.
> 
> > Also, what is the reason to install the server using Python 2 on Fedora?
> 
> There's no python3 server version.

Having a Fedora package SPEC depend on something packaged in COPR doesn't seem very conforming to the guidelines.

Comment 11 Miro Hrončok 2019-01-10 13:19:57 UTC
I'm lost. What Fedora package would depend on what COPR package?

All Python 2 packages would be in COPR.

Comment 12 Charalampos Stratakis 2019-01-10 13:22:24 UTC
(In reply to Miro Hrončok from comment #11)
> I'm lost. What Fedora package would depend on what COPR package?
> 
> All Python 2 packages would be in COPR.

(In reply to Michael Mráka from comment #8)
> (In reply to Miro Hrončok from comment #7)
> > And the server is in what particular package?
> 
> Server isn't packaged in Fedora, it has a separate COPR repo.
>

Comment 13 Michael Mráka 2019-01-10 13:37:07 UTC
Well, let's try again:
Client + shared packages are in Fedora, server packages are not in Fedora. To be able to act both as client and server it needs to have all packages in the same version. They are built from the same source rpm.
So I see two possibilities here:
either a) both python2 and python3 packages are in Fedora,
or b) none of them.

Comment 14 Zbigniew Jędrzejewski-Szmek 2019-01-10 13:41:06 UTC
I assume that the python3 client can interact with the python2 server.

c) python2 server is built in copr, python3 client is build in Fedora. They may even be both built from the same source package.

Comment 15 Miroslav Suchý 2019-01-11 09:30:38 UTC
> I assume that the python3 client can interact with the python2 server.

Yes, but this is not the case. I will try to explain.

There is python-rhn-client-tools package in Fedora. It provides some client commands like rhn-register etc. - these tools are already migrated to python3 and yes they can talk to python2 server over network. But python-rhn-client-tools also contains common python libraries called 'up2date_client'. This module is used in server - package called 'spacewalk-backend' and this package lives in Copr [*]. For example, a tool called `satellite-sync` from `spacewalk-backend` use client python module. And whole `spacewalk-backend` is python2 only. And it is very big beast and migrating to python3 is multi-month effort (maybe more than a year). 

There is definitely an option to stop building python2 in Fedora and build the package in Copr, but it is more likely that the versions in Copr and in Fedora will be off-sync and I guess that Spacewalk devels will not like it.

[*] spacewalk-backend is only one example. Whole Spacewalk project is a layered application and consist of more than one hundred packages. Some are in Fedora, but most of them are not in Fedora as it is not technically possible.

Comment 16 Miro Hrončok 2019-01-11 10:00:04 UTC
> There is definitely an option to stop building python2 in Fedora.

Please do. We need to get rid of as many Legacy Python packages as possible (ultimately all, but we are not that optimistic).

> but it is more likely that the versions in Copr and in Fedora will be off-sync

Is this about Python 3 client (Fedora) being off-sync with the Python 2 server (Copr)?

You can use Fedora dist-git to trigger a build in Copr. The off-sync will than only last for a couple hours. Does that work for you? I can help you with the setup in case you would like to.

(Also, I must say that *developers* being blocked by a certain library of a stack *they develop* not hitting *official Fedora repos* seems like a bad design. I suggest you move you entire *development stack* to Copr and use Fedora official repos as a platform to ship the client bits to the actual users. IMHO Official Fedora repos are not supposed to be used as a "development platform as a service". However, you probably know better than me about your dev environment.)

Comment 17 Miro Hrončok 2019-01-11 10:06:07 UTC
One more observation.



Current state as I see it:

Fedora           Copr
==============   ===============
python2-common   python2-server
python2-client
python3-common
python3-client


I.e. the python2-server can *currently* go off sync with anything.


The state we would like to get into:


Fedora           Copr
==============   ===============
python3-common   python2-server
python3-client   python2-common
                 python2-client


I.e. the python2-server could then go off sync only with python3 client (already happened before).

Only new combo that can go off sync is: python3 client/common can go off sync with python2 client/common. I fail to see how that is a problem as a Python 2 client won't communicate with Python 3 client.

Comment 18 Miro Hrončok 2019-01-17 10:42:46 UTC
Can we do this?

Comment 19 Michael Mráka 2019-01-17 12:21:23 UTC
Unfortunately no, it's more complicated than that (and we actually tried).

As I already wrote in comment #13:
> I see two possibilities here:
> either a) both python2 and python3 packages are in Fedora,
> or b) none of them.

Comment 20 Petr Viktorin (pviktori) 2019-01-17 12:34:55 UTC
Is there any ETA on this situation changing?
Is the problem something we could help with?

Comment 21 Michael Mráka 2019-01-21 08:42:09 UTC
(In reply to Petr Viktorin from comment #20)
> Is there any ETA on this situation changing?

No.

> Is the problem something we could help with?

No.

Comment 22 Miro Hrončok 2019-01-21 08:48:44 UTC
I'm seriously confused here.

We nee to understand the problem in order to propose a solution that works for you and that works for us.

Why is that you fight so hard not to do so?

Comment 23 Michael Mráka 2019-01-21 08:54:52 UTC
I do not.
I already explained the situation and proposed two solutions.

Comment 24 Miro Hrončok 2019-01-21 08:58:58 UTC
You've explained that it needs to be synced. When I asked what needs to be synced with what and proposed a solution, you've said it's more complicated than that. That not explaining the situation at all.

Given that you proposed solution is: a) both python2 and python3 packages are in Fedora, or b) none of them are; and you are not willing to help us find a third solution, I won't bother you any longer. Please go with option b).

Note that we are still willing to help you find a working solution that is less drastic.

Comment 26 Charalampos Stratakis 2019-01-21 11:30:03 UTC
(In reply to Michael Mráka from comment #23)
> I do not.
> I already explained the situation and proposed two solutions.

Alright, let's reiterate a bit here.

This bugzilla is part of an accepted system-wide change for Fedora. That means that package maintainers will have to comply with that change.

As Python-SIG is mainly driving the change, we have decided to be graceful and help the packagers as much as possible for potential migration paths, issues that could arise with their package removals, pull requests and so on. However we don't see here any information on the technical side of things, just a refusal to conform with the system-wide change. Please elaborate further.

If the refusal continues without actually sharing the issue here or proposing a solution, we'll move forward with it ourselves. If the removal is reverted for no reason, the matter will go to Fesco.

Comment 27 Charalampos Stratakis 2019-01-21 11:47:04 UTC
(In reply to Charalampos Stratakis from comment #26)
> (In reply to Michael Mráka from comment #23)
> > I do not.
> > I already explained the situation and proposed two solutions.
> 
> Alright, let's reiterate a bit here.
> 
> This bugzilla is part of an accepted system-wide change for Fedora. That
> means that package maintainers will have to comply with that change.
> 
> As Python-SIG is mainly driving the change, we have decided to be graceful
> and help the packagers as much as possible for potential migration paths,
> issues that could arise with their package removals, pull requests and so
> on. However we don't see here any information on the technical side of
> things, just a refusal to conform with the system-wide change. Please
> elaborate further.
> 
> If the refusal continues without actually sharing the issue here or
> proposing a solution, we'll move forward with it ourselves. If the removal
> is reverted for no reason, the matter will go to Fesco.

And to add to that. If it's possible to entirely remove the package from Fedora, that would work even better as it could help with removing potential dependents as well.

However the reason we are asking you to provide us with the technical aspects of the problem, is in order to try and help you whatever way possible. If the final solution is to remove the packages we'll be ok with that. If they have to stay in Fedora we'll need to remove the python2 subpackages. Sorry if that sounds counter-productive, discussion wise, but we've been dealing with thousands of packages here and we can't get stuck on one for such a long time.

Comment 28 Miro Hrončok 2019-01-21 11:53:56 UTC
Also note that the Python 3 subpackages of rhn-client-tools are (unlike the Python 2 ones) not leaves in Fedora, so their removal may be disruptive.

Comment 29 Michael Mráka 2019-01-23 10:04:02 UTC
All rhn-* packages and dependencies has been removed from Fedora.

Comment 30 Miro Hrončok 2019-01-23 13:06:55 UTC
Thank You I guess.

Note that we are still happy to help you figure out how to keep the useful bits in Fedora.

Comment 31 Neal Gompa 2019-02-19 13:54:21 UTC
(In reply to Miro Hrončok from comment #30)
> Thank You I guess.
> 
> Note that we are still happy to help you figure out how to keep the useful
> bits in Fedora.

I just found out the stack was removed from Fedora, which is a huge problem for me, since I need it for our systems.

That said, I know what the problem is: they currently build from a monorepo, and they don't have a way in COPR to set a build condition to build the Python 2 stuff for the server components to work while simultaneously not having it built in Fedora at all.

The real solution would be to port the Spacewalk code to Python 3, which is definitely doable, since a lot of the work has been done by the SUSE folks in Uyuni[1]. I'm working on porting the YUM-specific bits in Spacewalk to DNF[2] to resolve the remaining issue server side after general Python 3 porting is done.

Alternatively, COPR would grow support for setting project-wide macros to influence rpmbuild behavior. That would require some work from clime.

[1]: https://github.com/uyuni-project/uyuni/pulls?utf8=%E2%9C%93&q=is%3Apr+python3
[2]: https://www.redhat.com/archives/spacewalk-devel/2019-February/msg00000.html

Comment 32 Michael Mráka 2019-02-19 14:25:58 UTC
You can simply install client packages from copr:

dnf copr enable @spacewalkproject/spacewalk-2.9-client
dnf install ...

Comment 33 Neal Gompa 2019-02-19 14:42:26 UTC
(In reply to Michael Mráka from comment #32)
> You can simply install client packages from copr:
> 
> dnf copr enable @spacewalkproject/spacewalk-2.9-client
> dnf install ...

I'm aware, but that's only a stopgap. As more things are removed on the Python 2 side, it'll break more. It's better to resolve the problem properly.

I'd personally prefer if all of Spacewalk was packaged in Fedora properly, as well, but we need it ported to Python 3 first...

Comment 34 Miro Hrončok 2019-02-19 16:27:21 UTC
(In reply to Neal Gompa from comment #31)
> That said, I know what the problem is: they currently build from a monorepo,
> and they don't have a way in COPR to set a build condition to build the
> Python 2 stuff for the server components to work while simultaneously not
> having it built in Fedora at all.
> 
> Alternatively, COPR would grow support for setting project-wide macros to
> influence rpmbuild behavior. That would require some work from clime.

I know how to do this and as said we are happy to help.


Note You need to log in before you can comment on or make changes to this bug.