Bug 1636020

What about python2 packages that are still needed in Fedora 30, but then become leaves in Fedora 31 and should remove python2 support then?
python2-progress can be removed now, but other packages might need more time.

What about EL7/EL6; should we add the macros there as well?

This should go through FPC first.

In my opinion, this is really useless and it really depends on packagers and not what fedora dictates.

(In reply to Petr Viktorin from comment #1)
> What about python2 packages that are still needed in Fedora 30, but then
> become leaves in Fedora 31 and should remove python2 support then?

Such packages would use those macros later, not yet there.  Of course
anyone can {re,un}define the defaults.

> What about EL7/EL6; should we add the macros there as well?

Yes, into epel-rpm-macros probably - as long as we are unable to patch
redhat-rpm-macros.

s/not yet there/not now/

(In reply to Igor Gnatenko from comment #2)
> This should go through FPC first.

Feel free, I don't see a problem in that.  It is one/two liner, right?

> In my opinion, this is really useless and it really depends on packagers and
> not what fedora dictates.

This macro doesn't dictate anything to packagers, neither I propose you
to make the macro mandatory for all python packages.  Those macros are
meant to give a convenient info about "what this system prefers".

But what does "what this system prefers" mean?

Can you just use %fedora? Python 3 is the default in all supported releases.

(In reply to Petr Viktorin from comment #6)
> But what does "what this system prefers" mean?

I thought that we (by default) expect/prefer this "build pattern":

Fedora 27, 28, 29 -->  build python package against Python 3 and Python 2
Fedora 30, 31     -->  build against Python 3
EPEL 6, 7         -->  build against Python 2

.. except for exceptions.  I don't really mind them here, as this is not
meant to cover everything.

> Can you just use %fedora? Python 3 is the default in all supported releases.

This proposal is absolutely orthogonal to what "default" python means.

Example to demonstrate "how this could be useful":
https://src.fedoraproject.org/rpms/python-progress/pull-request/4#request_diff

That package is otherwise buildable on el6+, fedora 27+ (et least).

(In reply to Pavel Raiskup from comment #5)
> (In reply to Igor Gnatenko from comment #2)
> > This should go through FPC first.
> 
> Feel free, I don't see a problem in that.  It is one/two liner, right?
> 
> > In my opinion, this is really useless and it really depends on packagers and
> > not what fedora dictates.
> 
> This macro doesn't dictate anything to packagers, neither I propose you
> to make the macro mandatory for all python packages.  Those macros are
> meant to give a convenient info about "what this system prefers".

It's not about what system prefers. It's what FPC says packagers to do (obviously by getting agreements with community).

So if you are not going to drive this effort through FPC, I believe that we can close this bug.

The rule for Fedora is roughly:
- build all Python packages against Python 3 only, if you can (e.g. if nothing depends on the py2 bits)
- never remove (sub)packages from a released Fedora version
- if nothing depends on the py2 bits and it's just an importable module, remove py2 *now* (that's a case we have automation for and file bugs for)

There's not much special about Fedora 30, we just started the semi-automated mass removal of leaf py2 packages.

(In reply to Petr Viktorin from comment #10)
> There's not much special about Fedora 30, we just started the semi-automated
> mass removal of leaf py2 packages.

If F30 is not "special", what Fedora version will be?  Because at that
point I want to have my python2 subpackages dropped (not sooner) and at
that point it would be convenient to have those macros in hand.

(In reply to Pavel Raiskup from comment #11)
> If F30 is not "special", what Fedora version will be? 

For python2-progress, it's F30 because nothing depends on python2-progress there.

I can't answer for all other packages. There are thousands of them, all different, and I don't see into the future.

I'm probably starting to understand where's the problem of this
discussion and misunderstanding.

We are **not OK** to claim (by trivial macro) on F30 that python2 is
(even if only preferably) "dead".  But at the same time we claim that
packages that hurt noone have to be removed (e.g. python2-progress).
_This_ is just plain wrong.

Unless we are brave enough to really deprecate python 2 runtime, tell
me a single reason to drop trivial packages like python2-progress.
It really depends on nothing, so we don't block anyone from removing
the python2 support.

Rephrasing my request to a more generic feature request:
Please, give me a system-wide "button" that you (Python team) will
control.  I (packager) will prepare my packages for that knob in advance,
and you can push the knob as soon as you really consider that Python 2
is dead.  The %py2_expected_support macro was meant to have this purpose.

(In reply to Pavel Raiskup from comment #13)
> I'm probably starting to understand where's the problem of this
> discussion and misunderstanding.
> 
> We are **not OK** to claim (by trivial macro) on F30 that python2 is
> (even if only preferably) "dead".  But at the same time we claim that
> packages that hurt noone have to be removed (e.g. python2-progress).
> _This_ is just plain wrong.

One package does not hurt, but there's thousands of them. We have no better way to filter out ones whose removal doesn't hurt, except go and remove them.

> Unless we are brave enough to really deprecate python 2 runtime, tell
> me a single reason to drop trivial packages like python2-progress.

python2 is deprecated:
https://src.fedoraproject.org/rpms/python2/blob/master/f/python2.spec#_108
But that's about adding new packages, not removing old ones.

> It really depends on nothing, so we don't block anyone from removing
> the python2 support.
> 
> Rephrasing my request to a more generic feature request:
> Please, give me a system-wide "button" that you (Python team) will
> control.  I (packager) will prepare my packages for that knob in advance,
> and you can push the knob as soon as you really consider that Python 2
> is dead.  The %py2_expected_support macro was meant to have this purpose.

So you want one for python2. Should we also make knobs for python2-setuptools, python2-six, python2-numpy, and others that many things depend on? Where's the boundary?

(In reply to Petr Viktorin from comment #14)
> One package does not hurt, but there's thousands of them. We have no better
> way to filter out ones whose removal doesn't hurt, except go and remove
> them.

I can not agree here.  If there was some knob, we can check whether package is
ready for it.  Or what exactly is the problem?

> python2 is deprecated:
> https://src.fedoraproject.org/rpms/python2/blob/master/f/python2.spec#_108
> But that's about adding new packages, not removing old ones.

That is very cool, but you forgot to let me know about this fact by
some RPM macro that I could respect in my packages.

> So you want one for python2. Should we also make knobs for
> python2-setuptools, python2-six, python2-numpy, and others that many things
> depend on? Where's the boundary?

No, no specific knob - but one for all python packages.  Why the
python-setuptools can not be prepared for the knob in advance?  If you
don't want to waste your time _now_ on those, the knob can be ignored
(it doesn't make any difference to the current approach).

(In reply to Pavel Raiskup from comment #15)
> That is very cool, but you forgot to let me know about this fact by
> some RPM macro that I could respect in my packages.

This is always true. There is no macro. No new python2 packages go into Fedora.

If you really need a way to query it, here it is:

     %if %{?fedora}

Note that we are not trying to fight you. We are trying to explain that while this macro might work for one package it certainly doesn't work for other packages. There are no usuals and no exceptions. It's just difeerent for every package.


If we fail to explain that, we are sorry.

Certainly if we could add a "knob" to all packages and than just turn the off switch at some point, it would be lovely. But it just won't happen, there are packages that will most likely require python2 for many years and removing them from the distro would be too painful (nodejs for example).

Packages need to remove their python2 part if those parts are not needed by other things. This is not something a macro can do.

Here's a specific example: If you define the macro in a way that works for python-progress, it would break PyGreSQL, it would break python-cassandra-driver.

It's not that python-progress is "normal" and PyGreSQL/python-cassandra-driver is an exception. They just live in a different area of the dependency graph.

> Note that we are not trying to fight you. We are trying to explain that while this macro might work for one package it certainly doesn't work for other packages.

What packages, and why?

(In reply to Pavel Raiskup from comment #19)
> > Note that we are not trying to fight you. We are trying to explain that while this macro might work for one package it certainly doesn't work for other packages.
> 
> What packages, and why?

Packages that are needed for other packages (reason is included).

Frankly I still don't get where the problem is.

You seem to ignore the fact that (a) you are not forced
to use the macro and (b) if you opt-in -- you still can override the
system default very easily (even this would be much nicer then the
%if-%else ugly thing).

If you claim that overriding the defaults I propose is difficult
(it is not) here is something which would make it trivial
(~/.rpmmacros for for F30):

   %py3_build_expected 1
   %fedora_py2_kill_in() %{lua:
   if tonumber(rpm.expand('%fedora')) < tonumber(rpm.expand('%1')) then
     rpm.define('py2_build_expected 1')
   else
     rpm.undefine('py2_build_expected')
   end
   }

Many packages can opt-in to use %pyX_build_expected pair, and those who
have to postpone the kill can opt-in too, but just use
"%{?fedora:%fedora_py2_kill_in 31}".  I fail to see how this could
cause any problems when we don't actually force anyone to use that.
It is just convenient thing for packagers who care to avoid this:

%if 0%{?fedora}
  %bcond_without python3
  %if 0%{?fedora} > 29
    %bcond_with python2
  %else
    %bcond_without python2
  %endif
%else
  %if 0%{?rhel} > 7
    %bcond_with    python2
    %bcond_without python3
  %else
    %bcond_without python2
    %bcond_with    python3
  %endif
%endif

When you give it some numbers, it starts to make more sense...

So what about: a macro, that sets with/without python2/3 according not to hardwired magic, but arguments?

Something like:

    %setup_with_pythons -f 30 -e 8

The interface could of course have more or less options...

The point here is: Not magical (you still need to read the docs for options to see what they mean, but at least there is no guessing and if a packager sees the numbers and the result they can figure out what's going on). Universal.

The scope however needs to be determined... ?

Do we only support MAX Fedora version (for python2+3 -> python3) and MIN EL version (for python2 -> python3), or do we want to support wider range of options (max/min for fedora/el for 2/3 = 8 options).

(I still won't create this and document this and support this and fix this when broken and add this to the guidelines, but at least I think this is something I could understand. It could probably live in python-srpm-macros, not to bother redhat-rpm-config.)

(In reply to Miro Hrončok from comment #22)
> So what about: a macro, that sets with/without python2/3 according not to
> hardwired magic, but arguments?

I like this idea; namely because it keeps the rpmbuild
--with{,out}=python{2,3} options for those who want to experiment.

That's starting to look good, but the -f and -e options still look quite magical to me.

What do you think about this generalization: let's add a macro, let's call it %bcond, where:

    %bcond NAME DEFAULT

conceptually expands to:

    %if DEFAULT
        %bcond_without NAME
    %else
        %bcond_with NAME
    %endif

which you would then call as:

    %bcond python2 (0%{?rhel} <= 7 || 0%{?fedora} < 30)
    %bcond python3 (0%{?rhel} > 7  || 0%{?fedora})

?

(The word "conceptually" hides a lot details, of course.)

Seems like this could be called:

    %bcond_if 

Or

    %bcond_when

I agree that we can have some kind of %bcond_if or something like that... But please open ticket on github.com/rpm-software-management/rpm about that. And then we can backport it.

Huh?  Can you please forward the discussion there, then?  Thank you.