Bug 1426737 - custodia: Provide a Python 3 subpackage
Summary: custodia: Provide a Python 3 subpackage
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: custodia   
(Show other bugs)
Version: 26
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Christian Heimes
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Keywords:
Depends On:
Blocks: PYTHON3
TreeView+ depends on / blocked
 
Reported: 2017-02-24 17:40 UTC by Iryna Shcherbina
Modified: 2017-06-09 19:08 UTC (History)
6 users (show)

Fixed In Version: custodia-0.3.1-2.fc26
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-06-09 19:08:59 UTC
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

Description Iryna Shcherbina 2017-02-24 17:40:12 UTC
The package `custodia` does not yet have a Python 3 equivalent subpackage. Please provide it so this package is fully Python 3–ready.


According to the Python packaging guidelines [0], software must be
packaged for Python 3 if upstream supports it.
The guidelines give detailed information on how to do this, and even
provide an example spec file [1].

The current best practice is to provide subpackages for the two Python
versions (called "Common SRPM" in the guidelines). Alternatively, if
nothing depends on your Python2 package, you can just switch to Python 3
entirely.

It's OK to do this in Rawhide only, however, it would be greatly
appreciated if you could push it to Fedora 25 as well.


If you need more instructions, a guide for porting Python-based RPMs is
available at [2].
If anything is unclear, or if you need any kind of assistance with the
porting, you can ask on IRC (#fedora-python on Freenode), or reply here.
We'll be happy to help!


[0] https://fedoraproject.org/wiki/Packaging:Python
[1] https://fedoraproject.org/wiki/Packaging:Python#Example_common_spec_file
[2] http://python-rpm-porting.readthedocs.io/

Comment 1 Fedora End Of Life 2017-02-28 12:26:16 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 26 development cycle.
Changing version to '26'.

Comment 2 Christian Heimes 2017-03-02 08:12:36 UTC
I released Custodia 0.3 today. It is the first version that fully supports and works on Python 3. For Fedora 26+ I can change the custodia base package to use Python 3 instead of Python 2. Is it necessary to also provide custodia-2 and custodia-3?

Proposal:

package `custodia`:
  * depends on python3-custodia instead of python2-custodia
  * /usr/bin/custodia-cli uses python3
  * /usr/sbin/custodia uses python3


In case 2/3 aliases are required:

package `python3-custodia`:
  * add /usr/sbin/custodia-3
  * add /usr/bin/custodia-cli-3

package `python2-custodia`:
  * add /usr/sbin/custodia-2
  * add /usr/bin/custodia-cli-2

Comment 3 Iryna Shcherbina 2017-03-02 09:18:04 UTC
Yes, you can just change the custodia main package to use Python 3.

Providing custodia-2 and custodia-3 subpackages is not necessary, if nothing depends on the Python 2 version.

Comment 4 Petr Viktorin 2017-03-02 09:28:00 UTC
The variants in /usr/bin are only necessary for code that interacts with Python code in some way: for example a "PEP8" linter or something that users write Python plugins for.

Comment 5 Christian Heimes 2017-03-02 14:13:44 UTC
I have created a build, https://koji.fedoraproject.org/koji/taskinfo?taskID=18137897 . I'll take care of F26 and Bodhi as soon as my time allows it.

Comment 6 Alexander Bokovoy 2017-04-18 09:20:04 UTC
Not sure *why* we should have Custodia in Python3 by default when depending FreeIPA code is all using Python 2 version. More so in F26.

Comment 7 Christian Heimes 2017-04-18 09:31:35 UTC
Why should Custodia use Python 2 when it works perfectly fine with Fedora's default Python? After all it's a standalone service and does not depend on FreeIPA.

Only the ipa-custodia service of FreeIPA is limited to Python 2 because FreeIPA 4.4 and 4.5 do not support Python 3 yet.

Comment 8 Jan Pazdziora 2017-04-18 09:59:48 UTC
(In reply to Christian Heimes from comment #7)
> Why should Custodia use Python 2 when it works perfectly fine with Fedora's
> default Python? After all it's a standalone service and does not depend on
> FreeIPA.
> 
> Only the ipa-custodia service of FreeIPA is limited to Python 2 because
> FreeIPA 4.4 and 4.5 do not support Python 3 yet.

That's all fine and neat but it looks like the current

python2-custodia-0.3.1-2.fc27.noarch conflicts with python2-ipalib < 4.5 provided by python2-ipalib-4.4.4-1.fc27.noarch

Why is python2-custodia picky about ipalib version?

Comment 9 Alexander Bokovoy 2017-04-18 10:00:23 UTC
In FreeIPA 4.4.4 we have:
 - python2-ipaserver requires python-custodia >= 0.3.1
 - freeipa-server requires custodia >= 0.3.1

In custodia package we have:

# FreeIPA 4.3 and 4.4 are not compatible with custodia because the custodia
# script now runs under Python 3.
Conflicts:          freeipa-server-common < 4.5
Conflicts:          ipa-server-common < 4.5

Both exist in the same repo and cannot be installed at the same time. Fedora Rawhide does not have FreeIPA 4.5 yet due to various reasons related to FreeIPA itself, mostly because it is still too unstable, so I think a reasonable solution right now is to make sure custodia package in rawhide works with FreeIPA 4.4.4 like it does in F26 where there is custodia 0.3.1 as well.

Please consider unblocking use of FreeIPA 4.4.4 in rawhide.

Comment 10 Jan Pazdziora 2017-04-18 11:23:05 UTC
(In reply to Alexander Bokovoy from comment #9)
> reasonable solution right now is to make sure custodia package in rawhide
> works with FreeIPA 4.4.4 like it does in F26 where there is custodia 0.3.1
> as well.

I agree in general but please note that currently there is only custodia-0.2.0-3.fc26.noarch.rpm in http://dl.fedoraproject.org/pub/fedora/linux/development/26/Server/x86_64/os/Packages/c/

Comment 11 Christian Heimes 2017-04-18 12:37:09 UTC
Let's put some things straight first. The problem is purely a packaging issue. Custodia 0.3 and FreeIPA 4.4 are compatible. Custodia 0.3 just changed /usr/sbin/custodia from Python 2 to 3.

FreeIPA 4.4

FreeIPA 4.4.x has no version requirements for custodia. python2-ipaserver and ipa-server depend on any version of custodia, python-custodia and python3-custodia. The ipa-custodia.service starts /usr/sbin/custodia. Since custodia 0.3, /usr/sbin/custodia is no longer Python 2.7 but whatever the main Python version is (e.g. 3.6 on F26/rawhide). I added the version requirement after some developers mixed custodia 0.3.x with FreeIPA 4.4 and complained about broken ipa-replica-install.


FreeIPA 4.5

FreeIPA 4.5.0 depends on custodia >= 0.3.1. ipa-custodia.service doesn't use /usr/sbin/custodia any more. Instead it uses a specialized start script /usr/libexec/ipa/ipa-custodia. The script serves two purposes: 1) it allows IPA to have a separate SELinux context for ipa-custodia, 2) it gives IPA full control over the Python version for ipa-custodia.

I see two ways to address the root cause of incompatibility between custodia 0.3 package and FreeIPA package:

1) Revert /usr/sbin/custodia to Python 2 and make the custodia package depend on python2-custodia instead of python3-custodia.
2) Change FreeIPA 4.4.x to depend on custodia 0.3.1 and to run /usr/sbin/custodia-2 instead of /usr/sbin/custodia.

I'd go for variant (2) for multiple reasons. I can keep 'custodia' package Python 3. FreeIPA 4.4 would also benefit from a more recent Custodia version. Custodia 0.2 and 0.3.0 have a bug related to handling of white space and other non-alphanumeric characters in secret names.

Comment 12 Jan Pazdziora 2017-04-18 12:57:02 UTC
(In reply to Christian Heimes from comment #11)
> Let's put some things straight first. The problem is purely a packaging
> issue. Custodia 0.3 and FreeIPA 4.4 are compatible. Custodia 0.3 just
> changed /usr/sbin/custodia from Python 2 to 3.

Is this change a critical part of the upgrade from Custodia 0.2 to 0.3, or just a coincidence? What would happen if you respun custodia 0.3* to run /usr/sbin/custodia with Python 2?

> FreeIPA 4.4
> 
> FreeIPA 4.4.x has no version requirements for custodia. python2-ipaserver
> and ipa-server depend on any version of custodia, python-custodia and
> python3-custodia. The ipa-custodia.service starts /usr/sbin/custodia. Since
> custodia 0.3, /usr/sbin/custodia is no longer Python 2.7 but whatever the
> main Python version is (e.g. 3.6 on F26/rawhide). I added the version
> requirement after some developers mixed custodia 0.3.x with FreeIPA 4.4 and
> complained about broken ipa-replica-install.

It might make sense for FreeIPA 4.4 in Fedora to be specific about using /usr/sbin/custodia-2.

> FreeIPA 4.5
> 
> FreeIPA 4.5.0 depends on custodia >= 0.3.1. ipa-custodia.service doesn't use
> /usr/sbin/custodia any more. Instead it uses a specialized start script
> /usr/libexec/ipa/ipa-custodia. The script serves two purposes: 1) it allows
> IPA to have a separate SELinux context for ipa-custodia, 2) it gives IPA
> full control over the Python version for ipa-custodia.
> 
> I see two ways to address the root cause of incompatibility between custodia
> 0.3 package and FreeIPA package:
> 
> 1) Revert /usr/sbin/custodia to Python 2 and make the custodia package
> depend on python2-custodia instead of python3-custodia.
> 2) Change FreeIPA 4.4.x to depend on custodia 0.3.1 and to run
> /usr/sbin/custodia-2 instead of /usr/sbin/custodia.
> 
> I'd go for variant (2) for multiple reasons. I can keep 'custodia' package
> Python 3. FreeIPA 4.4 would also benefit from a more recent Custodia
> version. Custodia 0.2 and 0.3.0 have a bug related to handling of white
> space and other non-alphanumeric characters in secret names.

In Fedora 26, I'd actually recommend to do both, freeing your hands to put Custodia 0.3 to Fedora 26 as soon as possible.

In rawhide, given FreeIPA 4.5 won't be available for another week, going with 1 would be a quick fix for now and once 4.5 lands, it would not longer be needed and you could move to Python 3 with /usr/sbin/custodia in controlled manner.

Comment 13 Jan Pazdziora 2017-04-18 12:59:42 UTC
By the way, the fact that FreeIPA 4.5 depends on Custodia 0.3+ does not really explain why we have

python2-custodia-0.3.1-2.fc27.noarch conflicts with python2-ipalib < 4.5 provided by python2-ipalib-4.4.4-1.fc27.noarch

there. Why is such a conflict needed for a Python 2 package?

Comment 14 Petr Vobornik 2017-04-18 13:45:39 UTC
Before FreeIPA 4.5.1 is released and put to rawhide, I don't see an issue with backporting https://pagure.io/freeipa/c/403263df7a3be61086c87c5577698cf32a912065 to FreeIPA 4-4 and then patching rawhide.

If custodia 3.x is needed in F26 then we can to the same with f26. 

Both will require to remove the conflict mentioned in comment 13.

Christian, do you agree and will it resolve the situation?

Comment 15 Fedora Update System 2017-05-31 07:46:18 UTC
custodia-0.3.1-2.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-c15bb90a4f

Comment 16 Fedora Update System 2017-06-01 18:05:18 UTC
custodia-0.3.1-2.fc26 freeipa-4.4.4-2.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-0d0ec3e0af

Comment 17 Fedora Update System 2017-06-01 18:06:19 UTC
custodia-0.3.1-2.fc26 freeipa-4.4.4-2.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-0d0ec3e0af

Comment 18 Fedora Update System 2017-06-04 19:38:58 UTC
custodia-0.3.1-2.fc26, freeipa-4.4.4-2.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-0d0ec3e0af

Comment 19 Fedora Update System 2017-06-09 19:08:59 UTC
custodia-0.3.1-2.fc26, freeipa-4.4.4-2.fc26 has been pushed to the Fedora 26 stable repository. If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.