Bug 1309259 - python-swiftclient: Provide a Python 3 subpackage
python-swiftclient: Provide a Python 3 subpackage
Status: CLOSED RAWHIDE
Product: Fedora
Classification: Fedora
Component: python-swiftclient (Show other bugs)
24
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Alan Pevec
Fedora Extras Quality Assurance
:
Depends On:
Blocks: PYTHON3
  Show dependency treegraph
 
Reported: 2016-02-17 05:35 EST by Petr Viktorin
Modified: 2016-07-12 19:53 EDT (History)
7 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-07-12 19:53:32 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Patchset 2 (3.82 KB, patch)
2016-05-27 06:51 EDT, Sachin
sacpatil: review?
Details | Diff

  None (edit)
Description Petr Viktorin 2016-02-17 05:35:39 EST
Upstream, this software supports Python 3. Please provide a Python 3
package for Fedora.


According to the Python packaging guidelines [0], software must be
packaged for Python 3 if upstream supports it.
The guidelines give detailed information on how to do this, and even
provide an example spec file [1].

The current best practice is to provide subpackages for the two Python
versions (called "Common SRPM" in the guidelines). Alternatively, if
nothing depends on your Python2 package, you can just switch to Python 3
entirely.

It's fine to do this in Rawhide only.


If anything is unclear, or if you need any kind of assistance with the
porting, you can ask on IRC (#fedora-python on Freenode), or reply here.
We'll be happy to help!


[0] https://fedoraproject.org/wiki/Packaging:Python
[1] https://fedoraproject.org/wiki/Packaging:Python#Example_common_spec_file
Comment 1 Jan Kurik 2016-02-24 10:23:46 EST
This bug appears to have been reported against 'rawhide' during the Fedora 24 development cycle.
Changing version to '24'.

More information and reason for this action is here:
https://fedoraproject.org/wiki/Fedora_Program_Management/HouseKeeping/Fedora24#Rawhide_Rebase
Comment 2 Dominika Krejčí 2016-04-20 08:42:13 EDT
Hello Alan,

Do you need any help adding Python 3 support to the RPM?

If you need more instructions, a [guide] for porting Python-based RPMs is available.

[guide] http://python-rpm-porting.readthedocs.org/en/latest/index.html
Comment 3 Sachin 2016-05-25 11:21 EDT
Created attachment 1161483 [details]
Provides Python3 subpackge

Hi Alan,

Can you review the patch and let me know if anything need to done.
Comment 4 Tomas Orsava 2016-05-25 12:02:10 EDT
Hi Sachin! Thank you for taking the time to make a patch!


As the maintainer has been unresponsive for over 3 months, I'll try to review the patch myself:

- Don't forget to increase the Release number with each change to the spec file you want to push. I.e. now it should be 2 instead of 1.

- Can I ask why did you add the BuildRequires on `python2/3-setuptools`?

- Beware that the Python 3 subpackage mistakenly Requires the Python 2 package `python-futures`.


And lastly, as this is a package containing a Python module as well as an application (a command line utility in this case), a special care needs to be taken with the executable that will be placed into `/usr/bin/`.

There is a dedicated chapter to this type of package on the Python RPM Porting Guide [0]. Specifically, check the %install [1] and %files section [2]. The last section [3] is important as well.

[0] http://python-rpm-porting.readthedocs.io/en/latest/application-modules.html
[1] http://python-rpm-porting.readthedocs.io/en/latest/application-modules.html#install
[2] http://python-rpm-porting.readthedocs.io/en/latest/application-modules.html#files
[3] http://python-rpm-porting.readthedocs.io/en/latest/application-modules.html#have-you-broken-third-party-packages


Thank you fow your time and effort,
and have a great day!
Comment 5 Tomas Orsava 2016-05-26 04:58:45 EDT
I am sorry, the BuildRequires on `python2-setuptools` was in the original spec file as well, I missed it, apologies.
Comment 6 Sachin 2016-05-26 05:07:13 EDT
Tomas,

- I'll keep in mind to update release henceforth.

- I m trying to re-use the existing spec file as much as possible, this is the reason I kept 'python2/3-setuptools' as BuildRequires. 

- Is it OK to include python-future? or not?
Comment 7 Tomas Orsava 2016-05-26 05:13:05 EDT
(In reply to Sachin from comment #6)
> - Is it OK to include python-future? or not?

If a Fedora sub/package has the prefix "python3-" you know it works (only) with Python 3. If it has profixes "python-" or "python2-", you know it (only) works with Python 2.

The Python 2 subpackage of this package needs the `python-future` package, so leave the requirement there. But the Python 3 subpackage CANNOT use `python-future` because it's only for Python 2.

Either drep the requirement on `python-future` from the Python 3 subpackage, or require the `python3-future` package instead.
Comment 8 Pete Zaitcev 2016-05-26 13:09:20 EDT
Alan has advanced to the management role, he runs the RDO overall now.
So it may take him forever. It's going to be either I or Haikel
actually fixing this. Sorry. If anyone can move it along, it would be great.

Re. Tomas' comment #4, what we really avoid here is the situation like this:

[root@lembas python-swiftclient-tip]# rpm -ql python3-os-testr | grep bin/ostestr
/usr/bin/ostestr
[root@lembas python-swiftclient-tip]# rpm -ql python-os-testr | grep bin/ostestr
[root@lembas python-swiftclient-tip]# rpm -q python-os-testr python3-os-testr
python-os-testr-0.4.1-3.fc23.noarch
python3-os-testr-0.4.1-3.fc23.noarch
[root@lembas python-swiftclient-tip]# 

It causes some aggravation when people forget to install py3 equivalents
and then things blow up because expected binaries aren't there.

In addition, Swift CLI has some issues with py3 upstream, which were on
my TODO forever. It's mostly repr-ing every string, so the output has
u'foo' in object listings. I thought I'd fix those first and then
package a py3 version that works properly. But since we face the
urgency, maybe just package whatever we already have and fix later.

For example:

[zaitcev@lembas ~]$ PYTHONPATH=/q/zaitcev/hail/python-swiftclient-tip /q/zaitcev/hail/python-swiftclient-tip/bin/swift -v -A http://localhost:8000/auth/v1.0 -U 'test:tester' -K "testing" list othercont
Container u'othercont' not found
[zaitcev@lembas ~]$ 
python-swiftclient 2.4.0.42.g788afdb
Comment 9 Sachin 2016-05-27 02:32:47 EDT
Tomas,

What do you think is good? python-swiftclient or python2-swiftclient (for py2)
Comment 10 Tomas Orsava 2016-05-27 06:18:22 EDT
(In reply to Pete Zaitcev from comment #8)
> Alan has advanced to the management role, he runs the RDO overall now.
> So it may take him forever. It's going to be either I or Haikel
> actually fixing this. Sorry. If anyone can move it along, it would be great.

Hi Alan! Thank you for responding!



> Re. Tomas' comment #4, what we really avoid here is the situation like this:
> 
> [root@lembas python-swiftclient-tip]# rpm -ql python3-os-testr | grep
> bin/ostestr
> /usr/bin/ostestr
> [root@lembas python-swiftclient-tip]# rpm -ql python-os-testr | grep
> bin/ostestr
> [root@lembas python-swiftclient-tip]# rpm -q python-os-testr python3-os-testr
> python-os-testr-0.4.1-3.fc23.noarch
> python3-os-testr-0.4.1-3.fc23.noarch
> [root@lembas python-swiftclient-tip]# 
> 
> It causes some aggravation when people forget to install py3 equivalents
> and then things blow up because expected binaries aren't there.

I understand the frustration, sadly there is no easy way around it.

To quote the Packaging guidelines [0]: "If the executables provide the same functionality independent of whether they are run on top of Python 2 or Python 3, then only the Python 3 version of the executable should be packaged."

[0] https://fedoraproject.org/wiki/Packaging:Python#Executables_in_.2Fusr.2Fbin

The transition is a bit painful in the short term, but long term it will lead to a much healthier ecosystem. So I think we should bite the bullet as soon as possible. 

To address the pains of this transition I've written a section in the Python RPM Porting Guide (PyRPG), which should make it as painless as possible [1].

[1] http://python-rpm-porting.readthedocs.io/en/latest/application-modules.html#have-you-broken-third-party-packages



> In addition, Swift CLI has some issues with py3 upstream, which were on
> my TODO forever. It's mostly repr-ing every string, so the output has
> u'foo' in object listings. I thought I'd fix those first and then
> package a py3 version that works properly. But since we face the
> urgency, maybe just package whatever we already have and fix later.
> 
> For example:
> 
> [zaitcev@lembas ~]$ PYTHONPATH=/q/zaitcev/hail/python-swiftclient-tip
> /q/zaitcev/hail/python-swiftclient-tip/bin/swift -v -A
> http://localhost:8000/auth/v1.0 -U 'test:tester' -K "testing" list othercont
> Container u'othercont' not found
> [zaitcev@lembas ~]$ 
> python-swiftclient 2.4.0.42.g788afdb

However, that does seem like a serious issue. If the Python 2 cli tool is working fine but the Python 3 isn't, it would be a regression to switch to it. Therefore I suggest we halt the package porting effort for the time being.

It would be Greatly appreciated if you could look at those issues!
And thank you for your time!
Comment 11 Tomas Orsava 2016-05-27 06:22:52 EDT
Sachin,
thank you very much for your initiative! Sadly it looks like the software needs some adjustments upstream first before it can be ported to Python 3 in Fedora.
May I interest you in another package per chance?

http://fedora.portingdb.xyz/#mispackaged
Comment 12 Sachin 2016-05-27 06:42:31 EDT
Hi Tomas,

Ok, I can turn my attention to something else, probably emacs-pymacs.
But let me submit the final patch of .spec file. This will be helpful when resuming the porting.
Comment 13 Sachin 2016-05-27 06:51 EDT
Created attachment 1162419 [details]
Patchset 2

Final patchset before halting porting
Comment 14 Tomas Orsava 2016-05-27 08:01:36 EDT
Sachin,
that's a very good idea, thank you!
Hopefully Pete can then use it later on.
Though I'll remove the bug from PY3PATCH-AVAILABLE as we cannot yet push the patch.
Comment 15 Sachin 2016-05-27 08:12:09 EDT
Tomas,

I tested the RPM with py3, works pretty well.
Comment 16 Tomas Orsava 2016-05-27 08:42:49 EDT
Sachin,
Pete says there are still problems with it. He probably knows it better than the two of us.
Comment 17 Haïkel Guémar 2016-07-12 19:53:32 EDT
Fixed in rawhide (I work with Alan and Pete).
And Pete is correct when he said that python3 support is not mature, it may work but it is not as fully tested in upstream CI. There are still corner case where it breaks. Especially, as we're still introducing python 3.5 jobs.

Note that we'll switch CLI of all OpenStack clients to python2 by default asap. Most of them will be switching to OpenStackClient architecture, so only python-openstackclient will be providing binary CLI (other *clients will be mere python libraries). Note that some clients may not switch that way, and swift is likely one of them.

Note You need to log in before you can comment on or make changes to this bug.