Bug 533290 - RFC: Emit python3 subpackage as part of the python-lxml build
RFC: Emit python3 subpackage as part of the python-lxml build
Status: CLOSED RAWHIDE
Product: Fedora
Classification: Fedora
Component: python-lxml (Show other bugs)
rawhide
All Linux
low Severity medium
: ---
: ---
Assigned To: Jeffrey C. Ollie
Fedora Extras Quality Assurance
:
Depends On: 531895
Blocks: Python3F13
  Show dependency treegraph
 
Reported: 2009-11-05 16:50 EST by Dave Malcolm
Modified: 2009-11-06 12:08 EST (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2009-11-05 21:08:28 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)
Patch to devel/python-lxml.spec that adds a python3- subpackage (2.67 KB, patch)
2009-11-05 16:55 EST, Dave Malcolm
no flags Details | Diff
Final patch (3.18 KB, patch)
2009-11-05 21:10 EST, Jeffrey C. Ollie
no flags Details | Diff

  None (edit)
Description Dave Malcolm 2009-11-05 16:50:45 EST
I'm working on adding a python 3 stack to Fedora 13, parallel-installable with the main python 2 stack.

There are two approaches I'm experimenting with to packaging modules for python 3:
  (a) create an separate specfile/srpm for the python 3 version
  (b) extend an existing specfile so that it emits a python3- subpackage as part of the build.

lxml's upstream appears to be attempting to support both python 2 and python 3 with a single release (cython does a lot of this work for them).

See the discussion in bug 531895 (for python(3)-setuptools); (a) gives more flexibility, but (b) may be simpler - provided that the upstream releases always build on both python 2 and python 3.

So I had a go at approach (b) for python-lxml.  I'm about to attach a patch to python-lxml.spec to add a boolean that adds a python3-lxml subpackage to the build.  

Seems to work (given locally built python3 and python3-setuptools packages, see the dependency tree associated with this bug); the debuginfo is shared by both 2 and 3 subpackages:

[david@brick devel]$ ls -al i386/
total 6392
drwxr-xr-x. 2 david david    4096 2009-11-05 16:33 .
drwxrwxr-x. 6 david david    4096 2009-11-05 16:33 ..
-rw-r--r--. 1 david david 2041324 2009-11-05 16:33 python3-lxml-2.2.3-1.fc13.i386.rpm
-rw-r--r--. 1 david david 2043300 2009-11-05 16:33 python-lxml-2.2.3-1.fc13.i386.rpm
-rw-r--r--. 1 david david 2445820 2009-11-05 16:33 python-lxml-debuginfo-2.2.3-1.fc13.i386.rpm

[david@brick devel]$ python3
Python 3.1.1 (r311:74480, Nov  4 2009, 12:20:53) 
[GCC 4.4.1 20090725 (Red Hat 4.4.1-2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from lxml import etree
>>> root = etree.fromstring("<root>data</root>")
>>> root
<Element root at 8a37234>
>>> etree.tostring(root)
b'<root>data</root>'
Comment 1 Dave Malcolm 2009-11-05 16:55:33 EST
Created attachment 367737 [details]
Patch to devel/python-lxml.spec that adds a python3- subpackage

Notes:
  - no %changelog entry yet
  - execfile() was a builtin function in python 2 but doesn't exist in python 3 AFAIK, so I've reworked the copy&pasted scriptlet fragments to avoid using it
Comment 2 Andrew McNabb 2009-11-05 17:49:57 EST
This is really cool.  I didn't expect this to be quite so easy.  Just a couple of quick questions:

1) Does %{py3dir} need to be manually deleted?

2) Does the code for lxml work unchanged for both Python 2 and Python 3, or is there anything magical happening to make changes for Python 3?

Keep up the good work.
Comment 3 Dave Malcolm 2009-11-05 17:59:43 EST
(In reply to comment #2)
> This is really cool.  I didn't expect this to be quite so easy.  Just a couple
> of quick questions:
> 
> 1) Does %{py3dir} need to be manually deleted?
Possibly in the %clean section, yes.

> 2) Does the code for lxml work unchanged for both Python 2 and Python 3, or is
> there anything magical happening to make changes for Python 3?

The magic is provided by "Cython", which autogenerates the c extension code from a .pyx file.  As I understand it, Cython handles the nasty details of 2 vs 3 differences to the extension API (and is thus a big win over its predecessor, Pyrex).

(Looks like the python-lxml specfile is using the .c files embedded in the upstream tarball, as there isn't a BuildRequires on cython)

> Keep up the good work.
Thanks!
Comment 4 Jeffrey C. Ollie 2009-11-05 21:08:28 EST
Can this even be built in devel yet?  As fas as I can see python3 hasn't made it into CVS or the F-13 buildroots. Anyway I've added a modified version of the patch to CVS that defaults to not building the Python 3 subpackage.  Once Python 3 makes it to the buildroots we'll need to flip the flag and rebuild.
Comment 5 Jeffrey C. Ollie 2009-11-05 21:10:03 EST
Created attachment 367772 [details]
Final patch
Comment 6 Dave Malcolm 2009-11-06 11:56:59 EST
> Can this even be built in devel yet?  As fas as I can see python3 hasn't made
> it into CVS or the F-13 buildroots. Anyway I've added a modified version of the

Correct, the "python3" package hasn't made it through review yet (bug 526126), likewise for python3-setuptools (bug 531648 or bug 531895, depending on which approach we follow).

I'm still experimenting with approaches to packaging modules for python 3, and lxml is an interesting case, as 2 and 3 seem to work cleanly from a single upstream tarball.  So I'm afraid I'm using lxml as an experimental subject to some extent.  Sorry about this, and thanks for bearing with me.

I've written some notes on python 3 packaging here:
https://fedoraproject.org/wiki/PackagingDrafts/Python3  (this is very much a work-in-progress).

> patch to CVS that defaults to not building the Python 3 subpackage.  Once
> Python 3 makes it to the buildroots we'll need to flip the flag and rebuild.  

Thanks - a good plan.

BTW, I noticed that the version you committed to CVS had a definition of python3_sitearch.  In python3-3.1.1-9 onwards I'm dropping a "macros.python3" file into /etc/rpm which should make this unnecessary (see https://bugzilla.redhat.com/show_bug.cgi?id=526126#c43 ), and perhaps we'll write it out of the packaging guidelines when we write them for python 3.    Though it doesn't hurt to have it, especially given that there isn't an "official" version in the buildroots yet.
Comment 7 Jeffrey C. Ollie 2009-11-06 12:08:51 EST
(In reply to comment #6)
> 
> BTW, I noticed that the version you committed to CVS had a definition of
> python3_sitearch.  In python3-3.1.1-9 onwards I'm dropping a "macros.python3"
> file into /etc/rpm which should make this unnecessary (see
> https://bugzilla.redhat.com/show_bug.cgi?id=526126#c43 ), and perhaps we'll
> write it out of the packaging guidelines when we write them for python 3.   
> Though it doesn't hurt to have it, especially given that there isn't an
> "official" version in the buildroots yet.  

I didn't see that until after I had made the changes to CVS, but in the specfile it's a conditional definition, if python3_sitearch is already defined it should take precedence.

Note You need to log in before you can comment on or make changes to this bug.