Created attachment 366691 [details] Hacked-up specfile Our python-setuptools rpm uses the "distribute" fork of setuptools. I had a go at changing the specfile to build against /usr/bin/python3 rather than /usr/bin/python, and it "just works". This approach gives us an almost-identical specfile. For the sake of experimentation I filed a review request for it as bug 531648 This is the approach I advocated in my proposal here: https://www.redhat.com/archives/fedora-devel-list/2009-October/msg00054.html Advantages of this approach: - if the python-foo maintainer doesn't care about python 3, he/she doesn't need to - the two specfiles can evolve separately; if 2 and 3 need to have different versions, they can Disadvantages of this approach: - the two specfiles have to be maintained separately - when upstream release e.g. security fixes, they have to be tracked in two places An alternative approach would be to support both 2 and 3 from one srpm build. It struck me that these specfiles are almost identical, so as an experiment, I had a go at hacking up the specfile to generate both 2 and 3 subpackages from one build. Attached is a version of the specfile that does so. Advantages of this approach: - single srpm and build; avoid having to update multiple packages when things change. Disadvantages of this approach: - Fedora maintainer needs to care about python 3. By adding python 3 to the mix, I'm giving them extra work. - 2 and 3 versions are in lockstep. Requires upstream to case about Python 3 as well. (Talking about setuptools, maybe it's possible to use setuptools to transparently do some of this work for other packages?) Thoughts?
This looks really cool. One question: how does the 2to3 tool work into this? Do source files have to have __future__ imports for Python 3 syntax, or can 2to3 be run just for the Python 3 version?
* I distrust creating the packages automatically with 2to3... not sure if that's a reasonable distrust or not, though. * When upstream doesn't have a unified source tree, we will need to have two separate packages at the srpm level. That means that not all packages will be the same there (but for the end user, it would look a lot alike). * Also requires upstream to care about python2. At some point this will become the issue instead of caring about python3. It will show up as foo-1.x supports python2 and foo-2.x supports python3. I find your solving of "when upstream release e.g. security fixes, they have to be tracked in two places" to be a great advantage, though....
Reviewed implementation -- that part looks good.
Another Con: Makes it harder to figure out what to checkout from cvs or what to file a bug against in bugzilla. For any python3-* package you first have to try: python3-foo. If that fails, then you try python-foo. (bugzilla is a worse problem than cvs as that touches end users).
This bug appears to have been reported against 'rawhide' during the Fedora 12 development cycle. Changing version to '12'. More information and reason for this action is here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping
(adding package name to bug title to make the dependency tree easier to read; fixing version)
Python 3 is in rawhide now (for more information see http://lists.fedoraproject.org/pipermail/devel/2010-January/129185.html ) I'm attaching a patch against the current CVS devel/python-setuptools.spec which adds a python3-setuptools subpackage. Note that the python3-devel subpackage defines __python3, python3_sitelib and python3_sitearch, so there's no need for them in this specfile. Successful scratch build into dist-f13 here: http://koji.fedoraproject.org/koji/taskinfo?taskID=1921992 I'm testing with a local build (on my F12 box) and it seems to work.
Created attachment 384449 [details] Adds python3 subpackage to the python-setuptools.spec file
Created attachment 384460 [details] modify python3 package, to not build on other dists than >fc12 It would be nice to be able to use the same spec for rhel ant other branches than devel. But I couldn'd disable building the python3 subpackage... With: 3 %if 0%{?fedora} > 12 4 %global with_python3 1 5 %endif and instead with_python3 -> 0%{?with_python3} this works. Koji builds: F-12 (without python3-*): http://koji.fedoraproject.org/koji/taskinfo?taskID=1922191 F-13 (with python3-*): http://koji.fedoraproject.org/koji/taskinfo?taskID=1922190
There's a bug in the proposed spec file. easy_install is the python3 version but placed in the python2 package.
dmalcolm: python question: Should %{_libdir}/python{VER}/lib2to3 be in the main python package instead of the tools package? It's placement makes it a public library that anyone can import and it's not in the Tools subdirectory. I can open a bug for both py2 and py3 if you'd like to consider that for a while before deciding.
Created attachment 384853 [details] Alternate spec building py2 and py3 Here's an alternate take on how this would look if you do not copy the source tree, just the build output. This is similar to the way other packages that are built twice work (like vim-enhanced/-minimal/-X11).
http://koji.fedoraproject.org/koji/taskinfo?taskID=1927333 dmalcolm a few questions, what's the command to verify that the proper byte code is in this file? I ran rpmlint on the generated rpms but I don't know if F-12's rpmlint will detect improper byte code. Also, I tried this experiment: $ rpmdev-extract python3-setuptool-*.rpm $ cd python3-setuptools*noarch/usr/lib/python3.1/site-packages $ python3 -c 'import pkg_resources; print(pkg_resources.__file__)' pkg_resources.py $ mv pkg_resources.py ../ $ python3 -c 'import pkg_resources; print(pkg_resources.__file__)' pkg_resources.pyc (This is with an older build of python3.1.1 on F-12) This happens with both tomspur's build and my build. When I cd to the python2 rpm's directory, python2 -c 'import pkg_resources ; print pkg_resources.__file__' prints pkg_resources.pyc Could be a misconfiguration in my old python3 build but also might be a product of how we're building these. Want to check that out?
(In reply to comment #11) > dmalcolm: python question: Should %{_libdir}/python{VER}/lib2to3 be in the main > python package instead of the tools package? It's placement makes it a public Yes, please file a bug.
(In reply to comment #12) > Created an attachment (id=384853) [details] > Alternate spec building py2 and py3 > > Here's an alternate take on how this would look if you do not copy the source > tree, just the build output. This is similar to the way other packages that > are built twice work (like vim-enhanced/-minimal/-X11). This approach makes the assumption that the %build phase doesn't have side-effects on the source tree. For a well-behave source tree that shouldn't matter, but I'm sure we'll encounter awkward cases where "%build" does something to the source tree. For these, the original approach of copying the source tree will work, whereas the suggested approach will fail. By way of example, if a %build does a "configure" without changing to a build subdirectory, you'll have "droppings" from the configure run, along with a "Makefile" in the root directory of the build, and so the two builds could start confusing each other.
(In reply to comment #12) > Created an attachment (id=384853) [details] > Alternate spec building py2 and py3 > > Here's an alternate take on how this would look if you do not copy the source > tree, just the build output. This is similar to the way other packages that > are built twice work (like vim-enhanced/-minimal/-X11). BTW, this doesn't seem to define the "py2dir" that it references in a few places.
(In reply to comment #13) > http://koji.fedoraproject.org/koji/taskinfo?taskID=1927333 > > dmalcolm a few questions, what's the command to verify that the proper byte > code is in this file? I ran rpmlint on the generated rpms but I don't know if > F-12's rpmlint will detect improper byte code. Bug 531102 is tracking the new rpmlint test; I'm not sure what its current status is (it's in the upstream rpmlint). I patched "file" to detect this, see bug 531082. You can test this manually using hexdump on a .pyo file, for 3.1, the first four bytes should be: 4f 0c 0d 0a (BTW I suspect that F-12's rpmbuild doesn't gets this right; this is bug 531117, which I believe is only fixed for F-13) [snip]
What's the state on this? With Toshios' new example spec from comment 13 python-minimock builds great with this: SRPM: http://tomspur.fedorapeople.org/python3/python-minimock-1.2.5-2.fc12.src.rpm I also tested the little example from upstream at: http://pypi.python.org/pypi/MiniMock#introduction Works as excepted. (Didn't test it with the spec file from comment 9.)
(Sorry, don't want to seem pushy... I just want to further test, before the python3 guidelines are approved, and something went wrong)
Created attachment 386647 [details] New version of the patch to python-setuptools.spec Hi Thomas! Thanks for looking at this. Toshio and I have been working on the guidelines on IRC. The biggest change has been to arrange for %{py3dir} to be automatically supplied by the python3-devel macros. I'm attaching an updated version of patch. I've tried to update this to reflect the latest state of https://fedoraproject.org/wiki/PackagingDrafts/Python3, as a sanity check of both the guidelines and this package. This version uses the py3dir definition from the latest build of python3-devel. Scratch build into dist-f13: http://koji.fedoraproject.org/koji/taskinfo?taskID=1943288
I've committed what we have so far to the development repo but with with_python3 disabled until we get the Guidelines finalized and the brp_python_bytecompile script fixed. here's a build from that spec file: http://koji.fedoraproject.org/koji/taskinfo?taskID=1951161
I've been looking at rewriting the last portions of the Python3 page to be in the style of Guidelines and see one thing -- The current page talks about the packager running 2to3 on the sources. This isn't really something that I was anticipating when I wrote the bit about deciding on split vs single srpm. That portion only anticipates upstreams actively distributing both python2 and python3 modules. Do we want packagers to run 2to3 on python2 code when upstream is not actively supporting python3? If we do, do we want to require packagers to make separate srpms for these?
*** Bug 531648 has been marked as a duplicate of this bug. ***
Toshio enabled the subpackage and built it into dist-f13 as python-setuptools-0.6.10-3.fc13: http://koji.fedoraproject.org/koji/taskinfo?taskID=1963883 The buildArch subtask is here http://koji.fedoraproject.org/koji/taskinfo?taskID=1963890 and does indeed contain python3-setuptools-0.6.10-3.fc13.noarch.rpm ; I quickly reviewed the build.log and it appeared sane. Closing; thanks!