Bug 537244 - RFE: python3-pygments
RFE: python3-pygments
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: python-pygments (Show other bugs)
rawhide
All Linux
low Severity medium
: ---
: ---
Assigned To: Steve Milner
Fedora Extras Quality Assurance
:
Depends On:
Blocks: Python3F13 590107 629303
  Show dependency treegraph
 
Reported: 2009-11-12 16:55 EST by Dave Malcolm
Modified: 2010-11-15 19:38 EST (History)
3 users (show)

See Also:
Fixed In Version: python-pygments-1.3.1-7.fc14
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2010-11-15 19:38:57 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Adaptation of python-pygments.spec to build for python3 instead (2.90 KB, text/plain)
2009-11-12 18:13 EST, Dave Malcolm
no flags Details
Patch against F-13 python-pygments.spec to add a python3 subpackage (3.02 KB, patch)
2010-04-03 17:43 EDT, Dave Malcolm
no flags Details | Diff
Patch against F-13 python-pygments.spec to add a python3 subpackage (3.03 KB, patch)
2010-04-03 17:44 EDT, Dave Malcolm
no flags Details | Diff
Patch against F-13 python-pygments.spec to add a python3 subpackage (3.04 KB, patch)
2010-04-03 17:46 EDT, Dave Malcolm
no flags Details | Diff
Patch against devel/python-pygments.spec to add a python3 subpackage (3.15 KB, patch)
2010-04-23 10:26 EDT, Dave Malcolm
no flags Details | Diff

  None (edit)
Description Dave Malcolm 2009-11-12 16:55:16 EST
I'm working on adding a python 3 stack to Fedora 13, parallel-installable with
the main python 2 stack:
https://fedoraproject.org/wiki/Features/Python3F13

There are two approaches I'm experimenting with to packaging modules for python
3:
  (a) create an separate specfile/srpm for the python 3 version
  (b) extend an existing specfile so that it emits a python3- subpackage as
part of the build.

See: https://fedoraproject.org/wiki/PackagingDrafts/Python3

I experimented with python-pygments, and I was able to create a python3-pygments srpm fairly easily, invoking 2to3 at the end of the %prep phase; the only thing that needs fixing is for it to own /usr/bin/pygmmentize3, rather than /usr/bin/pygmentize, to ensure that both rpms are installable in parallel.

Would you be OK with a split SRPM arrangement?

Alternatively, do you know if upstream are planning to keep pygments working with both 2 and 3 (assuming a 2to3 run during build).  If so, would you accept a patch to python-pygments.spec to do this?
Comment 1 Steve Milner 2009-11-12 17:07:43 EST
I'm currently not aware of the plan upstream has for Python 3. I've opened a ticket up with them to find out:

    http://dev.pocoo.org/projects/pygments/ticket/448

Honestly, I'd be fine with doing them in a shared or split SRPM. Attach the patch to this BZ. If upstream notes that 2to3 is the proper way for Python 3 usage until they cut over then I'm all for it.
Comment 2 Dave Malcolm 2009-11-12 18:13:33 EST
Created attachment 369343 [details]
Adaptation of python-pygments.spec to build for python3 instead

Attached specfile gives me a somewhat working python3-pygments (via running 2to3 at the end of %prep).

If you'drather have a single specfile emit both rpms, I can put together a patch.

Works outputting to a file:
[david@brick devel]$ pygmentize3 -o foo.html -l python /usr/bin/pygmentize3

but fails writing to stdout:
[david@brick devel]$ pygmentize3 -l python /usr/bin/pygmentize3

*** Error while highlighting:
TypeError: must be str, not bytes
   (file "/usr/lib/python3.1/codecs.py", line 356, in write)
Comment 3 Dave Malcolm 2009-11-12 18:17:10 EST
FWIW, the full traceback when writing to terminal is:
Traceback (most recent call last):
  File "pygmentize", line 5, in <module>
    sys.exit(pygments.cmdline.main(sys.argv))
  File "/home/david/coding/python3/packaging/modules/by-hand/python-pygments/devel/Pygments-1.1.1/pygments/cmdline.py", line 416, in main
    highlight(code, lexer, fmter, outfile)
  File "/home/david/coding/python3/packaging/modules/by-hand/python-pygments/devel/Pygments-1.1.1/pygments/__init__.py", line 85, in highlight
    return format(lex(code, lexer), formatter, outfile)
  File "/home/david/coding/python3/packaging/modules/by-hand/python-pygments/devel/Pygments-1.1.1/pygments/__init__.py", line 68, in format
    formatter.format(tokens, outfile)
  File "/home/david/coding/python3/packaging/modules/by-hand/python-pygments/devel/Pygments-1.1.1/pygments/formatters/terminal.py", line 91, in format
    return Formatter.format(self, tokensource, outfile)
  File "/home/david/coding/python3/packaging/modules/by-hand/python-pygments/devel/Pygments-1.1.1/pygments/formatter.py", line 92, in format
    return self.format_unencoded(tokensource, outfile)
  File "/home/david/coding/python3/packaging/modules/by-hand/python-pygments/devel/Pygments-1.1.1/pygments/formatters/terminal.py", line 107, in format_unencoded
    outfile.write(ansiformat(color, spl[-1]))
  File "/usr/lib/python3.1/codecs.py", line 356, in write
    self.stream.write(data)
TypeError: must be str, not bytes
Comment 4 Steve Milner 2009-11-12 19:08:42 EST
OK, thanks for the upfront testing. I'm compiling the latest python3 from your SRPM and will take a look at it.
Comment 5 Dave Malcolm 2010-04-03 17:43:09 EDT
Created attachment 404345 [details]
Patch against F-13 python-pygments.spec to add a python3 subpackage

Given that upstream's tarballs are dual-purpose, the better approach is probably to build both python2 and python3 subpackages from a shared build.

Attached is a patch that adds a python3-pygments subpackage to the build.

I've tried to follow https://fedoraproject.org/wiki/Packaging:Python which has more notes on this.

The subpackage only provides the "pygments" module, not an executable; /usr/bin/pygmentize remains Python 2.

I've briefly tested the code, and it works (e.g. formatting iself thus):
[david@f13 F-13]$ python3 -c"import pygments.cmdline ; pygments.cmdline.main()" Pygments-1.3.1/setup.py | cat

(I had to pipe to cat, otherwise I get:
*** Error while highlighting:
TypeError: must be str, not bytes
   (file "/usr/lib64/python3.1/codecs.py", line 356, in write)
but that's a minor issue, I feel)

How does this look?
Comment 6 Dave Malcolm 2010-04-03 17:44:58 EDT
Created attachment 404346 [details]
Patch against F-13 python-pygments.spec to add a python3 subpackage

(hopefully with my correct email address in the %changelog this time)
Comment 7 Dave Malcolm 2010-04-03 17:46:21 EDT
Created attachment 404348 [details]
Patch against F-13 python-pygments.spec to add a python3 subpackage

(trying again, perhaps 3rd time is the charm)
Comment 8 Dave Malcolm 2010-04-23 10:26:04 EDT
Created attachment 408626 [details]
Patch against devel/python-pygments.spec to add a python3 subpackage

Here's an updated version of the patch, against the latest state of "devel" (and F-13).
Comment 9 Steve Milner 2010-04-23 11:16:19 EDT
Cool, I'll take a look at it soon.
Comment 10 Steve Milner 2010-04-23 11:29:14 EDT
Looks great! Thanks! Merged into main spec.

Build 2.x: http://koji.fedoraproject.org/koji/taskinfo?taskID=2134288
Comment 11 Dave Malcolm 2010-04-23 12:03:34 EDT
http://cvs.fedoraproject.org/viewvc/rpms/python-pygments/devel/python-pygments.spec?r1=1.15&r2=1.16

I noticed that you changed:

  -%global with_python3 1
  +%global with_python3 0

in the specfile, so that build didn't emit the python3 subpackage.
Comment 12 Steve Milner 2010-04-23 13:20:08 EDT
I just did a scratch build and it worked: 

    http://koji.fedoraproject.org/koji/taskinfo?taskID=2134415

I'll set it back to 1.
Comment 13 Thomas Spura 2010-04-25 20:46:26 EDT
(In reply to comment #12)
> I'll set it back to 1.    

When you use the code snipped from [1] you could also copy this spec file to other branches e.g. EL or fedora < 13:

%if 0%{?fedora} > 12 || 0%{?rhel} > 6
%global with_python3 1
%else
%{!?python_sitelib: %global python_sitelib %(%{__python} -c "from distutils.sysconfig import get_python_lib; print (get_python_lib())")}
%endif


[1] https://fedoraproject.org/wiki/Packaging:Python#Building_more_than_once
Comment 14 Terje Røsten 2010-09-05 09:50:59 EDT
Hi guys, what is going now here? 

I need 1.3.1-5.fc13 (with pyhton 3 support) and it's here:

 http://koji.fedoraproject.org/koji/buildinfo?buildID=173183

However, it's not pushed in bodhi, may I ask why?
Comment 15 Steve Milner 2010-09-11 15:55:16 EDT
Forgetfulness :-). I'll see if I can push it into Bodhi. Last attempt said it wasn't a valid build.
Comment 16 Steve Milner 2010-09-11 16:01:51 EDT
There has been some rebuilds done by mass builds that have not made it down. I'll see if I can't get them merged downward and released.
Comment 17 Steve Milner 2010-09-11 16:24:18 EDT
- http://koji.fedoraproject.org/koji/taskinfo?taskID=2461932
- http://koji.fedoraproject.org/koji/taskinfo?taskID=2461943

For some reason Bodhi doesn't see these builds. I'll try to find out why.
Comment 18 Terje Røsten 2010-09-12 04:52:47 EDT
In the web giu, you must push python-pygments-1.3.1-7.fc13 and not only
the subpackage python3-pygments-1.3.1-7.fc13. However, I guess you did that any way.
Comment 19 Fedora Update System 2010-09-13 15:09:56 EDT
python-pygments-1.3.1-7.fc13 has been submitted as an update for Fedora 13.
https://admin.fedoraproject.org/updates/python-pygments-1.3.1-7.fc13
Comment 20 Fedora Update System 2010-09-13 15:10:01 EDT
python-pygments-1.3.1-7.fc14 has been submitted as an update for Fedora 14.
https://admin.fedoraproject.org/updates/python-pygments-1.3.1-7.fc14
Comment 21 Steve Milner 2010-09-13 15:13:15 EDT
I stopped trying to add the subpackge through the webui and it seems to have worked (logs point to both 2.x and 3.x results) even though the actual links only state 2.x.
Comment 22 Fedora Update System 2010-09-24 16:41:34 EDT
python-pygments-1.3.1-7.fc13 has been pushed to the Fedora 13 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 23 Fedora Update System 2010-09-25 01:32:54 EDT
python-pygments-1.3.1-7.fc14 has been pushed to the Fedora 14 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 24 Thomas Spura 2010-11-15 19:38:57 EST
The python3 package is there.

-> Closing

Note You need to log in before you can comment on or make changes to this bug.