Bug 839321

Summary: Request for help in a sphinx patch for sagemath
Product: [Fedora] Fedora Reporter: Paulo Andrade <paulo.cesar.pereira.de.andrade>
Component: python-sphinxAssignee: Michel Lind <michel>
Status: CLOSED EOL QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 19CC: a.badger, alex, michel, samuel-rhbugs
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-02-17 14:20:42 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Paulo Andrade 2012-07-11 15:01:56 UTC
In my current work in progress sagemath package, I noticed/remembered of a patch I applied to mandriva python-sphinx package, but I believe is not really correct and probably too much intrusive. I opened a sagemath trac about it at http://trac.sagemath.org/sage_trac/ticket/13230, and the patch is basically:

diff -ur src/sphinx/highlighting.py src.patched/sphinx/highlighting.py
--- src/sphinx/highlighting.py	2010-08-11 17:17:48.000000000 +0200
+++ src.patched/sphinx/highlighting.py	2010-11-28 12:04:44.068642703 +0100
@@ -161,7 +161,7 @@
 
         # find out which lexer to use
         if lang in ('py', 'python'):
-            if source.startswith('>>>'):
+            if source.startswith('>>>') or source.startswith('sage: '):
                 # interactive session
                 lexer = lexers['pycon']
             else:

what causes the sage notebook documentation to be "interactive" instead of static.

Example of notebook with the sagemath package built without the sphynx patch:
http://fedorapeople.org/~pcpa/rawhide-sage-notebook2.png

and built with the above sphinx patch:
http://fedorapeople.org/~pcpa/rawhide-sage-notebook3.png

Comment 1 Toshio Ernie Kuratomi 2012-07-11 15:24:37 UTC
Reading your patch, it looks like it can be fixed from within the documentation using code-block:
  http://sphinx.pocoo.org/markup/code.html
  http://pygments.org/docs/lexers/#lexers-for-agile-languages

The pygments name for the python console highlighter is pycon.  So something like this seems like it will do the same thing as your patch:

.. code-block:: pycon
    # sage: interactive code

Note that I was not aware that the pycon lexer could handle alternate prompts ("sage:" seems to replace ">>>" in those examples on all lines. Not just for purposes of detecting an interactive session).  If this doesn't work for you, I suspect that there's other changes to the lexer in addition to the patch you have above.  If that's the case, you may need to derive your own lexer from the pycon lexer with those changes.  The pygments documentation on that is here: http://pygments.org/docs/lexerdevelopment/

Comment 2 Paulo Andrade 2012-09-07 16:24:22 UTC
I believe it would be easier if there was some way to tell sphynx to handle "sage: " as it would handle ">>>". It would be prohibitive to maintain patches for extra "... code-block:: pycon" notations on every sagemath file, and upstream would not like it much either.

I believe the sagemath patch is really bad in their sphynx package. For now, when building the rpm package, it should

find spkg/build/sage-%{version} \
    \( -name \*.pxd -o -name \*.pyx -o -name \*.py -o -name \*.rst \) | xargs \
    sed -i 's|\(^[[:space:]]*\)sage: |\1>>> |g'

after any %patch.

Another very ugly hack, but that would make it easier to make or rediff patches  on a live build, would be to put a patched copy of sphinx's highlighting.py in $PYTHONPATH during documentation build.

Comment 3 Paulo Andrade 2012-09-07 19:15:38 UTC
I need to use the "very ugly hack" or do nothing (resulting in no interactive forms in the tutorials), because the s/sage:/>>>/ broke doctest and related scripts, that expect the sage: prefix.

Should use something like:

%if %{with_sphinx_hack}
    cp -far %{python_sitelib}/sphinx %{buildroot}%{python_sitearch}
    sed -i "s|\(source.startswith('>>>')\)|\1 or source.startswith('sage: ')|" \
	%{buildroot}%{python_sitearch}/sphinx/highlighting.py
%endif
[...]
%if %{with_sphinx_hack}
    rm -fr %{buildroot}%{python_sitearch}/sphinx
%endif

as it already adds that path to PYTHONPATH to be able to start sage during %install and build the documentation.

Comment 4 Michel Lind 2012-11-19 04:03:20 UTC
(In reply to comment #2)
> I believe it would be easier if there was some way to tell sphynx to handle
> "sage: " as it would handle ">>>". It would be prohibitive to maintain
> patches for extra "... code-block:: pycon" notations on every sagemath file,
> and upstream would not like it much either.

Why would upstream not like it? I agree that it'd be prohibitively tedious for you to maintain the patches yourself, but if this work is delegated either to sagemath packagers, or to Sphinx package maintainers, you're multiplying the amount of work to be done as well, as different distributions would all have to be patched.

Comment 5 Paulo Andrade 2012-11-23 01:07:49 UTC
(In reply to comment #4)
> (In reply to comment #2)
> > I believe it would be easier if there was some way to tell sphynx to handle
> > "sage: " as it would handle ">>>". It would be prohibitive to maintain
> > patches for extra "... code-block:: pycon" notations on every sagemath file,
> > and upstream would not like it much either.
> 
> Why would upstream not like it? I agree that it'd be prohibitively tedious
> for you to maintain the patches yourself, but if this work is delegated
> either to sagemath packagers, or to Sphinx package maintainers, you're
> multiplying the amount of work to be done as well, as different
> distributions would all have to be patched.

I did not look further on it besides doing a lot of work
in the sagemath package, that now has a review request
(#877651), because so far I am happy with the hack in #c3,
but did not ping upstream developers in the trac I opened
at http://trac.sagemath.org/sage_trac/ticket/13230

The simplest solution would be to somehow, in an "standard"
way, tell sphinx to handle 'sage: ' the same way as '>>>'.
But I wonder if the ipython notebook does not have some
examples of how to do it, need to dig in the code :-)

I said upstream would not like it, but in truth I just
assumed it :-), and only if it would requires patching
every code block, because there is a lot of documentation,
and could be required to be done as a manual process,
that would also happen to cause a lot of problems
with pending patches in tracs that would no longer apply
cleanly.

Well, so far not much else, but will try to debug it...

Comment 6 Fedora End Of Life 2013-04-03 16:19:43 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 19 development cycle.
Changing version to '19'.

(As we did not run this process for some time, it could affect also pre-Fedora 19 development
cycle bugs. We are very sorry. It will help us with cleanup during Fedora 19 End Of Life. Thank you.)

More information and reason for this action is here:
https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Fedora19

Comment 7 Fedora End Of Life 2015-01-09 17:15:03 UTC
This message is a notice that Fedora 19 is now at end of life. Fedora 
has stopped maintaining and issuing updates for Fedora 19. It is 
Fedora's policy to close all bug reports from releases that are no 
longer maintained. Approximately 4 (four) weeks from now this bug will
be closed as EOL if it remains open with a Fedora 'version' of '19'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 19 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 8 Fedora End Of Life 2015-02-17 14:20:42 UTC
Fedora 19 changed to end-of-life (EOL) status on 2015-01-06. Fedora 19 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.