Bug 2127371 - python-pybtex fails to build docs with Pygments 2.13
Summary: python-pybtex fails to build docs with Pygments 2.13
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: python-pybtex
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Jerry James
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-09-16 07:24 UTC by Karolina Surma
Modified: 2022-09-20 15:16 UTC (History)
1 user (show)

Fixed In Version: python-pybtex-0.24.0-8.fc38
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-09-20 15:16:18 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Karolina Surma 2022-09-16 07:24:12 UTC
python-pybtex fails to build documentation with Pygments 2.13

I suspect the changein the discovery of plugins, described in the changelog, could cause it: https://pygments.org/docs/changelog/#version-2-13-0 

I managed to reproduce the issue also with Python 3.10 venv with pip-installed Pygments 2.13 on the master branch of pybtex but don't have any idea how to make it work.


+ PYTHONPATH=/builddir/build/BUILD/pybtex-0.24.0:/builddir/build/BUILD/pybtex-0.24.0/build/lib
+ make -C docs html man
make: Entering directory '/builddir/build/BUILD/pybtex-0.24.0/docs'
sphinx-build -b html -d build/doctrees   source build/html
Running Sphinx v5.1.1
making output directory... done

Exception occurred:
  File "/usr/lib/python3.11/site-packages/pygments/styles/__init__.py", line 84, in get_style_by_name
    raise ClassNotFound("Could not find style module %r" % mod +
pygments.util.ClassNotFound: Could not find style module 'pybtex'.

The extended traceback is:

# Sphinx version: 5.1.1
# Python version: 3.11.0rc2 (CPython)
# Docutils version: 0.18.1 release
# Jinja2 version: 3.0.3
# Last messages:

# Loaded extensions:
Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/pygments/styles/__init__.py", line 82, in get_style_by_name
    mod = __import__('pygments.styles.' + mod, None, None, [cls])
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'pygments.styles.pybtex'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/sphinx/cmd/build.py", line 272, in build_main
    app = Sphinx(args.sourcedir, args.confdir, args.outputdir,
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/sphinx/application.py", line 261, in __init__
    self._init_builder()
  File "/usr/lib/python3.11/site-packages/sphinx/application.py", line 333, in _init_builder
    self.builder.init()
  File "/usr/lib/python3.11/site-packages/sphinx/builders/html/__init__.py", line 249, in init
    self.init_highlighter()
  File "/usr/lib/python3.11/site-packages/sphinx/builders/html/__init__.py", line 312, in init_highlighter
    self.highlighter = PygmentsBridge('html', style)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/sphinx/highlighting.py", line 106, in __init__
    style = self.get_style(stylename)
            ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/sphinx/highlighting.py", line 123, in get_style
    return get_style_by_name(stylename)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/pygments/styles/__init__.py", line 84, in get_style_by_name
    raise ClassNotFound("Could not find style module %r" % mod +
pygments.util.ClassNotFound: Could not find style module 'pybtex'.


Version-Release number of selected component (if applicable):
0.24.0-7 

How reproducible:
Always

Steps to Reproduce:
Run mockbuild:

mock -r fedora-rawhide-x86_64 --addrepo=https://download.copr.fedorainfracloud.org/results/ksurma/pygments-2.13/fedora-rawhide-x86_64/ --no-clean python-pybtex-0.24.0-7.fc38.src.rpm

Comment 1 Jerry James 2022-09-16 17:04:25 UTC
I think the issue is that docs/pybtex_doctools/__init__.py uses the pkg_resources way of registering entry points.  I'm not sure how to fix it either, but I'll poke at it for a bit and see if I get lucky. :-)

Comment 2 Jerry James 2022-09-16 17:27:13 UTC
If I am reading the importlib.metadata code correctly, it is no longer possible to register entry points in code.  It reads entry_points.txt every time it is asked for a list of entry points, and what you get is an exact representation of what is in that file.  Right?  So the only way pybtex is going to build documentation successfully is if its pygments entry points appear in entry_points.txt.

On the other hand, this is code that is only executed during documentation build.  It is not installed, nor referenced when pybtex is actually used, so we don't want phantom entries to show up in the installed entry_points.txt.

That leads me to this version of %build.

```
%build
%pyproject_wheel

# Build documentation
# Cheating, hacky workaround for pygments 2.13.
# I cannot be responsible for damage to your eyes from seeing the following.
cp -p pybtex.egg-info/entry_points.txt pybtex.egg-info/entry_points.txt.orig
cat >> pybtex.egg-info/entry_points.txt << EOF

[pygments.styles]
pybtex = pybtex_doctools.pygments:PybtexStyle

[pygments.lexers]
bibtex-pybtex = pybtex_doctools.pygments:BibTeXLexer
bst-pybtex = pybtex_doctools.pygments:BSTLexer
EOF

PYTHONPATH=$PWD:%{pyproject_build_lib} make -C docs html man
rm -f docs/build/html/.buildinfo

# Undo the cheating, hacky workaround for pygments 2.13.
mv pybtex.egg-info/entry_points.txt.orig pybtex.egg-info/entry_points.txt
```

That actually works.  I get good builds of the python-pybtex package with both Rawhide and your pygments 2.13 repository.  Just looking at that makes me want to go wash my brain off ... but it does work.

I'm open to suggestions for a better approach.

Comment 3 Karolina Surma 2022-09-20 13:29:42 UTC
Thank you for investigating the issue so promptly! I now understand where the problem comes from. It's tricky.

We discussed this approach and looked for other possibilities with lbalhar today. It seems your way is currently the way to achieve the goal.
If upstream defined entry points differently and got them to the distribution, we could grab them, but as you pointed out, having phantom entries in the installed distribution is not what we want.

I guess that strictly speaking, the backup and restore of the unpacked egg-info/entry_points.txt is not necessary at this point of the build process.
The wheel with the dist-info/ metadata is already stored in pyproject-wheeldir/ and the %install macros use only that built wheel to proceed, anything else is discarded.

For the record, another option would be dropping the docs altogether.

Comment 4 Jerry James 2022-09-20 14:47:29 UTC
(In reply to Karolina Surma from comment #3)
> We discussed this approach and looked for other possibilities with lbalhar
> today. It seems your way is currently the way to achieve the goal.
> If upstream defined entry points differently and got them to the
> distribution, we could grab them, but as you pointed out, having phantom
> entries in the installed distribution is not what we want.

Okay, I will clean that up a little and commit.

> I guess that strictly speaking, the backup and restore of the unpacked
> egg-info/entry_points.txt is not necessary at this point of the build
> process.
> The wheel with the dist-info/ metadata is already stored in
> pyproject-wheeldir/ and the %install macros use only that built wheel to
> proceed, anything else is discarded.

Ah, good.  Okay, that simplifies things slightly.

> For the record, another option would be dropping the docs altogether.

Yes, but I myself frequently refer to offline documentation.  I'm sure I'm not the only one who does that, so I think it is worthwhile to make a reasonable effort to produce such documentation.

There comes a point where the effort is just too much, of course, but I don't think we're there yet with this package.

Comment 5 Fedora Update System 2022-09-20 14:58:21 UTC
FEDORA-2022-6d0c9bff39 has been submitted as an update to Fedora 38. https://bodhi.fedoraproject.org/updates/FEDORA-2022-6d0c9bff39

Comment 6 Fedora Update System 2022-09-20 15:16:18 UTC
FEDORA-2022-6d0c9bff39 has been pushed to the Fedora 38 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.