Description of problem: Jupyter notebooks no long work. Version-Release number of selected component (if applicable): python3-nbconvert-7.2.5-3.fc38.noarch How reproducible: Every time Steps to Reproduce: 1. Run jupyter nbclassic 2. Create a notebook Actual results: 500 : Internal Server Error [E 2023-03-14 01:28:08.754 ServerApp] Uncaught exception GET /notebooks/Untitled.ipynb?kernel_name=python3 (127.0.0.1) HTTPServerRequest(protocol='http', host='localhost:8888', method='GET', uri='/notebooks/Untitled.ipynb?kernel_name=python3', version='HTTP/1.1', remote_ip='127.0.0.1') Traceback (most recent call last): File "/usr/lib64/python3.11/site-packages/tornado/web.py", line 1713, in _execute result = await result ^^^^^^^^^^^^ File "/usr/lib64/python3.11/site-packages/tornado/gen.py", line 782, in run yielded = self.gen.send(value) ^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/site-packages/nbclassic/notebook/handlers.py", line 101, in get self.write(self.render_template('notebook.html', ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/site-packages/jupyter_server/base/handlers.py", line 633, in render_template return template.render(**ns) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/site-packages/jinja2/environment.py", line 1291, in render self.environment.handle_exception() File "/usr/lib/python3.11/site-packages/jinja2/environment.py", line 925, in handle_exception raise rewrite_traceback_stack(source=source) File "/usr/lib/python3.11/site-packages/nbclassic/templates/notebook.html", line 1, in top-level template code {% extends "page.html" %} File "/usr/lib/python3.11/site-packages/nbclassic/templates/page.html", line 169, in top-level template code {% block header %} File "/usr/lib/python3.11/site-packages/nbclassic/templates/notebook.html", line 115, in block 'header' {% for exporter in get_frontend_exporters() %} ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/site-packages/nbclassic/notebook/handlers.py", line 29, in get_frontend_exporters from nbconvert.exporters.base import get_export_names, get_exporter File "/usr/lib/python3.11/site-packages/nbconvert/__init__.py", line 7, in <module> from .exporters import * File "/usr/lib/python3.11/site-packages/nbconvert/exporters/__init__.py", line 4, in <module> from .html import HTMLExporter File "/usr/lib/python3.11/site-packages/nbconvert/exporters/html.py", line 26, in <module> from nbconvert.filters.markdown_mistune import IPythonRenderer, MarkdownWithMath File "/usr/lib/python3.11/site-packages/nbconvert/filters/markdown_mistune.py", line 24, in <module> from mistune import PLUGINS, BlockParser, HTMLRenderer, InlineParser, Markdown ImportError: cannot import name 'PLUGINS' from 'mistune' (/usr/lib/python3.11/site-packages/mistune.py) [E 2023-03-14 01:28:08.761 NotebookApp] { "Host": "localhost:8888", "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8", "Referer": "http://localhost:8888/tree", "User-Agent": "Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/110.0" } [E 2023-03-14 01:28:08.761 NotebookApp] 500 GET /notebooks/Untitled.ipynb?kernel_name=python3 (0e4979b43457428ab394 Expected results: No errors Additional info: I have python3-mistune08-0.8.4-8.fc38.noarch installed. While installing python3-mistune-2.0.4-2.fc38.noarch seems to solve the problem, this shouldn't happen.
Forgot to mention that I've upgraded from Fedora 37 to 38.
Seeing the same behavior as in the original report. After upgrade to F38, viewing Jupyter notebooks fails with the error in comment 0. Installing python3-mistune with `--allowerasing` solves it, so I wonder if the python-nbconvert spec file needs to be explicit about the package dependency? I'll also follow up with the python-mistune and python-mistune08 maintainers to see if we can resolve it in those packages, which is probably the better fix.
nbconvert requires python3.11dist(mistune). In https://src.fedoraproject.org/rpms/python-nbconvert/c/2f320b163c233a94806d61b84be1a8edf8863f98?branch=rawhide the version of the mistune dependency was stripped and probably should have not been. The original requirement upstream is mistune>=2.0.3,<3 which is satisfiable in Fedora 37+. We need to fix this by: --- a/python-nbconvert.spec +++ b/python-nbconvert.spec @@ -63,7 +63,6 @@ sed -i '/"pytest-cov",/d' pyproject.toml sed -i '/"pytest-dependency",/d' pyproject.toml sed -i '/pyppeteer/d' pyproject.toml sed -i 's/"sphinx==.*"/"sphinx"/' pyproject.toml -sed -i 's/"mistune>=.*"/"mistune"/' pyproject.toml About mistune08, it is required by: $ comm -23 <(repoquery -q --repo=fedora --releasever=38 --whatrequires python3-mistune08) <(repoquery -q --repo=fedora --releasever=38 --whatrequires python3-mistune) python3-m2r-0:0.3.1-2.fc38.noarch So it cannot be removed yet. I am unsure what happens on Fedora 37->38 upgrade if we do the change in nbconvert without messing with obsoleted etc., but that can be probably tested. The change needs to land anyhow.
https://src.fedoraproject.org/rpms/python-nbconvert/pull-request/19 https://src.fedoraproject.org/rpms/python-nbconvert/pull-request/20
Will test the distro upgrade via https://copr.fedorainfracloud.org/coprs/churchyard/mistune-upgrade-test/
$ podman run --rm -ti fedora:37 /usr/bin/bash [root@3afafe3a35e3 /]# dnf --nodocs --setopt=install_weak_deps=False --disablerepo='*modul*' install python3-{notebook,nbconvert,mistune08} 'dnf-command(copr)' 'dnf-command(system-upgrade)' ... [root@3afafe3a35e3 /]# rpm -q python3-{notebook,nbconvert,mistune08} python3-notebook-6.4.12-1.fc37.noarch python3-nbconvert-6.4.5-4.fc37.noarch python3-mistune08-0.8.4-7.fc37.noarch [root@3afafe3a35e3 /]# curl https://copr.fedorainfracloud.org/coprs/churchyard/mistune-upgrade-test/repo/fedora-38/churchyard-mistune-upgrade-test-fedora-38.repo > /etc/yum.repos.d/copr.repo ... [root@3afafe3a35e3 /]# dnf --setopt=install_weak_deps=False --disablerepo='*modul*' system-upgrade download --releasever=38 --exclude 'python3-nbconvert-7.2.5-3.fc38' ... Error: Problem: package python3-nbconvert-7.2.5-5.fc38.noarch requires (python3.11dist(mistune) < 3~~ with python3.11dist(mistune) >= 2.0.3), but none of the providers can be installed - package python3-mistune08-0.8.4-8.fc38.noarch conflicts with python3-mistune provided by python3-mistune-2.0.4-2.fc38.noarch - problem with installed package python3-nbconvert-6.4.5-4.fc37.noarch - problem with installed package python3-mistune08-0.8.4-7.fc37.noarch - python3-nbconvert-6.4.5-4.fc37.noarch does not belong to a distupgrade repository - python3-mistune08-0.8.4-7.fc37.noarch does not belong to a distupgrade repository (try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages) ------------------------ So, this won't be that easy. Next step: stick in obsoletes for python3-mistune08 from Fedora 37 into python3-mistune and see if that does not break m2r.
Added: Obsoletes: python3-mistune08 < 0.8.4-8 To python3-mistune. [root@3afafe3a35e3 /]# dnf --setopt=install_weak_deps=False --disablerepo='*modul*' system-upgrade download --releasever=38 --exclude 'python3-nbconvert-7.2.5-3.fc38' --exclude 'python3-mistune-2.0.4-2.fc38' ... Installing: python3-mistune noarch 2.0.4-3.fc38 copr:copr.fedorainfracloud.org:churchyard:mistune-upgrade-test 68 k replacing python3-mistune08.noarch 0.8.4-7.fc37 ... So this clearly works for the notebook case, but let's try with m2r: [root@3afafe3a35e3 /]# dnf --setopt=install_weak_deps=False --disablerepo='*modul*' remove python3-nbconvert ... [root@3afafe3a35e3 /]# dnf --setopt=install_weak_deps=False --disablerepo='*modul*' install python3-m2r ... [root@3afafe3a35e3 /]# rpm -q python3-{m2r,mistune08} python3-m2r-0.2.1-12.20190604git66f4a5a.fc37.noarch python3-mistune08-0.8.4-7.fc37.noarch [root@3afafe3a35e3 /]# dnf --setopt=install_weak_deps=False --disablerepo='*modul*' system-upgrade download --releasever=38 --exclude 'python3-nbconvert-7.2.5-3.fc38' --exclude 'python3-mistune-2.0.4-2.fc38' ... Upgrading: ... python3-mistune08 noarch 0.8.4-8.fc38 fedora 40 k Still good. ========== Obviously, this case will break: [root@3afafe3a35e3 /]# rpm -q python3-{notebook,nbconvert,m2r,mistune08} python3-notebook-6.4.12-1.fc37.noarch python3-nbconvert-6.4.5-4.fc37.noarch python3-m2r-0.2.1-12.20190604git66f4a5a.fc37.noarch python3-mistune08-0.8.4-7.fc37.noarch [root@3afafe3a35e3 /]# dnf --setopt=install_weak_deps=False --disablerepo='*modul*' system-upgrade download --releasever=38 --exclude 'python3-nbconvert-7.2.5-3.fc38' --exclude 'python3-mistune-2.0.4-2.fc38' ... Error: Problem: package python3-mistune08-0.8.4-8.fc38.noarch conflicts with python3-mistune provided by python3-mistune-2.0.4-3.fc38.noarch - package python3-nbconvert-7.2.5-5.fc38.noarch requires (python3.11dist(mistune) < 3~~ with python3.11dist(mistune) >= 2.0.3), but none of the providers can be installed - package python3-m2r-0.3.1-2.fc38.noarch requires python3.11dist(mistune) < 2~~, but none of the providers can be installed - problem with installed package python3-nbconvert-6.4.5-4.fc37.noarch - problem with installed package python3-m2r-0.2.1-12.20190604git66f4a5a.fc37.noarch - python3-nbconvert-6.4.5-4.fc37.noarch does not belong to a distupgrade repository - python3-mistune08-0.8.4-7.fc37.noarch does not belong to a distupgrade repository - python3-m2r-0.2.1-12.20190604git66f4a5a.fc37.noarch does not belong to a distupgrade repository - package python3-mistune-2.0.4-2.fc38.noarch is filtered out by exclude filtering (try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages) But that is probably not fixable at all. When packages conflict, upgrades need to be done explcitly.
https://src.fedoraproject.org/rpms/python-mistune/pull-request/8 https://src.fedoraproject.org/rpms/python-mistune/pull-request/9
I am afraid that as long as the old version of nbconvert without the versioned mistune requirement is available in the GA repository, it might be preferred by dnf system-upgrade Hence, I'd like to put this to the Fedora repo instead of updates.
Upgrade path using your Copr works without issue for my case (python3-notebook). Looks like m2r is abandoned upstream and the one package that requires it dropped that requirement in a recent release. I'll try a scratch build of that tomorrow and if it builds, I'll open a PR in dist-git. Then we should be able to retire mistune08 and m2r and avoid the unfixable scenario.
FTR: $ repoquery -q --repo=fedora --releasever=38 --whatrequires python3-m2r python3-sphinxcontrib-openapi-0:0.7.0-7.fc38.noarch Also CCing nforro who maintains python-m2r
And nothing requires python3-sphinxcontrib-openapi, so we could retire and obsolete both, if desired.
https://github.com/sphinx-contrib/openapi/commit/160a126fdd3239717ccc8e3a99a01f791e390807 exists, and so does https://src.fedoraproject.org/rpms/python-sphinx-mdinclude
I made a mistake in my initial repoqury (not including source repository): $ repoquery -q --repo=fedora{,-source} --releasever=38 --whatrequires python3-m2r python-binary-memcached-0:0.31.1-2.fc38.src python-pyrtlsdr-0:0.2.93-1.fc37.src python-qdarkstyle-0:3.0.2-7.fc38.src python-sphinxcontrib-openapi-0:0.7.0-7.fc38.src python-usort-0:0.6.3-8.fc38.src python3-sphinxcontrib-openapi-0:0.7.0-7.fc38.noarch rebase-helper-0:0.28.0-1.fc38.src I don't think we can remove all this that fast. I'll ship the fix as is -- and if we actually remove m2r or migrate it to mistune 2, we can amend.
https://koji.fedoraproject.org/koji/taskinfo?taskID=99579720 https://koji.fedoraproject.org/koji/taskinfo?taskID=99579740
FEDORA-2023-79bd5167bb has been submitted as an update to Fedora 39. https://bodhi.fedoraproject.org/updates/FEDORA-2023-79bd5167bb
FEDORA-2023-79bd5167bb has been pushed to the Fedora 39 stable repository. If problem still persists, please make note of it in this bug report.
FEDORA-2023-617e4ab78a has been submitted as an update to Fedora 38. https://bodhi.fedoraproject.org/updates/FEDORA-2023-617e4ab78a
(In reply to Ben Cotton from comment #10) > I'll try a scratch build of [python-sphinxcontrib-openapi] tomorrow and if it builds, I'll open a PR in dist-git. Alas, it's a little more complicated. It looks like 0.8 adds an unpackaged dependency. I opened bug 2185034 for the python-sphinxcontrib-openapi maintainers, but it looks like we'll have to stick with what you've done for now. Thanks again!
Thanks for flagging this Ben, and Miro for fixing!
FEDORA-2023-617e4ab78a has been pushed to the Fedora 38 testing repository. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2023-617e4ab78a See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
+4 in https://pagure.io/fedora-qa/blocker-review/issue/1141 , marking accepted.
FEDORA-2023-617e4ab78a has been pushed to the Fedora 38 stable repository. If problem still persists, please make note of it in this bug report.