Bug 2177923

Forgot to mention that I've upgraded from Fedora 37 to 38.

Seeing the same behavior as in the original report. After upgrade to F38, viewing Jupyter notebooks fails with the error in comment 0. Installing python3-mistune with `--allowerasing` solves it, so I wonder if the python-nbconvert spec file needs to be explicit about the package dependency?

I'll also follow up with the python-mistune and python-mistune08 maintainers to see if we can resolve it in those packages, which is probably the better fix.

nbconvert requires python3.11dist(mistune).

In https://src.fedoraproject.org/rpms/python-nbconvert/c/2f320b163c233a94806d61b84be1a8edf8863f98?branch=rawhide the version of the mistune dependency was stripped and probably should have not been.

The original requirement upstream is mistune>=2.0.3,<3 which is satisfiable in Fedora 37+. We need to fix this by:

--- a/python-nbconvert.spec
+++ b/python-nbconvert.spec
@@ -63,7 +63,6 @@ sed -i '/"pytest-cov",/d' pyproject.toml
 sed -i '/"pytest-dependency",/d' pyproject.toml
 sed -i '/pyppeteer/d' pyproject.toml
 sed -i 's/"sphinx==.*"/"sphinx"/' pyproject.toml
-sed -i 's/"mistune>=.*"/"mistune"/' pyproject.toml



About mistune08, it is required by:

$ comm -23 <(repoquery -q --repo=fedora --releasever=38 --whatrequires python3-mistune08) <(repoquery -q --repo=fedora --releasever=38 --whatrequires python3-mistune)
python3-m2r-0:0.3.1-2.fc38.noarch

So it cannot be removed yet. I am unsure what happens on Fedora 37->38 upgrade if we do the change in nbconvert without messing with obsoleted etc., but that can be probably tested. The change needs to land anyhow.

https://src.fedoraproject.org/rpms/python-nbconvert/pull-request/19
https://src.fedoraproject.org/rpms/python-nbconvert/pull-request/20

Will test the distro upgrade via https://copr.fedorainfracloud.org/coprs/churchyard/mistune-upgrade-test/

$ podman run --rm -ti fedora:37 /usr/bin/bash
[root@3afafe3a35e3 /]# dnf --nodocs --setopt=install_weak_deps=False --disablerepo='*modul*' install python3-{notebook,nbconvert,mistune08} 'dnf-command(copr)' 'dnf-command(system-upgrade)'
...
[root@3afafe3a35e3 /]# rpm -q python3-{notebook,nbconvert,mistune08}
python3-notebook-6.4.12-1.fc37.noarch
python3-nbconvert-6.4.5-4.fc37.noarch
python3-mistune08-0.8.4-7.fc37.noarch
[root@3afafe3a35e3 /]# curl https://copr.fedorainfracloud.org/coprs/churchyard/mistune-upgrade-test/repo/fedora-38/churchyard-mistune-upgrade-test-fedora-38.repo > /etc/yum.repos.d/copr.repo
...
[root@3afafe3a35e3 /]# dnf --setopt=install_weak_deps=False --disablerepo='*modul*' system-upgrade download --releasever=38 --exclude 'python3-nbconvert-7.2.5-3.fc38'
...
Error: 
 Problem: package python3-nbconvert-7.2.5-5.fc38.noarch requires (python3.11dist(mistune) < 3~~ with python3.11dist(mistune) >= 2.0.3), but none of the providers can be installed
  - package python3-mistune08-0.8.4-8.fc38.noarch conflicts with python3-mistune provided by python3-mistune-2.0.4-2.fc38.noarch
  - problem with installed package python3-nbconvert-6.4.5-4.fc37.noarch
  - problem with installed package python3-mistune08-0.8.4-7.fc37.noarch
  - python3-nbconvert-6.4.5-4.fc37.noarch does not belong to a distupgrade repository
  - python3-mistune08-0.8.4-7.fc37.noarch does not belong to a distupgrade repository
(try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages)



------------------------

So, this won't be that easy. Next step: stick in obsoletes for python3-mistune08 from Fedora 37 into python3-mistune and see if that does not break m2r.

Added:

  Obsoletes:      python3-mistune08 < 0.8.4-8

To python3-mistune.

[root@3afafe3a35e3 /]# dnf --setopt=install_weak_deps=False --disablerepo='*modul*' system-upgrade download --releasever=38 --exclude 'python3-nbconvert-7.2.5-3.fc38' --exclude 'python3-mistune-2.0.4-2.fc38'
...
Installing:
 python3-mistune                   noarch 2.0.4-3.fc38           copr:copr.fedorainfracloud.org:churchyard:mistune-upgrade-test
                                                                           68 k
     replacing  python3-mistune08.noarch 0.8.4-7.fc37
...


So this clearly works for the notebook case, but let's try with m2r:


[root@3afafe3a35e3 /]# dnf --setopt=install_weak_deps=False --disablerepo='*modul*' remove python3-nbconvert
...
[root@3afafe3a35e3 /]# dnf --setopt=install_weak_deps=False --disablerepo='*modul*' install python3-m2r
...
[root@3afafe3a35e3 /]# rpm -q python3-{m2r,mistune08}
python3-m2r-0.2.1-12.20190604git66f4a5a.fc37.noarch
python3-mistune08-0.8.4-7.fc37.noarch
[root@3afafe3a35e3 /]# dnf --setopt=install_weak_deps=False --disablerepo='*modul*' system-upgrade download --releasever=38 --exclude 'python3-nbconvert-7.2.5-3.fc38' --exclude 'python3-mistune-2.0.4-2.fc38'
...
Upgrading:
...
 python3-mistune08                 noarch 0.8.4-8.fc38           fedora    40 k

Still good.

==========

Obviously, this case will break:

[root@3afafe3a35e3 /]# rpm -q python3-{notebook,nbconvert,m2r,mistune08}
python3-notebook-6.4.12-1.fc37.noarch
python3-nbconvert-6.4.5-4.fc37.noarch
python3-m2r-0.2.1-12.20190604git66f4a5a.fc37.noarch
python3-mistune08-0.8.4-7.fc37.noarch
[root@3afafe3a35e3 /]# dnf --setopt=install_weak_deps=False --disablerepo='*modul*' system-upgrade download --releasever=38 --exclude 'python3-nbconvert-7.2.5-3.fc38' --exclude 'python3-mistune-2.0.4-2.fc38'
...
Error: 
 Problem: package python3-mistune08-0.8.4-8.fc38.noarch conflicts with python3-mistune provided by python3-mistune-2.0.4-3.fc38.noarch
  - package python3-nbconvert-7.2.5-5.fc38.noarch requires (python3.11dist(mistune) < 3~~ with python3.11dist(mistune) >= 2.0.3), but none of the providers can be installed
  - package python3-m2r-0.3.1-2.fc38.noarch requires python3.11dist(mistune) < 2~~, but none of the providers can be installed
  - problem with installed package python3-nbconvert-6.4.5-4.fc37.noarch
  - problem with installed package python3-m2r-0.2.1-12.20190604git66f4a5a.fc37.noarch
  - python3-nbconvert-6.4.5-4.fc37.noarch does not belong to a distupgrade repository
  - python3-mistune08-0.8.4-7.fc37.noarch does not belong to a distupgrade repository
  - python3-m2r-0.2.1-12.20190604git66f4a5a.fc37.noarch does not belong to a distupgrade repository
  - package python3-mistune-2.0.4-2.fc38.noarch is filtered out by exclude filtering
(try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages)

But that is probably not fixable at all. When packages conflict, upgrades need to be done explcitly.

https://src.fedoraproject.org/rpms/python-mistune/pull-request/8
https://src.fedoraproject.org/rpms/python-mistune/pull-request/9

I am afraid that as long as the old version of nbconvert without the versioned mistune requirement is available in the GA repository, it might be preferred by dnf system-upgrade Hence, I'd like to put this to the Fedora repo instead of updates.

Upgrade path using your Copr works without issue for my case (python3-notebook).

Looks like m2r is abandoned upstream and the one package that requires it dropped that requirement in a recent release. I'll try a scratch build of that tomorrow and if it builds, I'll open a PR in dist-git. Then we should be able to retire mistune08 and m2r and avoid the unfixable scenario.

FTR:

$ repoquery -q --repo=fedora --releasever=38 --whatrequires python3-m2r
python3-sphinxcontrib-openapi-0:0.7.0-7.fc38.noarch

Also CCing nforro who maintains python-m2r

And nothing requires python3-sphinxcontrib-openapi, so we could retire and obsolete both, if desired.

https://github.com/sphinx-contrib/openapi/commit/160a126fdd3239717ccc8e3a99a01f791e390807 exists, and so does https://src.fedoraproject.org/rpms/python-sphinx-mdinclude

I made a mistake in my initial repoqury (not including source repository):

$ repoquery -q --repo=fedora{,-source} --releasever=38 --whatrequires python3-m2r
python-binary-memcached-0:0.31.1-2.fc38.src
python-pyrtlsdr-0:0.2.93-1.fc37.src
python-qdarkstyle-0:3.0.2-7.fc38.src
python-sphinxcontrib-openapi-0:0.7.0-7.fc38.src
python-usort-0:0.6.3-8.fc38.src
python3-sphinxcontrib-openapi-0:0.7.0-7.fc38.noarch
rebase-helper-0:0.28.0-1.fc38.src


I don't think we can remove all this that fast. I'll ship the fix as is -- and if we actually remove m2r or migrate it to mistune 2, we can amend.

https://koji.fedoraproject.org/koji/taskinfo?taskID=99579720
https://koji.fedoraproject.org/koji/taskinfo?taskID=99579740

FEDORA-2023-79bd5167bb has been submitted as an update to Fedora 39. https://bodhi.fedoraproject.org/updates/FEDORA-2023-79bd5167bb

FEDORA-2023-79bd5167bb has been pushed to the Fedora 39 stable repository.
If problem still persists, please make note of it in this bug report.

FEDORA-2023-617e4ab78a has been submitted as an update to Fedora 38. https://bodhi.fedoraproject.org/updates/FEDORA-2023-617e4ab78a

(In reply to Ben Cotton from comment #10)
> I'll try a scratch build of [python-sphinxcontrib-openapi] tomorrow and if it builds, I'll open a PR in dist-git.

Alas, it's a little more complicated. It looks like 0.8 adds an unpackaged dependency. I opened bug 2185034 for the python-sphinxcontrib-openapi maintainers, but it looks like we'll have to stick with what you've done for now. Thanks again!

Thanks for flagging this Ben, and Miro for fixing!

FEDORA-2023-617e4ab78a has been pushed to the Fedora 38 testing repository.

You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2023-617e4ab78a

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

+4 in https://pagure.io/fedora-qa/blocker-review/issue/1141 , marking accepted.

FEDORA-2023-617e4ab78a has been pushed to the Fedora 38 stable repository.
If problem still persists, please make note of it in this bug report.