Bug 1929101 - Retire "python-mysql" package in favor of "python-mysqlclient" PyPI package [NEEDINFO]
Summary: Retire "python-mysql" package in favor of "python-mysqlclient" PyPI package
Keywords:
Status: ASSIGNED
Alias: None
Product: Fedora
Classification: Fedora
Component: python-mysql
Version: 37
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Michal Schorm
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 1978350 2152628 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-02-16 08:19 UTC by Fabian Affolter
Modified: 2022-12-25 02:54 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Type: Bug
mschorm: needinfo? (mail)
quantum.analyst: needinfo? (mschorm)


Attachments (Terms of Use)

Description Fabian Affolter 2021-02-16 08:19:54 UTC
python-mysql is named differently than the PyPI package. This lead to an additional packages python-mysqlclient. Now those packages conflict.


Looks like that python-mysql should be retired and properly replaced by python-mysqlclient.

Comment 1 Michal Schorm 2021-02-22 11:47:50 UTC
Hello,
makes sense to me.

---

I can see several packages which require 'python-mysql'.

The first step should IMO be moving them to depend on 'python-mysqlclient instead'

Called
  $ grep -r -i "python3-mysql" ./
on top of extracted
  http://src.fedoraproject.org/repo/rpm-specs-latest.tar.xz

Found:
  ./trytond.spec:Requires:       python3-mysql
  ./python-sqlobject.spec:Requires:       python3-mysql
  ./pagure.spec:Recommends:         ((python3-mysql or python3-PyMySQL) if mysql-server)
  ./mysql-connector-python.spec:%{?python_provide:%python_provide python3-mysql-connector}
  ./holland.spec:Requires: python3-mysql

Would you begin with creating PRs against those packages and testing that they builds with your package ?

Comment 2 Fabian Affolter 2021-03-02 15:50:40 UTC
At least holland and trytond are out-dated. Let's see what will happen with the PRs.

Comment 3 Michal Schorm 2021-05-02 00:00:50 UTC
Package "holland":
  there is another occurence that has to be fixed:
  https://src.fedoraproject.org/rpms/holland/blob/rawhide/f/holland.spec#_108

Package "python-sqlobject"
  The PR is intact: https://src.fedoraproject.org/rpms/python-sqlobject/pull-request/1
  I sent a message to the Fedora devel list and the maintainer himself in hope he will respond
  https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/T7C5CJT4GI7JU667STN5X7VCLAIR7R3X/

Package "trytond"
  The PR is instact: https://src.fedoraproject.org/rpms/trytond/pull-request/3
  I sent a message to the Fedora devel list and the maintainer himself in hope he will respond
  https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/RHR3QTUKKL6RGHCVSNTM5DZ3DFOOVKQL/

After that, I believe, we can start the package retirement process for the "python-mysql" package.

Comment 4 Michal Schorm 2021-05-02 11:14:38 UTC
Package "holland"
  I created the PR:
  https://src.fedoraproject.org/rpms/holland/pull-request/8

The attempt to reach the maintainers of "python-sqlobject" and "trytond" was an immediate success.
PRs to both are merged now.

Comment 5 Michal Schorm 2021-05-10 22:28:37 UTC
The PR against the package "holland" has been merged.

---

There are no more result for the following search:
  grep -r -i "python3-mysql" ./ | grep -v -e "python3-mysqlclient" -e "python3-mysql-connector" -e "python-mysql.spec"
inside of the extracted:
  http://src.fedoraproject.org/repo/rpm-specs-latest.tar.xz

---

Let's begin with the package retirement process:
  https://fedoraproject.org/wiki/How_to_remove_a_package_at_end_of_life

Step 1/
  If the package is being replaced by some other package, ensure that the Obsoletes/Provides tags are properly set by the new package

---

Please implement / verify that the 'python-mysqlclient' correctly obsoletes the current and older versions of the "python-mysql" package, and that is replaces (provides) it:
  https://docs.fedoraproject.org/en-US/packaging-guidelines/#renaming-or-replacing-existing-packages

Please link the PR URL(s) or the URL(s) of the exact lines of code which resolves Step (1) to a new comment to this BZ ticket.

Only then I will proceed with next steps in the "python-mysql" package GIT.

Comment 6 Ben Cotton 2021-08-10 12:51:48 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 35 development cycle.
Changing version to 35.

Comment 7 Fabian Affolter 2021-08-26 19:27:19 UTC
*** Bug 1978350 has been marked as a duplicate of this bug. ***

Comment 8 Ben Cotton 2022-11-29 16:52:08 UTC
This message is a reminder that Fedora Linux 35 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora Linux 35 on 2022-12-13.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
'version' of '35'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, change the 'version' 
to a later Fedora Linux version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora Linux 35 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora Linux, you are encouraged to change the 'version' to a later version
prior to this bug being closed.

Comment 9 Steve Bonneville 2022-12-12 17:33:31 UTC
This still affects Fedora 36 and 37, and I'm concerned this will auto-close tomorrow....

Comment 10 Richard Shaw 2022-12-12 18:18:18 UTC
I updated the release for now, but are you saying this does not affect rawhide? Has the package been retired?

Also, catching up here on the history. All packages referencing the old name should indeed be updated, but the new package could add a "Provides: ..." to not break things for now.

Comment 11 Steve Bonneville 2022-12-12 20:57:47 UTC
I don't know if it affects rawhide or not, I didn't check rawhide.  I hit it when upgrading a Fedora 35 system to Fedora 37, and a package wanted python3-mysqlclient as a dependency, but python3-mysql was still installed and was causing conflicts.  Both python3-mysql and python3-mysqlclient are included in the distribution for both Fedora 36 and 37.

Comment 12 Richard Shaw 2022-12-13 03:54:55 UTC
In that case, python3-mysqlclient should add an Obsoletes:/Provides: to provide an upgrade path. 

See https://docs.fedoraproject.org/en-US/packaging-guidelines/#renaming-or-replacing-existing-packages for details.

Comment 13 stan 2022-12-13 17:00:54 UTC
I ran into this problem today during a dnf update.  The conflict did not allow several packages to update.  Fortunately, running
dnf swap python3-mysql python3-mysqlclient
worked a treat, and all the updates then succeeded.

Comment 14 Chris Caudle 2022-12-15 14:06:19 UTC
*** Bug 2152628 has been marked as a duplicate of this bug. ***

Comment 15 Chris Caudle 2022-12-15 14:10:04 UTC
I just marked 2152628 as a duplicate of this.  I filed 2152628 this week against python3-mysqlclient, since that package needs to change to flag that it obsoletes python3-mysql.  This bug was filed against python3-mysql which is why I overlooked the existing ticket.  Just hit it this week because the new MythTV build from RPMFusion changed dependency from p3-mysql to p3-mysqlclient, and now the upgrade won't progress because of the conflict between the two packages.

Comment 16 Patrick Mansfield 2022-12-19 00:57:26 UTC
(In reply to Steve Bonneville from comment #11)
> I don't know if it affects rawhide or not, I didn't check rawhide.  I hit it
> when upgrading a Fedora 35 system to Fedora 37, and a package wanted
> python3-mysqlclient as a dependency, but python3-mysql was still installed
> and was causing conflicts.  Both python3-mysql and python3-mysqlclient are
> included in the distribution for both Fedora 36 and 37.

Steve or anyone else - were you able to workaround the above?

I'm trying a dnf system-upgrade from 35 to 37 and it's failing.

I see workarounds if you're already on 36 or 37, but nothing for the system-upgrade case.

Comment 17 Edgar Hoch 2022-12-19 01:31:25 UTC
Note:

python3-mysql is listed in comps.xml for Fedora 37 in group "mysql" as a default package! So the wrong package is installed by default, instead of python3-mysqlclient.

  <group>
    <id>mysql</id>
    <name>MariaDB (MySQL) Database</name>
    <description>This package group contains packages useful for use with MariaDB (MySQL).</description>
    <packagelist>
      ...
      <packagereq type="default">python3-mysql</packagereq>
      ...
    </packagelist>
  </group>


Also note: Simple exchange of python3-mysql by python3-mysqlclient doesn't work, because other packages still require python3-mysql.
python3-mysqlclient should obsolete python3-mysql and provide python3-mysql itself.

# LANG=C dnf swap python3-mysql python3-mysqlclient
Last metadata expiration check: 1:05:17 ago on Mon Dec 19 01:19:01 2022.
Error: 
 Problem: problem with installed package python3-biopython-1.79-4.fc37.x86_64
  - package python3-biopython-1.79-4.fc37.x86_64 requires python3-mysql, but none of the providers can be installed
  - conflicting requests

Comment 18 Edgar Hoch 2022-12-19 01:46:07 UTC
Temporary workaround for upgrade:

- Remove package python3-mysql prior to upgrade, and note which packages have been removed as dependencies.
- Run upgrade.
- Add python3-mysqlclient later manually (dnf install python3-mysqlclient).
- Add packages that was removed as dependencies. Note that some of these packages may not install again because of missing dependency on python3-mysql.

Temporary workaround for fresh installation using kickstart:

- If package group @mysql is included in the package list, exclude package package python3-mysql (add a line "-python3-mysql").

Comment 19 Patrick Mansfield 2022-12-19 02:57:15 UTC
(In reply to Edgar Hoch from comment #18)
> Temporary workaround for upgrade:
> 
> - Remove package python3-mysql prior to upgrade, and note which packages
> have been removed as dependencies.
> - Run upgrade.
> - Add python3-mysqlclient later manually (dnf install python3-mysqlclient).
> - Add packages that was removed as dependencies. Note that some of these packages may not install again because of missing dependency on python3-mysql.

I'm concerned that some of the removed packages config files and/or database values will get screwed up - really mythtv here again. At least the main program does not need to be removed (mythtvbackend).

Is there a way to use dnf shell and "swap" the packages as part of the dnf system-upgrade, similar to the workaround on already upgraded systems?

Comment 20 Elliott Sales de Andrade 2022-12-19 10:03:00 UTC
Using leafdrop, I checked what requires both:

$ ./leafdrop python-mysql
python3-mysql is Required by:
- python3-ara+mysql
- python3-biopython
- python3-sqlalchemy+mysql
python3-mysql is BuildRequired by:
- python-biopython
- python-django
- python-pytest-services

$ ./leafdrop python-mysqlclient
python3-mysqlclient is Required by:
- holland-mysql
- holland-xtrabackup
- python3-ara+mysql
- python3-sqlalchemy+mysql
python3-mysqlclient is BuildRequired by:
- python-django
- python-pytest-services

One can see that there are packages that "require" both. This is because they use the Python dependency generator and depend on the virtual package name which can be fulfilled by either one. Only python-biopython explicitly depends on python-mysql. In the interest of preserving its dependencies, I have opened https://src.fedoraproject.org/rpms/python-mysqlclient/pull-request/2 to add Obsoletes and Provides.

Secondly, I looked at the change log for mysqlclient https://github.com/PyMySQL/mysqlclient/blob/78caa9ed05d8c27a27abe5eafd96d782279f1fca/HISTORY.rst; the major break of v2 was to drop support for Python 2 and Django 1.11. Neither of these are available in stable Fedora, so it should be okay to have the newer python-mysqlclient replace python-mysql.

If nothing happens on the PR by tomorrow, I will go ahead and build it myself.

Comment 21 Elliott Sales de Andrade 2022-12-19 10:22:25 UTC
Also opened https://src.fedoraproject.org/rpms/python-biopython/pull-request/3 so that python-biopython will stop depending on the old package.

Comment 22 Edgar Hoch 2022-12-19 10:24:06 UTC
I have created bug 2154807 to request rebuild of python-biopython with new dependency.

Comment 23 Elliott Sales de Andrade 2022-12-24 23:49:05 UTC
Well, tomorrow turned in next weekend, but I have merged the PR and am building now. An update should appear shortly.

Comment 24 Elliott Sales de Andrade 2022-12-25 00:03:25 UTC
Updates are now pending, and should be in testing at the next compose.

Rawhide/F38: https://bodhi.fedoraproject.org/updates/FEDORA-2022-13aa6fbdb1
F37: https://bodhi.fedoraproject.org/updates/FEDORA-2022-713080ea38
F36: https://bodhi.fedoraproject.org/updates/FEDORA-2022-dbbe1b44c0

There may be some delay for it to get to stable as it will need to go through testing first.

@mschorm you can retire this python-mysql package in Rawhide already as that build is stable, though perhaps you may want to request co-maintainership on python-mysqlclient as well.

Comment 25 Gary Buhrmaster 2022-12-25 02:22:42 UTC
Would it be possible to rebuild the epel9 package with the provides/obsoletes changes?  That should allow closing of https://bugzilla.redhat.com/show_bug.cgi?id=2033144

Comment 26 Elliott Sales de Andrade 2022-12-25 02:54:11 UTC
Sorry, you will have to find an EPEL packager to do that for you.


Note You need to log in before you can comment on or make changes to this bug.