Bug 1929101 - Retire "python-mysql" package in favor of "python-mysqlclient" PyPI package
Summary: Retire "python-mysql" package in favor of "python-mysqlclient" PyPI package
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: python-mysql
Version: 40
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Michal Schorm
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 1978350 2152628 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-02-16 08:19 UTC by Fabian Affolter
Modified: 2024-04-08 07:44 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2024-04-08 07:44:25 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Fabian Affolter 2021-02-16 08:19:54 UTC
python-mysql is named differently than the PyPI package. This lead to an additional packages python-mysqlclient. Now those packages conflict.


Looks like that python-mysql should be retired and properly replaced by python-mysqlclient.

Comment 1 Michal Schorm 2021-02-22 11:47:50 UTC
Hello,
makes sense to me.

---

I can see several packages which require 'python-mysql'.

The first step should IMO be moving them to depend on 'python-mysqlclient instead'

Called
  $ grep -r -i "python3-mysql" ./
on top of extracted
  http://src.fedoraproject.org/repo/rpm-specs-latest.tar.xz

Found:
  ./trytond.spec:Requires:       python3-mysql
  ./python-sqlobject.spec:Requires:       python3-mysql
  ./pagure.spec:Recommends:         ((python3-mysql or python3-PyMySQL) if mysql-server)
  ./mysql-connector-python.spec:%{?python_provide:%python_provide python3-mysql-connector}
  ./holland.spec:Requires: python3-mysql

Would you begin with creating PRs against those packages and testing that they builds with your package ?

Comment 2 Fabian Affolter 2021-03-02 15:50:40 UTC
At least holland and trytond are out-dated. Let's see what will happen with the PRs.

Comment 3 Michal Schorm 2021-05-02 00:00:50 UTC
Package "holland":
  there is another occurence that has to be fixed:
  https://src.fedoraproject.org/rpms/holland/blob/rawhide/f/holland.spec#_108

Package "python-sqlobject"
  The PR is intact: https://src.fedoraproject.org/rpms/python-sqlobject/pull-request/1
  I sent a message to the Fedora devel list and the maintainer himself in hope he will respond
  https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/T7C5CJT4GI7JU667STN5X7VCLAIR7R3X/

Package "trytond"
  The PR is instact: https://src.fedoraproject.org/rpms/trytond/pull-request/3
  I sent a message to the Fedora devel list and the maintainer himself in hope he will respond
  https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/RHR3QTUKKL6RGHCVSNTM5DZ3DFOOVKQL/

After that, I believe, we can start the package retirement process for the "python-mysql" package.

Comment 4 Michal Schorm 2021-05-02 11:14:38 UTC
Package "holland"
  I created the PR:
  https://src.fedoraproject.org/rpms/holland/pull-request/8

The attempt to reach the maintainers of "python-sqlobject" and "trytond" was an immediate success.
PRs to both are merged now.

Comment 5 Michal Schorm 2021-05-10 22:28:37 UTC
The PR against the package "holland" has been merged.

---

There are no more result for the following search:
  grep -r -i "python3-mysql" ./ | grep -v -e "python3-mysqlclient" -e "python3-mysql-connector" -e "python-mysql.spec"
inside of the extracted:
  http://src.fedoraproject.org/repo/rpm-specs-latest.tar.xz

---

Let's begin with the package retirement process:
  https://fedoraproject.org/wiki/How_to_remove_a_package_at_end_of_life

Step 1/
  If the package is being replaced by some other package, ensure that the Obsoletes/Provides tags are properly set by the new package

---

Please implement / verify that the 'python-mysqlclient' correctly obsoletes the current and older versions of the "python-mysql" package, and that is replaces (provides) it:
  https://docs.fedoraproject.org/en-US/packaging-guidelines/#renaming-or-replacing-existing-packages

Please link the PR URL(s) or the URL(s) of the exact lines of code which resolves Step (1) to a new comment to this BZ ticket.

Only then I will proceed with next steps in the "python-mysql" package GIT.

Comment 6 Ben Cotton 2021-08-10 12:51:48 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 35 development cycle.
Changing version to 35.

Comment 7 Fabian Affolter 2021-08-26 19:27:19 UTC
*** Bug 1978350 has been marked as a duplicate of this bug. ***

Comment 8 Ben Cotton 2022-11-29 16:52:08 UTC
This message is a reminder that Fedora Linux 35 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora Linux 35 on 2022-12-13.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
'version' of '35'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, change the 'version' 
to a later Fedora Linux version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora Linux 35 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora Linux, you are encouraged to change the 'version' to a later version
prior to this bug being closed.

Comment 9 Steve Bonneville 2022-12-12 17:33:31 UTC
This still affects Fedora 36 and 37, and I'm concerned this will auto-close tomorrow....

Comment 10 Richard Shaw 2022-12-12 18:18:18 UTC
I updated the release for now, but are you saying this does not affect rawhide? Has the package been retired?

Also, catching up here on the history. All packages referencing the old name should indeed be updated, but the new package could add a "Provides: ..." to not break things for now.

Comment 11 Steve Bonneville 2022-12-12 20:57:47 UTC
I don't know if it affects rawhide or not, I didn't check rawhide.  I hit it when upgrading a Fedora 35 system to Fedora 37, and a package wanted python3-mysqlclient as a dependency, but python3-mysql was still installed and was causing conflicts.  Both python3-mysql and python3-mysqlclient are included in the distribution for both Fedora 36 and 37.

Comment 12 Richard Shaw 2022-12-13 03:54:55 UTC
In that case, python3-mysqlclient should add an Obsoletes:/Provides: to provide an upgrade path. 

See https://docs.fedoraproject.org/en-US/packaging-guidelines/#renaming-or-replacing-existing-packages for details.

Comment 13 stan 2022-12-13 17:00:54 UTC
I ran into this problem today during a dnf update.  The conflict did not allow several packages to update.  Fortunately, running
dnf swap python3-mysql python3-mysqlclient
worked a treat, and all the updates then succeeded.

Comment 14 Chris Caudle 2022-12-15 14:06:19 UTC
*** Bug 2152628 has been marked as a duplicate of this bug. ***

Comment 15 Chris Caudle 2022-12-15 14:10:04 UTC
I just marked 2152628 as a duplicate of this.  I filed 2152628 this week against python3-mysqlclient, since that package needs to change to flag that it obsoletes python3-mysql.  This bug was filed against python3-mysql which is why I overlooked the existing ticket.  Just hit it this week because the new MythTV build from RPMFusion changed dependency from p3-mysql to p3-mysqlclient, and now the upgrade won't progress because of the conflict between the two packages.

Comment 16 Patrick Mansfield 2022-12-19 00:57:26 UTC
(In reply to Steve Bonneville from comment #11)
> I don't know if it affects rawhide or not, I didn't check rawhide.  I hit it
> when upgrading a Fedora 35 system to Fedora 37, and a package wanted
> python3-mysqlclient as a dependency, but python3-mysql was still installed
> and was causing conflicts.  Both python3-mysql and python3-mysqlclient are
> included in the distribution for both Fedora 36 and 37.

Steve or anyone else - were you able to workaround the above?

I'm trying a dnf system-upgrade from 35 to 37 and it's failing.

I see workarounds if you're already on 36 or 37, but nothing for the system-upgrade case.

Comment 17 Edgar Hoch 2022-12-19 01:31:25 UTC
Note:

python3-mysql is listed in comps.xml for Fedora 37 in group "mysql" as a default package! So the wrong package is installed by default, instead of python3-mysqlclient.

  <group>
    <id>mysql</id>
    <name>MariaDB (MySQL) Database</name>
    <description>This package group contains packages useful for use with MariaDB (MySQL).</description>
    <packagelist>
      ...
      <packagereq type="default">python3-mysql</packagereq>
      ...
    </packagelist>
  </group>


Also note: Simple exchange of python3-mysql by python3-mysqlclient doesn't work, because other packages still require python3-mysql.
python3-mysqlclient should obsolete python3-mysql and provide python3-mysql itself.

# LANG=C dnf swap python3-mysql python3-mysqlclient
Last metadata expiration check: 1:05:17 ago on Mon Dec 19 01:19:01 2022.
Error: 
 Problem: problem with installed package python3-biopython-1.79-4.fc37.x86_64
  - package python3-biopython-1.79-4.fc37.x86_64 requires python3-mysql, but none of the providers can be installed
  - conflicting requests

Comment 18 Edgar Hoch 2022-12-19 01:46:07 UTC
Temporary workaround for upgrade:

- Remove package python3-mysql prior to upgrade, and note which packages have been removed as dependencies.
- Run upgrade.
- Add python3-mysqlclient later manually (dnf install python3-mysqlclient).
- Add packages that was removed as dependencies. Note that some of these packages may not install again because of missing dependency on python3-mysql.

Temporary workaround for fresh installation using kickstart:

- If package group @mysql is included in the package list, exclude package package python3-mysql (add a line "-python3-mysql").

Comment 19 Patrick Mansfield 2022-12-19 02:57:15 UTC
(In reply to Edgar Hoch from comment #18)
> Temporary workaround for upgrade:
> 
> - Remove package python3-mysql prior to upgrade, and note which packages
> have been removed as dependencies.
> - Run upgrade.
> - Add python3-mysqlclient later manually (dnf install python3-mysqlclient).
> - Add packages that was removed as dependencies. Note that some of these packages may not install again because of missing dependency on python3-mysql.

I'm concerned that some of the removed packages config files and/or database values will get screwed up - really mythtv here again. At least the main program does not need to be removed (mythtvbackend).

Is there a way to use dnf shell and "swap" the packages as part of the dnf system-upgrade, similar to the workaround on already upgraded systems?

Comment 20 Elliott Sales de Andrade 2022-12-19 10:03:00 UTC
Using leafdrop, I checked what requires both:

$ ./leafdrop python-mysql
python3-mysql is Required by:
- python3-ara+mysql
- python3-biopython
- python3-sqlalchemy+mysql
python3-mysql is BuildRequired by:
- python-biopython
- python-django
- python-pytest-services

$ ./leafdrop python-mysqlclient
python3-mysqlclient is Required by:
- holland-mysql
- holland-xtrabackup
- python3-ara+mysql
- python3-sqlalchemy+mysql
python3-mysqlclient is BuildRequired by:
- python-django
- python-pytest-services

One can see that there are packages that "require" both. This is because they use the Python dependency generator and depend on the virtual package name which can be fulfilled by either one. Only python-biopython explicitly depends on python-mysql. In the interest of preserving its dependencies, I have opened https://src.fedoraproject.org/rpms/python-mysqlclient/pull-request/2 to add Obsoletes and Provides.

Secondly, I looked at the change log for mysqlclient https://github.com/PyMySQL/mysqlclient/blob/78caa9ed05d8c27a27abe5eafd96d782279f1fca/HISTORY.rst; the major break of v2 was to drop support for Python 2 and Django 1.11. Neither of these are available in stable Fedora, so it should be okay to have the newer python-mysqlclient replace python-mysql.

If nothing happens on the PR by tomorrow, I will go ahead and build it myself.

Comment 21 Elliott Sales de Andrade 2022-12-19 10:22:25 UTC
Also opened https://src.fedoraproject.org/rpms/python-biopython/pull-request/3 so that python-biopython will stop depending on the old package.

Comment 22 Edgar Hoch 2022-12-19 10:24:06 UTC
I have created bug 2154807 to request rebuild of python-biopython with new dependency.

Comment 23 Elliott Sales de Andrade 2022-12-24 23:49:05 UTC
Well, tomorrow turned in next weekend, but I have merged the PR and am building now. An update should appear shortly.

Comment 24 Elliott Sales de Andrade 2022-12-25 00:03:25 UTC
Updates are now pending, and should be in testing at the next compose.

Rawhide/F38: https://bodhi.fedoraproject.org/updates/FEDORA-2022-13aa6fbdb1
F37: https://bodhi.fedoraproject.org/updates/FEDORA-2022-713080ea38
F36: https://bodhi.fedoraproject.org/updates/FEDORA-2022-dbbe1b44c0

There may be some delay for it to get to stable as it will need to go through testing first.

@mschorm you can retire this python-mysql package in Rawhide already as that build is stable, though perhaps you may want to request co-maintainership on python-mysqlclient as well.

Comment 25 Gary Buhrmaster 2022-12-25 02:22:42 UTC
Would it be possible to rebuild the epel9 package with the provides/obsoletes changes?  That should allow closing of https://bugzilla.redhat.com/show_bug.cgi?id=2033144

Comment 26 Elliott Sales de Andrade 2022-12-25 02:54:11 UTC
Sorry, you will have to find an EPEL packager to do that for you.

Comment 27 Aoife Moloney 2023-11-23 00:04:46 UTC
This message is a reminder that Fedora Linux 37 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora Linux 37 on 2023-12-05.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
'version' of '37'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, change the 'version' 
to a later Fedora Linux version. Note that the version field may be hidden.
Click the "Show advanced fields" button if you do not see it.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora Linux 37 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora Linux, you are encouraged to change the 'version' to a later version
prior to this bug being closed.

Comment 28 Elliott Sales de Andrade 2023-11-23 06:44:47 UTC
Even though python-mysqlclient has been Obsoleting this package since F37, this package has still not been retired.

Comment 29 Richard Shaw 2023-11-25 03:00:20 UTC
I might be able to do this as a proven packager but I'm reticent to do so. Perhaps a FESCo ticket?

Comment 30 Gary Buhrmaster 2023-11-25 03:07:49 UTC
(In reply to Elliott Sales de Andrade from comment #26)
> Sorry, you will have to find an EPEL packager to do that for you.

Would you add me to the package as a maintainer so that I can properly maintain the package versions, including EPEL, since you do not wish to?

Comment 31 Elliott Sales de Andrade 2023-11-25 04:16:00 UTC
I'm not admin on either package; I can't do that for you.

Comment 32 Michal Schorm 2023-12-18 13:40:07 UTC
Hello,

I haven't got the capacity for this (that was unfortunately rather obvious), but I also won't have in the upcoming ~3 months.

If there's anyone willing to resolve the current state, I'd be grateful.

Tell me what you need, I'll get you what I can - packager / admin access or whatever.
Anyone wanna become the admin / main admin of the package ?

Comment 33 Elliott Sales de Andrade 2023-12-19 02:31:44 UTC
The only thing you'd need to do is retire python-mysql; it's obsoleted by python-mysqlclient already, so there's no need for other admins: https://bugzilla.redhat.com/show_bug.cgi?id=1929101#c28 Or you can confirm that a proven packager such as myself or Richard can do it for you.

Comment 34 Michal Schorm 2023-12-19 03:40:49 UTC
I have executed:

---------------------------------------

# fedpkg retire "Retire "python-mysql" package in favor of "python-mysqlclient" PyPI package; rhbz#1929101"
rm '.gitignore'
rm 'python-mysql.spec'
rm 'sources'
[rawhide 492cf90] Retire python-mysql package in favor of python-mysqlclient PyPI package; rhbz#1929101
 4 files changed, 1 insertion(+), 218 deletions(-)
 delete mode 100644 .gitignore
 create mode 100644 dead.package
 delete mode 100644 python-mysql.spec
 delete mode 100644 sources
Enumerating objects: 4, done.
Counting objects: 100% (4/4), done.
Delta compression using up to 12 threads
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 364 bytes | 364.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
remote: Emitting a message to the fedora-messaging message bus.
remote: * Publishing information for 1 commits
remote: Sending to redis to log activity and send commit notification emails
remote: * Publishing information for 1 commits
remote:   - to fedora-message
remote: 2023-12-19 03:34:47,674 [WARNING] pagure.lib.notify: pagure is about to send a message that has no schemas: pagure.git.receive
To ssh://pkgs.fedoraproject.org/rpms/python-mysql
   a51416f..492cf90  rawhide -> rawhide
Could not execute retire: The following error occurred while disabling monitoring: Invalid or expired token. Please visit https://src.fedoraproject.org/settings#nav-api-tab to get or renew your API token.
For invalid or expired tokens please set a new token in your user configuration with:

	fedpkg set-distgit-token <token>

---------------------------------------

I've created a new temporary token with maximum privileges and tried again:

---------------------------------------

# fedpkg retire "Retire "python-mysql" package in favor of "python-mysqlclient" PyPI package; rhbz#1929101"
dead.package found, package or module is already retired. Will not remove files from git or overwrite existing dead.package file.
Monitoring of the project was successfully disabled.

---------------------------------------

Now I'm not sure, whether the retirement has been fully done :(
However since the first try failed on "disabling monitoring step" and the second try seems to picked up there, I hope it's fine.

---------------------------------------

Anything else needed ?

Comment 35 Michal Schorm 2023-12-19 03:43:29 UTC
What is supposed to happen to the Pagure members of the package?
Should I remove all and then myself, so the package would be without a maintainer ?

Comment 36 Fabio Valentini 2023-12-19 11:51:04 UTC
> Now I'm not sure, whether the retirement has been fully done :(

It should be. The API token is only needed for some nonessential "cleanup" tasks.

> What is supposed to happen to the Pagure members of the package?

Nothing. They will remain maintainers so long as there are any branches of Fedora that still ship the package.
When the last branch that has the package reaches EOL, all maintainers are removed from the package automatically.

> Should I remove all and then myself, so the package would be without a maintainer ?

No, unless you want to mark the package as "unmaintained" for stable branches as well.

> Anything else needed ?

Probably not. Retirements are processed once compose, so you will know tomorrow whether the package has been removed or not.

Comment 37 Aoife Moloney 2024-02-15 22:53:47 UTC
This bug appears to have been reported against 'rawhide' during the Fedora Linux 40 development cycle.
Changing version to 40.


Note You need to log in before you can comment on or make changes to this bug.