Bug 1599069 - Latest python3 updates broke dnf / yum.
Summary: Latest python3 updates broke dnf / yum.
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Fedora
Classification: Fedora
Component: dnf
Version: rawhide
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: ---
Assignee: rpm-software-management
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-07-08 13:06 UTC by David Hill
Modified: 2018-08-14 07:09 UTC (History)
12 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-08-14 07:09:51 UTC


Attachments (Terms of Use)

Description David Hill 2018-07-08 13:06:24 UTC
Description of problem:
Latest python3 updates broke dnf / yum.

[root@zappa site-packages]# yum update -y
Traceback (most recent call last):
  File "/usr/bin/yum", line 57, in <module>
    from dnf.cli import main
ModuleNotFoundError: No module named 'dnf'


[root@zappa log]# dnf
Traceback (most recent call last):
  File "/usr/bin/dnf", line 57, in <module>
    from dnf.cli import main
ModuleNotFoundError: No module named 'dnf'


[root@zappa log]# python2 /usr/bin/dnf > /dev/null 2>&1
[root@zappa log]# echo $?
0
[root@zappa log]# python3 /usr/bin/dnf 2>&1 
Traceback (most recent call last):
  File "/usr/bin/dnf", line 57, in <module>
    from dnf.cli import main
ModuleNotFoundError: No module named 'dnf'



Version-Release number of selected component (if applicable):


How reproducible:
Always

Steps to Reproduce:
1. Update python3 
2. Run DNF or YUM
3.

Actual results:
They fail with python3 but works with python2

Expected results:
Shouldn't fail

Additional info:

Comment 1 David Hill 2018-07-08 13:23:21 UTC
It's like if python3 would no longer be able to find the modules installed in the previous versions .  If  I create a symlink in /usr/lib/python3.7/site-packages, dnf "works" .

Comment 2 David Hill 2018-07-08 13:27:54 UTC
It works until I hit this:

[root@zappa usr]# dnf
Traceback (most recent call last):
  File "/usr/bin/dnf", line 57, in <module>
    from dnf.cli import main
  File "/usr/lib/python3.7/site-packages/dnf/__init__.py", line 31, in <module>
    import dnf.base
  File "/usr/lib/python3.7/site-packages/dnf/base.py", line 29, in <module>
    from dnf.comps import CompsQuery
  File "/usr/lib/python3.7/site-packages/dnf/comps.py", line 32, in <module>
    import dnf.util
  File "/usr/lib/python3.7/site-packages/dnf/util.py", line 29, in <module>
    import dnf.callback
  File "/usr/lib/python3.7/site-packages/dnf/callback.py", line 22, in <module>
    import dnf.yum.rpmtrans
  File "/usr/lib/python3.7/site-packages/dnf/yum/rpmtrans.py", line 25, in <module>
    import rpm
  File "/usr/lib64/python3.7/site-packages/rpm/__init__.py", line 38, in <module>
    from rpm._rpm import *
ModuleNotFoundError: No module named 'rpm._rpm'

Comment 3 David Hill 2018-07-08 13:35:55 UTC
[root@zappa rpm]# python3.7
Python 3.7.0 (default, Jun 28 2018, 07:54:44) 
[GCC 8.1.1 20180626 (Red Hat 8.1.1-3)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import rpm._rpm
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python3.7/site-packages/rpm/__init__.py", line 38, in <module>
    from rpm._rpm import *
ModuleNotFoundError: No module named 'rpm._rpm'
>>> quit
Use quit() or Ctrl-D (i.e. EOF) to exit
>>> 
[root@zappa rpm]# python3.6
Python 3.6.3 (default, Jan  4 2018, 16:40:53) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-16)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import rpm._rpm

Comment 4 David Hill 2018-07-08 13:37:01 UTC
So it works with python3.6 but not python3.7 .

Comment 5 David Hill 2018-07-08 13:52:48 UTC
Looks like the packages in fedora-devel are lacking the dnf/rpm/libdnf modules.   Is that my mistake ?

Comment 6 Miro Hrončok 2018-07-08 15:09:15 UTC
Could you please run:

$ rpm -q python3
$ rpm -q python37
$ rpm -q dnf
$ rpm -q python3-dnf

Comment 7 Miro Hrončok 2018-07-08 15:10:28 UTC
Also:

$ rpm -qf /usr/bin/dnf
$ rpm -qf /usr/bin/python3

And:

$ head -n2 /usr/bin/dnf

Comment 8 Miro Hrončok 2018-07-08 15:11:46 UTC
$ rpm -qf /usr/lib64/python3.7/site-packages/rpm/__init__.py
$ rpm -qf /usr/lib64/python3.7/site-packages/rpm/_rpm.cpython*

Comment 9 David Hill 2018-07-08 15:13:55 UTC
[root@zappa site-packages]# rpm -q python3
python3-3.7.0-1.fc29.x86_64
[root@zappa site-packages]# rpm -q python37
package python37 is not installed
[root@zappa site-packages]# rpm -q dnf
dnf-3.0.1-1.fc29.noarch
[root@zappa site-packages]# rpm -q python3-dnf
python3-dnf-3.0.1-1.fc29.noarch
[root@zappa site-packages]# rpm -qf /usr/bin/dnf
dnf-3.0.1-1.fc29.noarch
[root@zappa site-packages]# rpm -qf /usr/bin/python3
python3-3.7.0-1.fc29.x86_64
[root@zappa site-packages]# head -n2 /usr/bin/dnf
#!/usr/bin/python3
# The dnf executable script.
[root@zappa site-packages]# rpm -qf /usr/lib64/python3.7/site-packages/rpm/_rpm.cpython*
error: file /usr/lib64/python3.7/site-packages/rpm/_rpm.cpython*: No such file or directory
[root@zappa site-packages]# rpm -qf /usr/lib64/python3.7/site-packages/rpm/__init__.py
error: file /usr/lib64/python3.7/site-packages/rpm/__init__.py: No such file or directory

Comment 10 Miro Hrončok 2018-07-08 15:24:36 UTC
I wonder how you can get:

/usr/lib64/python3.7/site-packages/rpm/__init__.py No such file or directory

And at the same time:

  File "/usr/lib64/python3.7/site-packages/rpm/__init__.py", line 38, in <module>
    from rpm._rpm import *
ModuleNotFoundError: No module named 'rpm._rpm'


However python3-dnf-3.0.1-1.fc29.noarch.rpm requires "python(abi) = 3.6" and you have python3-3.7.0-1.fc29.x86_64. look like your deps are in broken state.

You need to update to dnf-3.0.2-1.fc29. Use the dnf-2 command to get updates.

$ sudo dnf-2 distrosync

Comment 11 Miro Hrončok 2018-07-08 15:30:03 UTC
Does this return anything?

$ dnf repoquery --duplicated

Comment 12 David Hill 2018-07-08 16:01:21 UTC
Sorry that's my bad.   I symlinked all the missing modules until I hit that final issue.

The real issue is this:

[root@zappa site-packages]# yum info
Traceback (most recent call last):
  File "/usr/bin/yum", line 57, in <module>
    from dnf.cli import main
ModuleNotFoundError: No module named 'dnf'

Comment 14 David Hill 2018-07-08 16:08:44 UTC
And if I change the symlink for python3 to python 3.6 instead of 3.7, I get this instead:

Traceback (most recent call last):
  File "yum", line 57, in <module>
    from dnf.cli import main
  File "/usr/lib/python3.6/site-packages/dnf/__init__.py", line 31, in <module>
    import dnf.base
  File "/usr/lib/python3.6/site-packages/dnf/base.py", line 32, in <module>
    from dnf.module.metadata_loader import ModuleMetadataLoader
  File "/usr/lib/python3.6/site-packages/dnf/module/metadata_loader.py", line 25, in <module>
    gi.require_version('Modulemd', '1.0')
AttributeError: module 'gi' has no attribute 'require_version'

Comment 15 Miro Hrončok 2018-07-08 16:09:21 UTC
Why do you change symlinks?

Comment 16 Miro Hrončok 2018-07-08 16:10:35 UTC
As long as you are changing where /usr/bin/python3 leads to and what directories packages install their files, I'm unable to help you.

Comment 17 David Hill 2018-07-08 17:45:30 UTC
Well I changed it to test it ...   because yum using python3 is broken now.   I didn't change the symlink of python3 to python3.7 myself .   Something's quite broken here.

Comment 18 David Hill 2018-07-08 18:04:13 UTC
Nevermind, I fixed the problem by downgrading python3 to 3.6.5 ... remember me to never use "--best --allowerasing" again ever.

Comment 19 Miro Hrončok 2018-07-08 18:54:29 UTC
If dnf --best --allowerasing pulls in python3 3.7.0 without updating dependent packages, that's definitively a bug.

Comment 20 John Ellson 2018-07-22 22:46:43 UTC
Could someone add more details about how to downgrade to python-3.6.5,  if thats the best fix you have?

I tried downgrading python3 and python3-libs,  but there are 40 or so dependent packages that would also need downgrading.   Do I have find each and every on in koji and dowgrade without help from dnf?

Comment 21 Miro Hrončok 2018-07-23 09:14:24 UTC
Why do you need to downgrade? What is the issue you get?

Comment 22 John Ellson 2018-07-23 12:31:57 UTC
# dnf update
Traceback (most recent call last):
  File "/usr/bin/dnf", line 57, in <module>
    from dmf.cli import main
ModuleNotFoundError: No module named 'dnf'

# rpm -qf /usr/bin/dnf
dnf-2.7.5-16.fc29.noarch

Comment 23 David Hill 2018-07-23 12:35:58 UTC
That's the issue you get when you upgrade to the latest python3 with --best --allowerasing isn't it ?

Comment 24 Miro Hrončok 2018-07-23 12:38:38 UTC
$ rpm -q dnf python3-dnf python3 python3-rpm
$ rpm -q --requires python3-dnf | grep python
$ rpm -q --requires python3-rpm | grep python

Comment 25 David Hill 2018-07-23 12:43:17 UTC
John has the issue currently ... I did rollback the python3 version to the previous one.

Comment 26 Miro Hrončok 2018-07-23 12:50:54 UTC
> I did rollback the python3 version to the previous one.

How? Could you please answer my queries?

Comment 27 Miro Hrončok 2018-07-23 12:51:54 UTC
Oh, sorry, I cannot read.

Comment 28 John Ellson 2018-07-23 13:19:16 UTC
# rpm -q dnf python3-dnf python3 python3-rpm
dnf-3.0.3-4.fc29.noarch
python3-dnf-3.0.3-4.fc29.noarch
python3-3.7.0-3.fc29.x86_64
python3-rpm-4.14.2-0.rc1.1.fc29.2.x86_64

# rpm -q --requires python3-dnf | grep python
/usr/bin/python3
python(abi) = 3.7
python3-gobject-base
python3-gpg
python3-hawkey >= 0.14.0
python3-iniparse
python3-libcomps >= 0.1.8
python3-libdnf
python3-libdnf >= 0.14.0
python3-librepo >= 1.9.0
python3-rpm >= 4.14.0
python3-smartcols >= 0.3.0

# rpm -q --requires python3-rpm | grep python
libpython3.7m.so.1.0()(64bit)
python(abi) = 3.7

Comment 29 John Ellson 2018-07-23 13:20:45 UTC
oops sorry,  thats not correct.  That was from a working system.  The broken system is different ....

Comment 30 John Ellson 2018-07-23 13:27:36 UTC
# rpm -q dnf python3-dnf python3 python3-rpm
dnf-2.7.5-16.fc29.noarch
python3-dnf-2-7-5-16.fc29.noarch
python3-3.6.5-4.fc29.x86_64
python3-3.7.0-3.fc29.x86_64
python3-rpm-4.14.1-10.rc1.1.fc29.2.x86_64

# rpm -q --requires python3-dnf | grep python
/usr/bin/python3
python(abi) = 3.6
python3-gobject-base
python3-gpg
python3-hawkey >= 0.11.1
python3-iniparse
python3-libcomps >= 0.1.8
python3-librepo >= 1.8.1
python3-rpm >= 4.14.0
python3-smartcols >= 0.3.0

# rpm -q --requires python3-rpm | grep python
libpython3.6m.so.1.0()(64bit)
python(abi) = 3.6

Comment 31 Miro Hrončok 2018-07-23 13:30:37 UTC
You have two python3 packages at the same time. That is not supposed to happen. Have you quit dnf in mid-transaction?

python3-3.6.5-4.fc29.x86_64
python3-3.7.0-3.fc29.x86_64


If you have python2-dnf installed you should be able to recover with /usr/bin/dnf-2. If not, you have to use rpm commands.

Comment 32 Daniel Mach 2018-08-14 07:09:51 UTC
This looks like an issue with the installed RPMs as mentinoned in comment#31.
I don't think there's anything to be fixed.


Note You need to log in before you can comment on or make changes to this bug.