Bug 1483342 - platform-python breaks dnf plugins
Summary: platform-python breaks dnf plugins
Keywords:
Status: CLOSED NEXTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: dnf
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Miro Hrončok
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 1483253 1483470 1483540 1483864 1483874 (view as bug list)
Depends On:
Blocks: TRACKER-bugs-affecting-libguestfs
TreeView+ depends on / blocked
 
Reported: 2017-08-20 18:14 UTC by Kevin Fenzi
Modified: 2017-08-23 11:21 UTC (History)
22 users (show)

Fixed In Version: dnf-2.6.3-4.fc28 dnf-2.6.3-4.fc27
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-08-22 13:38:16 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Kevin Fenzi 2017-08-20 18:14:59 UTC
When dnf is using system-python, builddep plugin isnt seen: 

# dnf builddep --help
No such command: builddep. Please use /bin/dnf --help
It could be a DNF plugin command, try: "dnf install 'dnf-command(builddep)'"
# dnf install 'dnf-command(builddep)'
Last metadata expiration check: 0:59:16 ago on Sun 20 Aug 2017 10:14:27 AM PDT.
Package dnf-plugins-core-2.1.3-2.fc27.noarch is already installed, skipping.
Dependencies resolved.
Nothing to do.
Complete!

Switching /bin/dnf to call python3 instead of system-python and everything works. 

There may be other plugins that fail, I haven't checked them all...

Comment 1 Miro Hrončok 2017-08-21 09:30:15 UTC
I was just about to report this. Dnf plugins are currently all broken (as per Modularity WG request "not to care about plugins" now).

The problems are:

 * our plugins are not packaged for platform-python
 * all the dnf(*) provides are provided for the python3 built plugins (or by packages that require that one), not platform-python built plugins

As a result, dnf system-upgrade is broken as well. Might be a blocker.

In order to fix this, we could either:

 * Declare Platform Python Broken. Apply contingency mechanism of https://fedoraproject.org/wiki/Changes/Platform_Python_Stack and break modularity

 * build dnf-plugins-core and -extras also for platform-python and switch how the virtual provides works. Note that we would also need to package any Python (Build)Requires needed. Will try to check those.

Comment 2 Miro Hrončok 2017-08-21 09:50:24 UTC
Not yet packaged for Platform Python Dependneices for plugins (only runtime) seem to be:

python3-kickstart
    python3-ordered-set
    python3-requests
        python3-chardet
        python3-idna
        python3-urllib3
            python3-cryptography
                python3-asn1crypto
                python3-cffi
                    python3-pycparser
                        python3-ply
            python3-pyOpenSSL
            python3-pysocks
python3-rpmconf
python3-dbus
python3-systemd
python3-pycurl
python3-tracer
    python3-argcomplete
    python3-beautifulsoup4
    python3-lxml
        python3-cssselect
        python3-html5lib
            python3-webencodings
    python3-psutil


For buildtime, there will be more. Not sure if this is good idea.

Comment 3 Miro Hrončok 2017-08-21 09:59:36 UTC
We could also switch dnf to use python3 on "normal" Fedora and only use platform-python in some branch that would be used in Platform. This would of course be a temporary fix because in modular Fedora, this would still be broken.

Comment 4 Petr Viktorin (pviktori) 2017-08-21 10:16:28 UTC
That would make a lot of sense to me.

Comment 5 Petr Pisar 2017-08-21 10:38:16 UTC
*** Bug 1483470 has been marked as a duplicate of this bug. ***

Comment 6 Miro Hrončok 2017-08-21 10:46:01 UTC
(In reply to Miro Hrončok from comment #3)
> We could also switch dnf to use python3 on "normal" Fedora and only use
> platform-python in some branch that would be used in Platform. This would of
> course be a temporary fix because in modular Fedora, this would still be
> broken.

This can by done in 2 ways:

1) turn off platform-python bcond in the spec

   (the platform-python-dnf package would then be "dead package" without any future updates)

2) add a bcond that switches what the dnf package and /usr/bin/dnf will require/use (platform-python or pythno3). A bit more complicated, but "cleaner".

Comment 7 Petr Viktorin (pviktori) 2017-08-21 11:34:34 UTC
*** Bug 1483540 has been marked as a duplicate of this bug. ***

Comment 8 Kamil Páral 2017-08-21 11:49:52 UTC
(In reply to Miro Hrončok from comment #1)
> Dnf plugins are currently all broken (as
> per Modularity WG request "not to care about plugins" now).

There are more WGs than just Modularity. Also, there are users.

Since Friday, all of our F27/F28 tasks crash in Taskotron, because "dnf config-manager" stopped working. Please revert/fix this fast. Thanks.

Comment 9 Miro Hrončok 2017-08-21 11:54:53 UTC
(In reply to Kamil Páral from comment #8)
> There are more WGs than just Modularity. Also, there are users.

Seems like this is often forgotten when it comes to Modularity. Sorry about breaking Fedora once again, we were just doing what we were told to do.

Comment 10 Tomas Orsava 2017-08-21 12:22:01 UTC
After conferring with others, we're reverting the default interpreter in dnf to python3 in standard Fedora.

Comment 11 Miroslav Suchý 2017-08-21 13:03:05 UTC
This can break mock builds as well.

The use case is when you build 'mock --bootstrap-chroot' then 'dnf builddep foo.src.rpm' is called in minimal chroot. And if platform-python is used and builddep plugin does not work, then mock builds are failing.

Bootstrap chroot is not used by default - yet. So this is not critical, but should be considered.

Comment 12 Tomas Orsava 2017-08-21 13:34:18 UTC
New builds of dnf (dnf-2.6.3-3.fc28, dnf-2.6.3-3.fc27) without platform-python are in koji and should fix the issues in standard Fedora.

Comment 13 Kamil Páral 2017-08-21 15:13:17 UTC
Thanks, folks, for a fast response here.

Comment 14 Tomas Orsava 2017-08-21 15:31:07 UTC
(In reply to Kamil Páral from comment #13)
> Thanks, folks, for a fast response here.

Thank you all for patience!


I'm closing this bug. Feel free to reopen if needed.

Comment 15 Igor Gnatenko 2017-08-22 08:01:29 UTC
*** Bug 1483874 has been marked as a duplicate of this bug. ***

Comment 16 Miro Hrončok 2017-08-22 11:35:35 UTC
It was decided that the platform-python-dnf package will be reintroduced, but dnf will require python3-dnf in "normal" Fedora.

There will be a bcond that selects what "dnf" is using (platform-python-dnf vs. python3-dnf), so modularity can switch that bcond.

Reopening and reassigning to dnf.

Comment 17 Miro Hrončok 2017-08-22 12:38:02 UTC
Actually added a %dnf_python macro, as is is more legible:

A PR: https://src.fedoraproject.org/rpms/dnf/pull-request/2

Comment 18 Miro Hrončok 2017-08-22 13:38:16 UTC
Built in Fedora rawhide and 27 with:

    %global dnf_python python3

Modularity would just need to do:

    %global dnf_python platform-python

Comment 19 David Vásquez 2017-08-23 05:23:04 UTC
Installing the updated version of dnf in koji doesn't solved the issue... Also the platform-python-dnf is a protected package then remove using dnf or yum-deprecated without affect the system is impossible.


[root@3cbc331f3292 tmp]# dnf install *.rpm
Fedora - Rawhide - Developmental packages for t 172 kB/s |  66 MB     06:29    
Last metadata expiration check: 0:04:28 ago on Wed Aug 23 04:33:56 2017.
Error: 
 Problem: problem with installed package platform-python-dnf-2.6.3-2.fc28.noarch
  - package platform-python-dnf-2.6.3-2.fc28.noarch requires dnf-conf = 2.6.3-2.fc28, but none of the providers can be installed
  - cannot install both dnf-conf-2.6.3-3.fc28.noarch and dnf-conf-2.6.3-2.fc28.noarch
  - conflicting requests
(try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstalable packages)


*** I made the next:

I download from koji the new update of dnf ( https://koji.fedoraproject.org/koji/buildinfo?buildID=959417 )

My old friend rpm...

rpm -e --nodeps platform-python-dnf

yum-deprecated install *.rpm


[root@3cbc331f3292 tmp]# yum-deprecated install *.rpm
Yum command has been deprecated, use dnf instead.
See 'man dnf' and 'man yum2dnf' for more information.

Examining dnf-2.6.3-3.fc28.noarch.rpm: dnf-2.6.3-3.fc28.noarch
Marking dnf-2.6.3-3.fc28.noarch.rpm as an update to dnf-2.6.3-2.fc28.noarch
Examining dnf-automatic-2.6.3-3.fc28.noarch.rpm: dnf-automatic-2.6.3-3.fc28.noarch
Marking dnf-automatic-2.6.3-3.fc28.noarch.rpm to be installed
Examining dnf-conf-2.6.3-3.fc28.noarch.rpm: dnf-conf-2.6.3-3.fc28.noarch
Marking dnf-conf-2.6.3-3.fc28.noarch.rpm as an update to dnf-conf-2.6.3-2.fc28.noarch
Examining dnf-yum-2.6.3-3.fc28.noarch.rpm: dnf-yum-2.6.3-3.fc28.noarch
Marking dnf-yum-2.6.3-3.fc28.noarch.rpm as an update to dnf-yum-2.6.3-2.fc28.noarch
Examining python2-dnf-2.6.3-3.fc28.noarch.rpm: python2-dnf-2.6.3-3.fc28.noarch
Marking python2-dnf-2.6.3-3.fc28.noarch.rpm as an update to python2-dnf-2.6.3-2.fc28.noarch
Examining python3-dnf-2.6.3-3.fc28.noarch.rpm: python3-dnf-2.6.3-3.fc28.noarch
Marking python3-dnf-2.6.3-3.fc28.noarch.rpm as an update to python3-dnf-2.6.3-2.fc28.noarch
Resolving Dependencies
--> Running transaction check
---> Package dnf.noarch 0:2.6.3-2.fc28 will be updated
---> Package dnf.noarch 0:2.6.3-3.fc28 will be an update
---> Package dnf-automatic.noarch 0:2.6.3-3.fc28 will be installed
---> Package dnf-conf.noarch 0:2.6.3-2.fc28 will be updated
---> Package dnf-conf.noarch 0:2.6.3-3.fc28 will be an update
---> Package dnf-yum.noarch 0:2.6.3-2.fc28 will be updated
---> Package dnf-yum.noarch 0:2.6.3-3.fc28 will be an update
---> Package python2-dnf.noarch 0:2.6.3-2.fc28 will be updated
---> Package python2-dnf.noarch 0:2.6.3-3.fc28 will be an update
---> Package python3-dnf.noarch 0:2.6.3-2.fc28 will be updated
---> Package python3-dnf.noarch 0:2.6.3-3.fc28 will be an update
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package        Arch    Version       Repository                           Size
================================================================================
Installing:
 dnf-automatic  noarch  2.6.3-3.fc28  /dnf-automatic-2.6.3-3.fc28.noarch   41 k
Updating:
 dnf            noarch  2.6.3-3.fc28  /dnf-2.6.3-3.fc28.noarch            1.2 M
 dnf-conf       noarch  2.6.3-3.fc28  /dnf-conf-2.6.3-3.fc28.noarch        32 k
 dnf-yum        noarch  2.6.3-3.fc28  /dnf-yum-2.6.3-3.fc28.noarch        739  
 python2-dnf    noarch  2.6.3-3.fc28  /python2-dnf-2.6.3-3.fc28.noarch    2.0 M
 python3-dnf    noarch  2.6.3-3.fc28  /python3-dnf-2.6.3-3.fc28.noarch    1.8 M

Transaction Summary
================================================================================
Install  1 Package
Upgrade  5 Packages

Total size: 5.1 M
Is this ok [y/d/N]: y
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Updating   : dnf-conf-2.6.3-3.fc28.noarch                                1/11 
  Updating   : python3-dnf-2.6.3-3.fc28.noarch                             2/11 
  Updating   : dnf-2.6.3-3.fc28.noarch                                     3/11 
  Updating   : dnf-yum-2.6.3-3.fc28.noarch                                 4/11 
  Installing : dnf-automatic-2.6.3-3.fc28.noarch                           5/11 
  Updating   : python2-dnf-2.6.3-3.fc28.noarch                             6/11 
  Cleanup    : python2-dnf-2.6.3-2.fc28.noarch                             7/11 
  Cleanup    : python3-dnf-2.6.3-2.fc28.noarch                             8/11 
  Cleanup    : dnf-yum-2.6.3-2.fc28.noarch                                 9/11 
  Cleanup    : dnf-2.6.3-2.fc28.noarch                                    10/11 
Failed to connect to bus: No such file or directory
  Cleanup    : dnf-conf-2.6.3-2.fc28.noarch                               11/11 
Failed to connect to bus: No such file or directory
  Verifying  : dnf-yum-2.6.3-3.fc28.noarch                                 1/11 
  Verifying  : dnf-automatic-2.6.3-3.fc28.noarch                           2/11 
  Verifying  : python3-dnf-2.6.3-3.fc28.noarch                             3/11 
  Verifying  : python2-dnf-2.6.3-3.fc28.noarch                             4/11 
  Verifying  : dnf-conf-2.6.3-3.fc28.noarch                                5/11 
  Verifying  : dnf-2.6.3-3.fc28.noarch                                     6/11 
  Verifying  : dnf-2.6.3-2.fc28.noarch                                     7/11 
  Verifying  : python2-dnf-2.6.3-2.fc28.noarch                             8/11 
  Verifying  : python3-dnf-2.6.3-2.fc28.noarch                             9/11 
  Verifying  : dnf-conf-2.6.3-2.fc28.noarch                               10/11 
  Verifying  : dnf-yum-2.6.3-2.fc28.noarch                                11/11 

Installed:
  dnf-automatic.noarch 0:2.6.3-3.fc28                                           

Updated:
  dnf.noarch 0:2.6.3-3.fc28              dnf-conf.noarch 0:2.6.3-3.fc28        
  dnf-yum.noarch 0:2.6.3-3.fc28          python2-dnf.noarch 0:2.6.3-3.fc28     
  python3-dnf.noarch 0:2.6.3-3.fc28     

Complete!


Then, I am not sure if the updated dnf will fix without the trick...

Comment 20 Kamil Páral 2017-08-23 08:52:59 UTC
(In reply to David Vásquez from comment #19)
> Installing the updated version of dnf in koji doesn't solved the issue...

Both dnf-2.6.3-4.fc28 and dnf-2.6.3-4.fc27 solve the problem for me.

> Also the platform-python-dnf is a protected package then remove using dnf or
> yum-deprecated without affect the system is impossible.

Use --allowerasing as suggested.

Comment 21 Miro Hrončok 2017-08-23 10:42:10 UTC
When the packages are in repository, you'll update dnf, dnf-conf, python3-dnf, platform-python-dnf to 2.6.3-4. Than you can safely remove platform-python*.

Comment 22 Igor Gnatenko 2017-08-23 11:08:00 UTC
*** Bug 1483253 has been marked as a duplicate of this bug. ***

Comment 23 Miroslav Suchý 2017-08-23 11:21:00 UTC
*** Bug 1483864 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.