Bug 1156551

Summary: [rfe] use dnf instead of yum
Product: [Fedora] Fedora Reporter: Rahul Sundaram <metherid>
Component: saltAssignee: Erik Johnson <erik>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: urgent    
Version: rawhideCC: andrewniemants, erik, herlo1, jmracek, rolf, tobias.jungel
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: salt-2015.5.9-2.fc22 salt-2015.5.10-2.fc22 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-06-11 02:48:42 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 907649    
Bug Blocks: 1156491    
Attachments:
Description Flags
Basic dnf support
none
Potential backported solution
none
Fix for dnf options --best and/or --allowerasing none

Description Rahul Sundaram 2014-10-24 16:18:18 UTC
Description of problem:

Fedora 22 is planning to switch over to dnf from yum as per

https://fedoraproject.org/wiki/Changes/ReplaceYumWithDNF

Please switch from yum/yum-utils to dnf/dnf-plugins-core. Thanks!

Comment 1 Michael Mráka 2015-06-12 13:14:03 UTC
Created attachment 1038046 [details]
Basic dnf support

Comment 2 Rolf Fokkens 2015-11-09 09:04:50 UTC
salt-minion fails sometimes due to the fact that yum install is executed as dnf install which is different: it doesn't do updates, except when additional options are passed like --best.

Comment 3 Rolf Fokkens 2015-11-09 09:50:23 UTC
Created attachment 1091673 [details]
Potential backported solution

Comment 4 Jaroslav Mracek 2016-01-06 16:02:30 UTC
Please could you provide any information regarding of switch over to dnf from yum? Thanks a lot

Comment 5 Erik Johnson 2016-01-06 18:25:55 UTC
Reliance on repoquery (which was the sole reason for pulling in yum-utils) has been removed in the develop branch of salt, so this will be a non-issue moving forward.

The current salt release cycle (2015.8.x) cannot be built on the FBS due to reliance on newer tornado and pycrypto, so until the releases catch up with the dependency versions we require, the Fedora/EPEL repos will not get new salt releases and our own repos will be needed to install salt.

Updates were submitted yesterday for 2015.5.8, which brings Fedora/EPEL up-to-date with the latest 2015.5 release. The 2015.5 release cycle doesn't support dnf, though, this was added in the 2015.8 branch.

I will confirm whether or not we plan on a 2015.5.9 release, and if so I will see to it that dnf support is backported from the 2015.8 release branch to the 2015.5 release branch, and that (where used), the repoquery plugin for dnf is used in place of repoquery from yum-utils.

Comment 6 Jaroslav Mracek 2016-01-11 11:23:14 UTC
Excellent, thanks a lot for information.

Comment 7 Erik Johnson 2016-01-15 20:29:57 UTC
OK, I'm working on backporting dnf support into the 2015.5 release branch of Salt right now. I didn't make the cut for 2015.5.9 though as it is going to be released soon, however when I build the 2015.5.9 packages I will patch the source with the code I'm working on now so that the Fedora RPMs work properly with dnf.

Comment 8 Erik Johnson 2016-01-20 21:44:39 UTC
Backport completed upstream in Salt here: https://github.com/saltstack/salt/pull/30484

I'll be building 2015.5.9 RPMs by the end of the week, which will contain the patch from that pull request.

Comment 9 Fedora Update System 2016-01-22 00:01:32 UTC
salt-2015.5.9-1.fc22 has been submitted as an update to Fedora 22. https://bodhi.fedoraproject.org/updates/FEDORA-2016-02f4d3fee1

Comment 10 Fedora Update System 2016-01-24 04:51:15 UTC
salt-2015.5.9-1.fc22 has been pushed to the Fedora 22 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-02f4d3fee1

Comment 11 Fedora Update System 2016-01-24 18:35:05 UTC
salt-2015.5.9-2.fc22 has been submitted as an update to Fedora 22. https://bodhi.fedoraproject.org/updates/FEDORA-2016-c1fd651bc0

Comment 12 Fedora Update System 2016-01-26 04:23:45 UTC
salt-2015.5.9-2.fc22 has been pushed to the Fedora 22 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-c1fd651bc0

Comment 13 Fedora Update System 2016-02-05 00:21:37 UTC
salt-2015.5.9-2.fc22 has been pushed to the Fedora 22 stable repository. If problems still persist, please make note of it in this bug report.

Comment 14 Rolf Fokkens 2016-02-08 13:28:26 UTC
Not sure if salt is to blame, but somewhere in the salt process dnf is called like this (on fedora 23):

-bash-4.3# /usr/bin/dnf repoquery --quiet --queryformat '%{NAME}_|-%{EPOCH}:%{VERSION}_|-%{RELEASE}_|-%{ARCH}_|-%{REPOID}' --whatprovides 'unzip tar nano'
*** Error in `/usr/bin/python3': corrupted double-linked list: 0x0000559cb8907420 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x77e15)[0x7fe77a78fe15]
/lib64/libc.so.6(+0x7ed43)[0x7fe77a796d43]
/lib64/libc.so.6(+0x818c6)[0x7fe77a7998c6]
/lib64/libc.so.6(__libc_malloc+0x50)[0x7fe77a79b870]
/lib64/libc.so.6(realloc+0x368)[0x7fe77a79c0b8]
/lib64/libpython3.4m.so.1.0(+0x1645e9)[0x7fe77b5635e9]
/lib64/libpython3.4m.so.1.0(+0x1687ca)[0x7fe77b5677ca]
/lib64/libpython3.4m.so.1.0(+0x16e423)[0x7fe77b56d423]
/lib64/libpython3.4m.so.1.0(+0x170127)[0x7fe77b56f127]
/lib64/libpython3.4m.so.1.0(+0x170a8f)[0x7fe77b56fa8f]
/lib64/libpython3.4m.so.1.0(PyEval_EvalFrameEx+0x82f1)[0x7fe77b519651]
/lib64/libpython3.4m.so.1.0(PyEval_EvalCodeEx+0x8c6)[0x7fe77b51a636]
/lib64/libpython3.4m.so.1.0(PyEval_EvalFrameEx+0x66cf)[0x7fe77b517a2f]
/lib64/libpython3.4m.so.1.0(PyEval_EvalFrameEx+0x7aeb)[0x7fe77b518e4b]
/lib64/libpython3.4m.so.1.0(PyEval_EvalFrameEx+0x7aeb)[0x7fe77b518e4b]
/lib64/libpython3.4m.so.1.0(PyEval_EvalFrameEx+0x7aeb)[0x7fe77b518e4b]
/lib64/libpython3.4m.so.1.0(PyEval_EvalFrameEx+0x7aeb)[0x7fe77b518e4b]
/lib64/libpython3.4m.so.1.0(PyEval_EvalFrameEx+0x7aeb)[0x7fe77b518e4b]
/lib64/libpython3.4m.so.1.0(PyEval_EvalFrameEx+0x7aeb)[0x7fe77b518e4b]
/lib64/libpython3.4m.so.1.0(PyEval_EvalCodeEx+0x8c6)[0x7fe77b51a636]
/lib64/libpython3.4m.so.1.0(PyEval_EvalFrameEx+0x66cf)[0x7fe77b517a2f]
/lib64/libpython3.4m.so.1.0(PyEval_EvalCodeEx+0x8c6)[0x7fe77b51a636]
/lib64/libpython3.4m.so.1.0(PyEval_EvalCode+0x1b)[0x7fe77b51a6db]
/lib64/libpython3.4m.so.1.0(+0x137954)[0x7fe77b536954]
/lib64/libpython3.4m.so.1.0(PyRun_FileExFlags+0x95)[0x7fe77b538b95]
/lib64/libpython3.4m.so.1.0(PyRun_SimpleFileExFlags+0x113)[0x7fe77b539c13]
/lib64/libpython3.4m.so.1.0(Py_Main+0xdc4)[0x7fe77b550964]
/usr/bin/python3(main+0x167)[0x559cb4e2fbe7]
/lib64/libc.so.6(__libc_start_main+0xf0)[0x7fe77a738580]
/usr/bin/python3(_start+0x29)[0x559cb4e2fc89]

Comment 15 Rolf Fokkens 2016-02-08 13:33:42 UTC
On fedora 22:
[rolf.fokkens@f22builder ~]$ sudo -i
-bash-4.3# /usr/bin/dnf repoquery --quiet --queryformat '%{NAME}_|-%{EPOCH}:%{VERSION}_|-%{RELEASE}_|-%{ARCH}_|-%{REPOID}' --whatprovides 'unzip tar nano'
*** Error in `/usr/bin/python': malloc(): smallbin double linked list corrupted: 0x000000000205fa90 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x77a8d)[0x7fcf558cba8d]
/lib64/libc.so.6(+0x81838)[0x7fcf558d5838]
/lib64/libc.so.6(__libc_calloc+0xd4)[0x7fcf558d8d04]
/lib64/libsolv.so.0(solv_calloc+0x1b)[0x7fcf42ca5cbb]
/lib64/libhawkey.so.2(hy_query_apply+0x1598)[0x7fcf42ef6f48]
/lib64/libhawkey.so.2(hy_query_run_set+0x9)[0x7fcf42ef7a39]
/usr/lib64/python2.7/site-packages/hawkey/_hawkeymodule.so(+0xdc12)[0x7fcf4310fc12]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x4b37)[0x7fcf56620c17]
/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x834)[0x7fcf566226b4]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x54e6)[0x7fcf566215c6]
/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x834)[0x7fcf566226b4]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x54e6)[0x7fcf566215c6]
/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x834)[0x7fcf566226b4]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x54e6)[0x7fcf566215c6]
/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x834)[0x7fcf566226b4]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x54e6)[0x7fcf566215c6]
/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x834)[0x7fcf566226b4]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x54e6)[0x7fcf566215c6]
/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x834)[0x7fcf566226b4]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x54e6)[0x7fcf566215c6]
/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x834)[0x7fcf566226b4]
/lib64/libpython2.7.so.1.0(PyEval_EvalCode+0x19)[0x7fcf566227d9]
/lib64/libpython2.7.so.1.0(+0xfcbdf)[0x7fcf5663bbdf]
/lib64/libpython2.7.so.1.0(PyRun_FileExFlags+0x72)[0x7fcf5663cdb2]
/lib64/libpython2.7.so.1.0(PyRun_SimpleFileExFlags+0xe7)[0x7fcf5663dfc7]
/lib64/libpython2.7.so.1.0(Py_Main+0xcc1)[0x7fcf566501e1]
/lib64/libc.so.6(__libc_start_main+0xf0)[0x7fcf55874700]
/usr/bin/python(_start+0x29)[0x400729]

Comment 16 Rolf Fokkens 2016-02-08 13:34:42 UTC
Seems a python/dnf issue like #907649.

Comment 17 Erik Johnson 2016-02-08 14:54:02 UTC
Yes, this is not a Salt issue. However, there is a bug in how that repoquery command is invoked, which is fixed in 2015.5.9-3 (currently in testing repos):

Fedora 22: https://bodhi.fedoraproject.org/updates/FEDORA-2016-b3098694c7
Fedora 23: https://bodhi.fedoraproject.org/updates/FEDORA-2016-a3dfb47f84

Comment 18 Rolf Fokkens 2016-02-08 21:34:21 UTC
Both on F22 and F23 it the problem is gone now!

Comment 19 Rolf Fokkens 2016-02-18 21:35:45 UTC
Created attachment 1128337 [details]
Fix for dnf options --best and/or --allowerasing

Unfortunately Comment #2 (still) applies (again). salt states pkg.lastest fail due to the requirement of passing of "--best" and/or "--allowerasing" are required for "dnf install" while they're not required for "yum install". The patch fixes this.

Comment 20 Fedora Update System 2016-02-22 06:58:34 UTC
salt-2015.5.9-4.fc22 has been submitted as an update to Fedora 22. https://bodhi.fedoraproject.org/updates/FEDORA-2016-40983ce38d

Comment 21 Erik Johnson 2016-02-22 07:03:17 UTC
2015.5.9-4 has been submitted to the testing repos and fixes the issue raised in Comment #2. I tested it myself on F22, but I'd appreciate you testing as well.

Comment 22 Rolf Fokkens 2016-02-22 20:29:55 UTC
With this version I get all kinds of:

     Comment: State 'pkg.removed' was not found in SLS '...'
              Reason: Module 'pkg' is not available.

     Comment: State 'pkg.installed' was not found in SLS 'states.sshd'
              Reason: Module 'pkg' is not available.

So for me the situation has not improved.

Comment 23 Fedora Update System 2016-02-23 02:52:38 UTC
salt-2015.5.9-4.fc22 has been pushed to the Fedora 22 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-40983ce38d

Comment 24 Erik Johnson 2016-03-01 17:19:56 UTC
Rolf,

Looks like a forgot to include a __init__.py in the patch. Oddly enough this did not get caught in my earlier testing, I think this might be due to the fact that the box on which I tested previously had Salt installed from source and I did not fully remove the prior installation before testing. Sorry for the oversight there.

Please try the new builds for F22/F23: 

http://koji.fedoraproject.org/koji/buildinfo?buildID=740495
http://koji.fedoraproject.org/koji/buildinfo?buildID=740512

Comment 25 Fedora Update System 2016-03-01 17:20:25 UTC
salt-2015.5.9-5.fc22 has been submitted as an update to Fedora 22. https://bodhi.fedoraproject.org/updates/FEDORA-2016-dc4b006a79

Comment 26 Fedora Update System 2016-03-02 03:31:54 UTC
salt-2015.5.9-5.fc22 has been pushed to the Fedora 22 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-dc4b006a79

Comment 27 Tobias Jungel 2016-03-04 08:42:46 UTC
just for reference pkgrepo related states do not work either without yum/yum-utils:

https://github.com/saltstack/salt/issues/31240

Comment 28 Tobias Jungel 2016-05-27 13:36:12 UTC
is anyone dealing with this? since the package rename of python-dnf-plugins-core to python2-dnf-plugins-core.noarch salt-2015.5.9-2.fc22 is broken.

Comment 29 Erik Johnson 2016-05-27 18:27:13 UTC
A newer 2015.5 release will be built this weekend, which will contain corrected Requires.

Comment 30 Fedora Update System 2016-05-27 20:30:37 UTC
salt-2015.5.10-1.fc22 has been submitted as an update to Fedora 22. https://bodhi.fedoraproject.org/updates/FEDORA-2016-1fdb5e296b

Comment 31 Fedora Update System 2016-05-28 03:21:10 UTC
salt-2015.5.10-1.fc22 has been pushed to the Fedora 22 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-1fdb5e296b

Comment 32 Fedora Update System 2016-05-31 14:11:23 UTC
salt-2015.5.10-2.fc22 has been submitted as an update to Fedora 22. https://bodhi.fedoraproject.org/updates/FEDORA-2016-bab7abfb49

Comment 33 Fedora Update System 2016-06-01 15:22:40 UTC
salt-2015.5.10-2.fc22 has been pushed to the Fedora 22 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-bab7abfb49

Comment 34 Fedora Update System 2016-06-11 02:48:36 UTC
salt-2015.5.10-2.fc22 has been pushed to the Fedora 22 stable repository. If problems still persist, please make note of it in this bug report.