Bug 1156551

Created attachment 1038046 [details]
Basic dnf support

salt-minion fails sometimes due to the fact that yum install is executed as dnf install which is different: it doesn't do updates, except when additional options are passed like --best.

Created attachment 1091673 [details]
Potential backported solution

Please could you provide any information regarding of switch over to dnf from yum? Thanks a lot

Reliance on repoquery (which was the sole reason for pulling in yum-utils) has been removed in the develop branch of salt, so this will be a non-issue moving forward.

The current salt release cycle (2015.8.x) cannot be built on the FBS due to reliance on newer tornado and pycrypto, so until the releases catch up with the dependency versions we require, the Fedora/EPEL repos will not get new salt releases and our own repos will be needed to install salt.

Updates were submitted yesterday for 2015.5.8, which brings Fedora/EPEL up-to-date with the latest 2015.5 release. The 2015.5 release cycle doesn't support dnf, though, this was added in the 2015.8 branch.

I will confirm whether or not we plan on a 2015.5.9 release, and if so I will see to it that dnf support is backported from the 2015.8 release branch to the 2015.5 release branch, and that (where used), the repoquery plugin for dnf is used in place of repoquery from yum-utils.

Excellent, thanks a lot for information.

OK, I'm working on backporting dnf support into the 2015.5 release branch of Salt right now. I didn't make the cut for 2015.5.9 though as it is going to be released soon, however when I build the 2015.5.9 packages I will patch the source with the code I'm working on now so that the Fedora RPMs work properly with dnf.

Backport completed upstream in Salt here: https://github.com/saltstack/salt/pull/30484

I'll be building 2015.5.9 RPMs by the end of the week, which will contain the patch from that pull request.

salt-2015.5.9-1.fc22 has been submitted as an update to Fedora 22. https://bodhi.fedoraproject.org/updates/FEDORA-2016-02f4d3fee1

salt-2015.5.9-1.fc22 has been pushed to the Fedora 22 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-02f4d3fee1

salt-2015.5.9-2.fc22 has been submitted as an update to Fedora 22. https://bodhi.fedoraproject.org/updates/FEDORA-2016-c1fd651bc0

salt-2015.5.9-2.fc22 has been pushed to the Fedora 22 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-c1fd651bc0

salt-2015.5.9-2.fc22 has been pushed to the Fedora 22 stable repository. If problems still persist, please make note of it in this bug report.

Not sure if salt is to blame, but somewhere in the salt process dnf is called like this (on fedora 23):

-bash-4.3# /usr/bin/dnf repoquery --quiet --queryformat '%{NAME}_|-%{EPOCH}:%{VERSION}_|-%{RELEASE}_|-%{ARCH}_|-%{REPOID}' --whatprovides 'unzip tar nano'
*** Error in `/usr/bin/python3': corrupted double-linked list: 0x0000559cb8907420 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x77e15)[0x7fe77a78fe15]
/lib64/libc.so.6(+0x7ed43)[0x7fe77a796d43]
/lib64/libc.so.6(+0x818c6)[0x7fe77a7998c6]
/lib64/libc.so.6(__libc_malloc+0x50)[0x7fe77a79b870]
/lib64/libc.so.6(realloc+0x368)[0x7fe77a79c0b8]
/lib64/libpython3.4m.so.1.0(+0x1645e9)[0x7fe77b5635e9]
/lib64/libpython3.4m.so.1.0(+0x1687ca)[0x7fe77b5677ca]
/lib64/libpython3.4m.so.1.0(+0x16e423)[0x7fe77b56d423]
/lib64/libpython3.4m.so.1.0(+0x170127)[0x7fe77b56f127]
/lib64/libpython3.4m.so.1.0(+0x170a8f)[0x7fe77b56fa8f]
/lib64/libpython3.4m.so.1.0(PyEval_EvalFrameEx+0x82f1)[0x7fe77b519651]
/lib64/libpython3.4m.so.1.0(PyEval_EvalCodeEx+0x8c6)[0x7fe77b51a636]
/lib64/libpython3.4m.so.1.0(PyEval_EvalFrameEx+0x66cf)[0x7fe77b517a2f]
/lib64/libpython3.4m.so.1.0(PyEval_EvalFrameEx+0x7aeb)[0x7fe77b518e4b]
/lib64/libpython3.4m.so.1.0(PyEval_EvalFrameEx+0x7aeb)[0x7fe77b518e4b]
/lib64/libpython3.4m.so.1.0(PyEval_EvalFrameEx+0x7aeb)[0x7fe77b518e4b]
/lib64/libpython3.4m.so.1.0(PyEval_EvalFrameEx+0x7aeb)[0x7fe77b518e4b]
/lib64/libpython3.4m.so.1.0(PyEval_EvalFrameEx+0x7aeb)[0x7fe77b518e4b]
/lib64/libpython3.4m.so.1.0(PyEval_EvalFrameEx+0x7aeb)[0x7fe77b518e4b]
/lib64/libpython3.4m.so.1.0(PyEval_EvalCodeEx+0x8c6)[0x7fe77b51a636]
/lib64/libpython3.4m.so.1.0(PyEval_EvalFrameEx+0x66cf)[0x7fe77b517a2f]
/lib64/libpython3.4m.so.1.0(PyEval_EvalCodeEx+0x8c6)[0x7fe77b51a636]
/lib64/libpython3.4m.so.1.0(PyEval_EvalCode+0x1b)[0x7fe77b51a6db]
/lib64/libpython3.4m.so.1.0(+0x137954)[0x7fe77b536954]
/lib64/libpython3.4m.so.1.0(PyRun_FileExFlags+0x95)[0x7fe77b538b95]
/lib64/libpython3.4m.so.1.0(PyRun_SimpleFileExFlags+0x113)[0x7fe77b539c13]
/lib64/libpython3.4m.so.1.0(Py_Main+0xdc4)[0x7fe77b550964]
/usr/bin/python3(main+0x167)[0x559cb4e2fbe7]
/lib64/libc.so.6(__libc_start_main+0xf0)[0x7fe77a738580]
/usr/bin/python3(_start+0x29)[0x559cb4e2fc89]

On fedora 22:
[rolf.fokkens@f22builder ~]$ sudo -i
-bash-4.3# /usr/bin/dnf repoquery --quiet --queryformat '%{NAME}_|-%{EPOCH}:%{VERSION}_|-%{RELEASE}_|-%{ARCH}_|-%{REPOID}' --whatprovides 'unzip tar nano'
*** Error in `/usr/bin/python': malloc(): smallbin double linked list corrupted: 0x000000000205fa90 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x77a8d)[0x7fcf558cba8d]
/lib64/libc.so.6(+0x81838)[0x7fcf558d5838]
/lib64/libc.so.6(__libc_calloc+0xd4)[0x7fcf558d8d04]
/lib64/libsolv.so.0(solv_calloc+0x1b)[0x7fcf42ca5cbb]
/lib64/libhawkey.so.2(hy_query_apply+0x1598)[0x7fcf42ef6f48]
/lib64/libhawkey.so.2(hy_query_run_set+0x9)[0x7fcf42ef7a39]
/usr/lib64/python2.7/site-packages/hawkey/_hawkeymodule.so(+0xdc12)[0x7fcf4310fc12]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x4b37)[0x7fcf56620c17]
/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x834)[0x7fcf566226b4]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x54e6)[0x7fcf566215c6]
/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x834)[0x7fcf566226b4]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x54e6)[0x7fcf566215c6]
/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x834)[0x7fcf566226b4]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x54e6)[0x7fcf566215c6]
/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x834)[0x7fcf566226b4]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x54e6)[0x7fcf566215c6]
/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x834)[0x7fcf566226b4]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x54e6)[0x7fcf566215c6]
/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x834)[0x7fcf566226b4]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x54e6)[0x7fcf566215c6]
/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x834)[0x7fcf566226b4]
/lib64/libpython2.7.so.1.0(PyEval_EvalCode+0x19)[0x7fcf566227d9]
/lib64/libpython2.7.so.1.0(+0xfcbdf)[0x7fcf5663bbdf]
/lib64/libpython2.7.so.1.0(PyRun_FileExFlags+0x72)[0x7fcf5663cdb2]
/lib64/libpython2.7.so.1.0(PyRun_SimpleFileExFlags+0xe7)[0x7fcf5663dfc7]
/lib64/libpython2.7.so.1.0(Py_Main+0xcc1)[0x7fcf566501e1]
/lib64/libc.so.6(__libc_start_main+0xf0)[0x7fcf55874700]
/usr/bin/python(_start+0x29)[0x400729]

Seems a python/dnf issue like #907649.

Yes, this is not a Salt issue. However, there is a bug in how that repoquery command is invoked, which is fixed in 2015.5.9-3 (currently in testing repos):

Fedora 22: https://bodhi.fedoraproject.org/updates/FEDORA-2016-b3098694c7
Fedora 23: https://bodhi.fedoraproject.org/updates/FEDORA-2016-a3dfb47f84

Both on F22 and F23 it the problem is gone now!

Created attachment 1128337 [details]
Fix for dnf options --best and/or --allowerasing

Unfortunately Comment #2 (still) applies (again). salt states pkg.lastest fail due to the requirement of passing of "--best" and/or "--allowerasing" are required for "dnf install" while they're not required for "yum install". The patch fixes this.

salt-2015.5.9-4.fc22 has been submitted as an update to Fedora 22. https://bodhi.fedoraproject.org/updates/FEDORA-2016-40983ce38d

2015.5.9-4 has been submitted to the testing repos and fixes the issue raised in Comment #2. I tested it myself on F22, but I'd appreciate you testing as well.

With this version I get all kinds of:

     Comment: State 'pkg.removed' was not found in SLS '...'
              Reason: Module 'pkg' is not available.

     Comment: State 'pkg.installed' was not found in SLS 'states.sshd'
              Reason: Module 'pkg' is not available.

So for me the situation has not improved.

salt-2015.5.9-4.fc22 has been pushed to the Fedora 22 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-40983ce38d

Rolf,

Looks like a forgot to include a __init__.py in the patch. Oddly enough this did not get caught in my earlier testing, I think this might be due to the fact that the box on which I tested previously had Salt installed from source and I did not fully remove the prior installation before testing. Sorry for the oversight there.

Please try the new builds for F22/F23: 

http://koji.fedoraproject.org/koji/buildinfo?buildID=740495
http://koji.fedoraproject.org/koji/buildinfo?buildID=740512

salt-2015.5.9-5.fc22 has been submitted as an update to Fedora 22. https://bodhi.fedoraproject.org/updates/FEDORA-2016-dc4b006a79

salt-2015.5.9-5.fc22 has been pushed to the Fedora 22 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-dc4b006a79

just for reference pkgrepo related states do not work either without yum/yum-utils:

https://github.com/saltstack/salt/issues/31240

is anyone dealing with this? since the package rename of python-dnf-plugins-core to python2-dnf-plugins-core.noarch salt-2015.5.9-2.fc22 is broken.

A newer 2015.5 release will be built this weekend, which will contain corrected Requires.

salt-2015.5.10-1.fc22 has been submitted as an update to Fedora 22. https://bodhi.fedoraproject.org/updates/FEDORA-2016-1fdb5e296b

salt-2015.5.10-1.fc22 has been pushed to the Fedora 22 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-1fdb5e296b

salt-2015.5.10-2.fc22 has been submitted as an update to Fedora 22. https://bodhi.fedoraproject.org/updates/FEDORA-2016-bab7abfb49

salt-2015.5.10-2.fc22 has been pushed to the Fedora 22 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-bab7abfb49

salt-2015.5.10-2.fc22 has been pushed to the Fedora 22 stable repository. If problems still persist, please make note of it in this bug report.