Bug 1502106

Summary: dnf packages update breaks package removal via ansible
Product: [Fedora] Fedora Reporter: Milan Zink <mzink>
Component: dnfAssignee: Jaroslav Mracek <jmracek>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: low Docs Contact:
Priority: unspecified    
Version: 26CC: cohoe.grant, dmach, ignatenko, jmracek, mhatina, msuchy, mzink, packaging-team-maint, rpm-software-management, vmukhame
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: dnf-2.7.5-1.fc26 dnf-2.7.5-1.fc27 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-10-21 22:21:48 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Milan Zink 2017-10-14 08:59:17 UTC
Description of problem:

Latest upgrade of dnf* packages (probably python2-dnf) breaks package removal via ansible dnf/package module 


Version-Release number of selected component (if applicable):

Working versions
dnf-conf-2.5.1-1.fc26.noarch
dnf-plugins-core-2.1.1-1.fc26.noarch
dnf-utils-2.1.1-1.fc26.noarch
dnf-yum-2.5.1-1.fc26.noarch
dnf-2.5.1-1.fc26.noarch
libdnf-0.9.1-1.fc26.x86_64
python2-dnf-2.5.1-1.fc26.noarch
python3-dnf-plugins-core-2.1.1-1.fc26.noarch
python3-dnf-plugins-extras-common-2.0.1-1.fc26.noarch
python3-dnf-plugin-system-upgrade-2.0.1-1.fc26.noarch
python3-dnf-2.5.1-1.fc26.noarch

Problematic version:
python2-dnf.noarch                                  2.7.3-1.fc26         updates
python2-hawkey.x86_64                               0.11.0-1.fc26        updates
python3-dnf.noarch                                  2.7.3-1.fc26         updates
python3-dnf-plugins-core.noarch                     2.1.5-1.fc26         updates
    python3-dnf-plugins-extras-common.noarch        2.0.1-1.fc26         @fedora
python3-dnf-plugins-extras-common.noarch            2.0.3-1.fc26         updates
python3-dnf-plugins-extras-common.noarch            2.0.3-1.fc26         updates
python3-dnf-plugin-system-upgrade.noarch            2.0.3-1.fc26         updates
python3-hawkey.x86_64                               0.11.0-1.fc26        updates



How reproducible:

Try to remove package with ansible.

Working with older packages:

$ ansible --connection local localhost --become -m dnf -a "name=htop state=absent"
 [WARNING]: Could not match supplied host pattern, ignoring: all

 [WARNING]: provided hosts list is empty, only localhost is available

localhost | SUCCESS => {
    "changed": true, 
    "failed": false, 
    "results": [
        "Removed: htop-2.0.2-2.fc26.x86_64"
    ]
}
$ rpm -qa | grep python2-dnf
python2-dnf-2.5.1-1.fc26.noarch


Not working with latest packages:
$ ansible --connection local localhost --become -m dnf -a "name=htop state=absent"
 [WARNING]: Could not match supplied host pattern, ignoring: all

 [WARNING]: provided hosts list is empty, only localhost is available

localhost | FAILED! => {
    "changed": false, 
    "failed": true, 
    "module_stderr": "No handlers could be found for logger \"dnf\"\nTraceback (most recent call last):\n  File \"/tmp/ansible_CwV96B/ansible_module_dnf.py\", line 534, in <module>\n    main()\n  File \"/tmp/ansible_CwV96B/ansible_module_dnf.py\", line 530, in main\n    ensure(module, base, params['state'], params['name'], params['autoremove'])\n  File \"/tmp/ansible_CwV96B/ansible_module_dnf.py\", line 465, in ensure\n    base.download_packages(base.transaction.install_set)\n  File \"/usr/lib/python2.7/site-packages/dnf/base.py\", line 1025, in download_packages\n    remote_pkgs, local_repository_pkgs = self._select_remote_pkgs(pkglist)\n  File \"/usr/lib/python2.7/site-packages/dnf/base.py\", line 2300, in _select_remote_pkgs\n    _(\"Some packages from local repository have incorrect checksum\"))\ndnf.exceptions.Error: <exception str() failed>\n", 
    "module_stdout": "", 
    "msg": "MODULE FAILURE", 
    "rc": 0
}
$ rpm -qa | grep python2-dnf
python2-dnf-2.7.3-1.fc26.noarch



Steps to Reproduce:
1. try remove package via ansible using dnf or package module
2. works with python2-dnf-2.5.X
3. not working with python2-dnf-2.7.X


Actual results:
module_stderr": "No handlers could be found for logger \"dnf\"\nTraceback (most recent call last):\n  File \"/tmp/ansible_CwV96B/ansible_module_dnf.py\", line 534, in <module>\n    main()\n  File \"/tmp/ansible_CwV96B/ansible_module_dnf.py\", line 530, in main\n    ensure(module, base, params['state'], params['name'], params['autoremove'])\n  File \"/tmp/ansible_CwV96B/ansible_module_dnf.py\", line 465, in ensure\n    base.download_packages(base.transaction.install_set)\n  File \"/usr/lib/python2.7/site-packages/dnf/base.py\", line 1025, in download_packages\n    remote_pkgs, local_repository_pkgs = self._select_remote_pkgs(pkglist)\n  File \"/usr/lib/python2.7/site-packages/dnf/base.py\", line 2300, in _select_remote_pkgs\n    _(\"Some packages from local repository have incorrect checksum\"))\ndnf.exceptions.Error: <exception str() failed>\n", 
    "module_stdout": "", 
    "msg": "MODULE FAILURE", 
    "rc": 0
}


Expected results:
    "changed": true, 
    "failed": false, 
    "results": [
        "Removed: htop-2.0.2-2.fc26.x86_64"
    ]


Additional info:
Install package works. Problem seems to be only with package removal.

Comment 1 Igor Gnatenko 2017-10-14 09:00:54 UTC
This should be fixed by https://github.com/rpm-software-management/dnf/pull/946

Comment 2 Milan Zink 2017-10-14 12:33:30 UTC
I've tried to replace dnf/base.py by file take from upstream/github. But still getting the same error.

Comment 3 Jaroslav Mracek 2017-10-17 06:53:17 UTC
I do not understand some part from report:

It is reported, that it happens for removal operation, but it fails on check of checksum of some packages from local repository (this is new feature). It means that base.transaction.install_set was not empty (for removal ?).

Additionally why some local repositories have packages with incorrect checksums?

Comment 4 Milan Zink 2017-10-17 07:20:19 UTC
Yes, it's complaining about checksum. But it should not. Works w/o issues with older python-dnf package version. 

Can I provide some more details? 

I have some local packages installed (like bluejeans), but why this should affect removal of htop package?

Comment 5 Jaroslav Mracek 2017-10-17 07:40:50 UTC
In ansible modules/packaging/os/dnf.py line 465 
```
base.download_packages(base.transaction.install_set)
```

please can you investigate content of base.transaction.install_set like
"print(base.transaction.install_set)"

Probably it can reveal the problem here.

Comment 6 Jaroslav Mracek 2017-10-17 08:49:05 UTC
I created a copr repo ("dnf copr enable jmracek/test") that have DNF (dnf-2.7.4-1.git.7981.1ce857e.fc2*) with patch that should solve the issue. Please could you test it? If it will not work I will create version for discover the primary cause.

Comment 7 Jaroslav Mracek 2017-10-17 10:01:23 UTC
The version with patch is dnf-2.7.4-1.git.7981.6b56aaf.fc2*

Comment 8 Milan Zink 2017-10-17 10:14:52 UTC
print(base.transaction.install_set) gives 

"module_stdout": "set([])\n"

base.transaction.install_set seems to be empty.

BTW: Simple comment out line  
465         #base.download_packages(base.transaction.install_set)
is a workaround:

 ~ > ansible localhost --connection local --become -m dnf -a 'name=htop state=absent'

localhost | SUCCESS => {
    "changed": true, 
    "failed": false, 
    "results": [
        "Removed: htop-2.0.2-4.fc27.x86_64"
    ]
}


I'm going to try your dnf-2.7.4-1.git.7981.6b56aaf.fc2* packages. Will post an update soon.

Comment 9 Milan Zink 2017-10-17 10:18:38 UTC
No issues with your testing packages: dnf-2.7.4-1.git.7981.6b56aaf.fc2*


 ~ > ansible localhost --connection local --become -m dnf -a 'name=htop state=absent'
 [WARNING]: Could not match supplied host pattern, ignoring: all

 [WARNING]: provided hosts list is empty, only localhost is available

localhost | SUCCESS => {
    "changed": true, 
    "failed": false, 
    "results": [
        "Removed: htop-2.0.2-4.fc27.x86_64"
    ]
}

Works as expected.

Comment 10 Jaroslav Mracek 2017-10-17 11:14:48 UTC
Here is a link with the patch https://github.com/rpm-software-management/dnf/pull/956.

Comment 11 Fedora Update System 2017-10-18 19:07:18 UTC
dnf-2.7.5-1.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-d2e8fdf5d8

Comment 12 Fedora Update System 2017-10-18 19:07:43 UTC
dnf-2.7.5-1.fc27 has been submitted as an update to Fedora 27. https://bodhi.fedoraproject.org/updates/FEDORA-2017-ee5c678e8b

Comment 13 Grant Cohoe 2017-10-18 21:17:50 UTC
(In reply to Milan Zink from comment #8)
> print(base.transaction.install_set) gives 
> 
> "module_stdout": "set([])\n"
> 
> base.transaction.install_set seems to be empty.
> 
> BTW: Simple comment out line  
> 465         #base.download_packages(base.transaction.install_set)
> is a workaround:
> 
>  ~ > ansible localhost --connection local --become -m dnf -a 'name=htop
> state=absent'
> 
> localhost | SUCCESS => {
>     "changed": true, 
>     "failed": false, 
>     "results": [
>         "Removed: htop-2.0.2-4.fc27.x86_64"
>     ]
> }
> 
> 
> I'm going to try your dnf-2.7.4-1.git.7981.6b56aaf.fc2* packages. Will post
> an update soon.

Re: the workaround by commenting out L465 of ansible/modules/packaging/os/dnf.py, that seems to have negative implications when installing packages. With the line commented out I get an error such as:

IOError: [Errno 2] No such file or directory: '/var/cache/dnf/(path-to-an-RPM)'

Uninstall works just fine. Un-comment the line (restoring to original) and install behavior returns to normal. I know a proper fix is coming but just a heads up for anyone who may have applied the workaround as a stopgap measure.

Comment 14 Milan Zink 2017-10-19 06:57:29 UTC
Yes, L465 of ansible/modules/packaging/os/dnf.py works only with uninstaling and it's more dirty hack, than workaround.

Gonig to test bodhi packages.

Comment 15 Fedora Update System 2017-10-19 15:23:12 UTC
dnf-2.7.5-1.fc27 has been pushed to the Fedora 27 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-ee5c678e8b

Comment 16 Fedora Update System 2017-10-19 19:52:36 UTC
dnf-2.7.5-1.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-d2e8fdf5d8

Comment 17 Milan Zink 2017-10-20 04:03:06 UTC
I've tested bodhi packages. No issues. Thanks for a quick fix.

Comment 18 Fedora Update System 2017-10-21 22:21:48 UTC
dnf-2.7.5-1.fc26 has been pushed to the Fedora 26 stable repository. If problems still persist, please make note of it in this bug report.

Comment 19 Igor Gnatenko 2017-11-01 12:05:22 UTC
*** Bug 1506907 has been marked as a duplicate of this bug. ***

Comment 20 Fedora Update System 2017-11-11 02:56:55 UTC
dnf-2.7.5-1.fc27 has been pushed to the Fedora 27 stable repository. If problems still persist, please make note of it in this bug report.