Bug 1502106 - dnf packages update breaks package removal via ansible
Summary: dnf packages update breaks package removal via ansible
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: dnf   
(Show other bugs)
Version: 26
Hardware: Unspecified
OS: Linux
unspecified
low
Target Milestone: ---
Assignee: Jaroslav Mracek
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Keywords: Triaged
: 1506907 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-10-14 08:59 UTC by Milan Zink
Modified: 2017-11-11 02:56 UTC (History)
10 users (show)

Fixed In Version: dnf-2.7.5-1.fc26 dnf-2.7.5-1.fc27
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-10-21 22:21:48 UTC
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

Description Milan Zink 2017-10-14 08:59:17 UTC
Description of problem:

Latest upgrade of dnf* packages (probably python2-dnf) breaks package removal via ansible dnf/package module 


Version-Release number of selected component (if applicable):

Working versions
dnf-conf-2.5.1-1.fc26.noarch
dnf-plugins-core-2.1.1-1.fc26.noarch
dnf-utils-2.1.1-1.fc26.noarch
dnf-yum-2.5.1-1.fc26.noarch
dnf-2.5.1-1.fc26.noarch
libdnf-0.9.1-1.fc26.x86_64
python2-dnf-2.5.1-1.fc26.noarch
python3-dnf-plugins-core-2.1.1-1.fc26.noarch
python3-dnf-plugins-extras-common-2.0.1-1.fc26.noarch
python3-dnf-plugin-system-upgrade-2.0.1-1.fc26.noarch
python3-dnf-2.5.1-1.fc26.noarch

Problematic version:
python2-dnf.noarch                                  2.7.3-1.fc26         updates
python2-hawkey.x86_64                               0.11.0-1.fc26        updates
python3-dnf.noarch                                  2.7.3-1.fc26         updates
python3-dnf-plugins-core.noarch                     2.1.5-1.fc26         updates
    python3-dnf-plugins-extras-common.noarch        2.0.1-1.fc26         @fedora
python3-dnf-plugins-extras-common.noarch            2.0.3-1.fc26         updates
python3-dnf-plugins-extras-common.noarch            2.0.3-1.fc26         updates
python3-dnf-plugin-system-upgrade.noarch            2.0.3-1.fc26         updates
python3-hawkey.x86_64                               0.11.0-1.fc26        updates



How reproducible:

Try to remove package with ansible.

Working with older packages:

$ ansible --connection local localhost --become -m dnf -a "name=htop state=absent"
 [WARNING]: Could not match supplied host pattern, ignoring: all

 [WARNING]: provided hosts list is empty, only localhost is available

localhost | SUCCESS => {
    "changed": true, 
    "failed": false, 
    "results": [
        "Removed: htop-2.0.2-2.fc26.x86_64"
    ]
}
$ rpm -qa | grep python2-dnf
python2-dnf-2.5.1-1.fc26.noarch


Not working with latest packages:
$ ansible --connection local localhost --become -m dnf -a "name=htop state=absent"
 [WARNING]: Could not match supplied host pattern, ignoring: all

 [WARNING]: provided hosts list is empty, only localhost is available

localhost | FAILED! => {
    "changed": false, 
    "failed": true, 
    "module_stderr": "No handlers could be found for logger \"dnf\"\nTraceback (most recent call last):\n  File \"/tmp/ansible_CwV96B/ansible_module_dnf.py\", line 534, in <module>\n    main()\n  File \"/tmp/ansible_CwV96B/ansible_module_dnf.py\", line 530, in main\n    ensure(module, base, params['state'], params['name'], params['autoremove'])\n  File \"/tmp/ansible_CwV96B/ansible_module_dnf.py\", line 465, in ensure\n    base.download_packages(base.transaction.install_set)\n  File \"/usr/lib/python2.7/site-packages/dnf/base.py\", line 1025, in download_packages\n    remote_pkgs, local_repository_pkgs = self._select_remote_pkgs(pkglist)\n  File \"/usr/lib/python2.7/site-packages/dnf/base.py\", line 2300, in _select_remote_pkgs\n    _(\"Some packages from local repository have incorrect checksum\"))\ndnf.exceptions.Error: <exception str() failed>\n", 
    "module_stdout": "", 
    "msg": "MODULE FAILURE", 
    "rc": 0
}
$ rpm -qa | grep python2-dnf
python2-dnf-2.7.3-1.fc26.noarch



Steps to Reproduce:
1. try remove package via ansible using dnf or package module
2. works with python2-dnf-2.5.X
3. not working with python2-dnf-2.7.X


Actual results:
module_stderr": "No handlers could be found for logger \"dnf\"\nTraceback (most recent call last):\n  File \"/tmp/ansible_CwV96B/ansible_module_dnf.py\", line 534, in <module>\n    main()\n  File \"/tmp/ansible_CwV96B/ansible_module_dnf.py\", line 530, in main\n    ensure(module, base, params['state'], params['name'], params['autoremove'])\n  File \"/tmp/ansible_CwV96B/ansible_module_dnf.py\", line 465, in ensure\n    base.download_packages(base.transaction.install_set)\n  File \"/usr/lib/python2.7/site-packages/dnf/base.py\", line 1025, in download_packages\n    remote_pkgs, local_repository_pkgs = self._select_remote_pkgs(pkglist)\n  File \"/usr/lib/python2.7/site-packages/dnf/base.py\", line 2300, in _select_remote_pkgs\n    _(\"Some packages from local repository have incorrect checksum\"))\ndnf.exceptions.Error: <exception str() failed>\n", 
    "module_stdout": "", 
    "msg": "MODULE FAILURE", 
    "rc": 0
}


Expected results:
    "changed": true, 
    "failed": false, 
    "results": [
        "Removed: htop-2.0.2-2.fc26.x86_64"
    ]


Additional info:
Install package works. Problem seems to be only with package removal.

Comment 1 Igor Gnatenko 2017-10-14 09:00:54 UTC
This should be fixed by https://github.com/rpm-software-management/dnf/pull/946

Comment 2 Milan Zink 2017-10-14 12:33:30 UTC
I've tried to replace dnf/base.py by file take from upstream/github. But still getting the same error.

Comment 3 Jaroslav Mracek 2017-10-17 06:53:17 UTC
I do not understand some part from report:

It is reported, that it happens for removal operation, but it fails on check of checksum of some packages from local repository (this is new feature). It means that base.transaction.install_set was not empty (for removal ?).

Additionally why some local repositories have packages with incorrect checksums?

Comment 4 Milan Zink 2017-10-17 07:20:19 UTC
Yes, it's complaining about checksum. But it should not. Works w/o issues with older python-dnf package version. 

Can I provide some more details? 

I have some local packages installed (like bluejeans), but why this should affect removal of htop package?

Comment 5 Jaroslav Mracek 2017-10-17 07:40:50 UTC
In ansible modules/packaging/os/dnf.py line 465 
```
base.download_packages(base.transaction.install_set)
```

please can you investigate content of base.transaction.install_set like
"print(base.transaction.install_set)"

Probably it can reveal the problem here.

Comment 6 Jaroslav Mracek 2017-10-17 08:49:05 UTC
I created a copr repo ("dnf copr enable jmracek/test") that have DNF (dnf-2.7.4-1.git.7981.1ce857e.fc2*) with patch that should solve the issue. Please could you test it? If it will not work I will create version for discover the primary cause.

Comment 7 Jaroslav Mracek 2017-10-17 10:01:23 UTC
The version with patch is dnf-2.7.4-1.git.7981.6b56aaf.fc2*

Comment 8 Milan Zink 2017-10-17 10:14:52 UTC
print(base.transaction.install_set) gives 

"module_stdout": "set([])\n"

base.transaction.install_set seems to be empty.

BTW: Simple comment out line  
465         #base.download_packages(base.transaction.install_set)
is a workaround:

 ~ > ansible localhost --connection local --become -m dnf -a 'name=htop state=absent'

localhost | SUCCESS => {
    "changed": true, 
    "failed": false, 
    "results": [
        "Removed: htop-2.0.2-4.fc27.x86_64"
    ]
}


I'm going to try your dnf-2.7.4-1.git.7981.6b56aaf.fc2* packages. Will post an update soon.

Comment 9 Milan Zink 2017-10-17 10:18:38 UTC
No issues with your testing packages: dnf-2.7.4-1.git.7981.6b56aaf.fc2*


 ~ > ansible localhost --connection local --become -m dnf -a 'name=htop state=absent'
 [WARNING]: Could not match supplied host pattern, ignoring: all

 [WARNING]: provided hosts list is empty, only localhost is available

localhost | SUCCESS => {
    "changed": true, 
    "failed": false, 
    "results": [
        "Removed: htop-2.0.2-4.fc27.x86_64"
    ]
}

Works as expected.

Comment 10 Jaroslav Mracek 2017-10-17 11:14:48 UTC
Here is a link with the patch https://github.com/rpm-software-management/dnf/pull/956.

Comment 11 Fedora Update System 2017-10-18 19:07:18 UTC
dnf-2.7.5-1.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-d2e8fdf5d8

Comment 12 Fedora Update System 2017-10-18 19:07:43 UTC
dnf-2.7.5-1.fc27 has been submitted as an update to Fedora 27. https://bodhi.fedoraproject.org/updates/FEDORA-2017-ee5c678e8b

Comment 13 Grant Cohoe 2017-10-18 21:17:50 UTC
(In reply to Milan Zink from comment #8)
> print(base.transaction.install_set) gives 
> 
> "module_stdout": "set([])\n"
> 
> base.transaction.install_set seems to be empty.
> 
> BTW: Simple comment out line  
> 465         #base.download_packages(base.transaction.install_set)
> is a workaround:
> 
>  ~ > ansible localhost --connection local --become -m dnf -a 'name=htop
> state=absent'
> 
> localhost | SUCCESS => {
>     "changed": true, 
>     "failed": false, 
>     "results": [
>         "Removed: htop-2.0.2-4.fc27.x86_64"
>     ]
> }
> 
> 
> I'm going to try your dnf-2.7.4-1.git.7981.6b56aaf.fc2* packages. Will post
> an update soon.

Re: the workaround by commenting out L465 of ansible/modules/packaging/os/dnf.py, that seems to have negative implications when installing packages. With the line commented out I get an error such as:

IOError: [Errno 2] No such file or directory: '/var/cache/dnf/(path-to-an-RPM)'

Uninstall works just fine. Un-comment the line (restoring to original) and install behavior returns to normal. I know a proper fix is coming but just a heads up for anyone who may have applied the workaround as a stopgap measure.

Comment 14 Milan Zink 2017-10-19 06:57:29 UTC
Yes, L465 of ansible/modules/packaging/os/dnf.py works only with uninstaling and it's more dirty hack, than workaround.

Gonig to test bodhi packages.

Comment 15 Fedora Update System 2017-10-19 15:23:12 UTC
dnf-2.7.5-1.fc27 has been pushed to the Fedora 27 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-ee5c678e8b

Comment 16 Fedora Update System 2017-10-19 19:52:36 UTC
dnf-2.7.5-1.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-d2e8fdf5d8

Comment 17 Milan Zink 2017-10-20 04:03:06 UTC
I've tested bodhi packages. No issues. Thanks for a quick fix.

Comment 18 Fedora Update System 2017-10-21 22:21:48 UTC
dnf-2.7.5-1.fc26 has been pushed to the Fedora 26 stable repository. If problems still persist, please make note of it in this bug report.

Comment 19 Igor Gnatenko 2017-11-01 12:05:22 UTC
*** Bug 1506907 has been marked as a duplicate of this bug. ***

Comment 20 Fedora Update System 2017-11-11 02:56:55 UTC
dnf-2.7.5-1.fc27 has been pushed to the Fedora 27 stable repository. If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.