RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1893176 - dnf aborts when running update
Summary: dnf aborts when running update
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: dnf
Version: 8.2
Hardware: Unspecified
OS: Linux
medium
low
Target Milestone: rc
: 8.0
Assignee: Marek Blaha
QA Contact: Eva Mrakova
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-10-30 12:50 UTC by Alain D D Williams
Modified: 2021-11-10 09:33 UTC (History)
3 users (show)

Fixed In Version: libdnf-0.63.0-2.el8
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-11-09 19:52:16 UTC
Type: Bug
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)
tar file of dnf logs (5.29 KB, application/gzip)
2020-11-11 20:20 UTC, Alain D D Williams 		no flags Details
View All


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2021:4464 0 None None None 2021-11-09 19:52:27 UTC

Description Alain D D Williams 2020-10-30 12:50:54 UTC 
Description of problem:

I ran dnf update and got a error message.


How reproducible:

Running it again - problem does not manifest - presumably because it is installed.

Steps to Reproduce:
1.
2.
3.

Actual results:

Traceback (most recent call last):
  File "/usr/bin/dnf", line 58, in <module>
    main.user_main(sys.argv[1:], exit_code=True)
  File "/usr/lib/python3.6/site-packages/dnf/cli/main.py", line 191, in user_main
    errcode = main(args)
  File "/usr/lib/python3.6/site-packages/dnf/cli/main.py", line 65, in main
    return _main(base, args, cli_class, option_parser_class)
  File "/usr/lib/python3.6/site-packages/dnf/cli/main.py", line 98, in _main
    return cli_run(cli, base)
  File "/usr/lib/python3.6/site-packages/dnf/cli/main.py", line 122, in cli_run
    ret = resolving(cli, base)
  File "/usr/lib/python3.6/site-packages/dnf/cli/main.py", line 166, in resolving
    base.do_transaction(display=displays)
  File "/usr/lib/python3.6/site-packages/dnf/cli/cli.py", line 243, in do_transaction
    tid = super(BaseCli, self).do_transaction(display)
  File "/usr/lib/python3.6/site-packages/dnf/base.py", line 881, in do_transaction
    self._plugins.unload_removed_plugins(self.transaction)
  File "/usr/lib/python3.6/site-packages/dnf/plugin.py", line 180, in unload_removed_plugins
    files_erased.update(pkg.files)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 50: invalid start byte


I saw this on another machine that I look after.

I thought that it might be an RPM of mine that creates some files with names that are deliberately bad UTF-8 -- as a test. But I uninstalled it & reinstalled it and the reinstall worked without issue.
Comment 1 Marek Blaha 2020-11-03 07:53:44 UTC 
Yes, your RPM with incorrect file names is probably a culprit.
If I understand it correctly, to reproduce the issue your package should be removed during the transaction, but not installed at the same time (so reinstall won't work) - may be `dnf remove your-package`?
Do you happen to have dnf logs from the failed operation? It would be also helpful if you could attach the rpm with invalid file names so I could try to reproduce the issue and eventually fix this ugly traceback.
Also versions of dnf and libdnf used may help (rpm -q dnf libdnf).
Comment 2 Alain D D Williams 2020-11-11 20:20:28 UTC 
Created attachment 1728503 [details]
tar file of dnf logs

extracts of dnf log file
Comment 3 Alain D D Williams 2020-11-12 10:38:40 UTC 
I worked out why remove/reinstall did not repeat the message/error.

The files with the bad names were in a directory TestFiles ... but I had been playing around with putting more of these in /usr/share/doc/ and this increased the footprint, so the next release had them in a tar file.
These files are now in TestFiles.tar.gz - so the problem will not manifest itself again (for this package).

However: I expect that the bug is still there and will show its head when a RPM has files with 'strange' names.
Comment 4 Marek Blaha 2020-11-12 10:54:09 UTC 
Is there a chance to get the rpm with files with the bad names (the older version before moving them into tar)? The bug is indeed still there and having package witch could reproduce the error would be great.
You can send it privately to my email mblaha
I looked into the logs and unfortunately did not find anything useful there.
Comment 5 Alain D D Williams 2020-11-12 12:08:15 UTC 
That was a quick reply ... what time zone are you in ?

Unfortunately I purged a lot of stuff a couple of days ago. I will build a small RPM that contains the files with 'bad' names - easier for you and not much harder than going back through version control, etc, etc.
Comment 6 Alain D D Williams 2020-11-12 12:40:55 UTC 
I have put the test RPM up here:

https://www.phcomp.co.uk/tmp/badfilename-1.13-0.0.noarch.rpm

Please let me know when you have grabbed it & I will zap it.

Unfortunately: installing/removing it does not give me nasty messages.

I will have to go and build the whole MicroEmacs package ... although I don't see if that will trigger it more ???

I am busy for a bit - stuff that I need to do :-(
Comment 7 Marek Blaha 2020-11-12 13:23:30 UTC 
Cool, thanks for the rpm! Now I can reproduce the error (and eventually fix it):

# dnf install badfilename-1.13-0.0.noarch.rpm
# dnf remove -y badfilename
...
...
Traceback (most recent call last):
  File "/usr/bin/dnf", line 58, in <module>
    main.user_main(sys.argv[1:], exit_code=True)
  File "/usr/lib/python3.8/site-packages/dnf/cli/main.py", line 201, in user_main
    errcode = main(args)
  File "/usr/lib/python3.8/site-packages/dnf/cli/main.py", line 67, in main
    return _main(base, args, cli_class, option_parser_class)
  File "/usr/lib/python3.8/site-packages/dnf/cli/main.py", line 106, in _main
    return cli_run(cli, base)
  File "/usr/lib/python3.8/site-packages/dnf/cli/main.py", line 130, in cli_run
    ret = resolving(cli, base)
  File "/usr/lib/python3.8/site-packages/dnf/cli/main.py", line 176, in resolving
    base.do_transaction(display=displays)
  File "/usr/lib/python3.8/site-packages/dnf/cli/cli.py", line 244, in do_transaction
    tid = super(BaseCli, self).do_transaction(display)
  File "/usr/lib/python3.8/site-packages/dnf/base.py", line 886, in do_transaction
    self._plugins.unload_removed_plugins(self.transaction)
  File "/usr/lib/python3.8/site-packages/dnf/plugin.py", line 180, in unload_removed_plugins
    files_erased.update(pkg.files)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 46: invalid start byte
Comment 8 Alain D D Williams 2020-11-12 14:42:38 UTC 
Interesting: I installed/removed it with the rpm command -> no errors; you did it with dnf and got errors.

Good luck squashing the bug.
Comment 9 Marek Blaha 2021-05-21 11:41:16 UTC 
Patch fixing the issue: https://github.com/rpm-software-management/libdnf/pull/1237
Test: https://github.com/rpm-software-management/ci-dnf-stack/pull/999/files
Comment 15 errata-xmlrpc 2021-11-09 19:52:16 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: dnf security and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:4464
Note You need to log in before you can comment on or make changes to this bug.