Bug 1893176

Summary: dnf aborts when running update
Product: Red Hat Enterprise Linux 8 Reporter: Alain D D Williams <addw>
Component: dnfAssignee: Marek Blaha <mblaha>
Status: CLOSED ERRATA QA Contact: Eva Mrakova <emrakova>
Severity: low Docs Contact:
Priority: medium    
Version: 8.2CC: james.antill, mblaha, pkratoch
Target Milestone: rcKeywords: Triaged
Target Release: 8.0   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: libdnf-0.63.0-2.el8 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-11-09 19:52:16 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
tar file of dnf logs none

Description Alain D D Williams 2020-10-30 12:50:54 UTC
Description of problem:

I ran dnf update and got a error message.


How reproducible:

Running it again - problem does not manifest - presumably because it is installed.

Steps to Reproduce:
1.
2.
3.

Actual results:

Traceback (most recent call last):
  File "/usr/bin/dnf", line 58, in <module>
    main.user_main(sys.argv[1:], exit_code=True)
  File "/usr/lib/python3.6/site-packages/dnf/cli/main.py", line 191, in user_main
    errcode = main(args)
  File "/usr/lib/python3.6/site-packages/dnf/cli/main.py", line 65, in main
    return _main(base, args, cli_class, option_parser_class)
  File "/usr/lib/python3.6/site-packages/dnf/cli/main.py", line 98, in _main
    return cli_run(cli, base)
  File "/usr/lib/python3.6/site-packages/dnf/cli/main.py", line 122, in cli_run
    ret = resolving(cli, base)
  File "/usr/lib/python3.6/site-packages/dnf/cli/main.py", line 166, in resolving
    base.do_transaction(display=displays)
  File "/usr/lib/python3.6/site-packages/dnf/cli/cli.py", line 243, in do_transaction
    tid = super(BaseCli, self).do_transaction(display)
  File "/usr/lib/python3.6/site-packages/dnf/base.py", line 881, in do_transaction
    self._plugins.unload_removed_plugins(self.transaction)
  File "/usr/lib/python3.6/site-packages/dnf/plugin.py", line 180, in unload_removed_plugins
    files_erased.update(pkg.files)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 50: invalid start byte


I saw this on another machine that I look after.

I thought that it might be an RPM of mine that creates some files with names that are deliberately bad UTF-8 -- as a test. But I uninstalled it & reinstalled it and the reinstall worked without issue.

Comment 1 Marek Blaha 2020-11-03 07:53:44 UTC
Yes, your RPM with incorrect file names is probably a culprit.
If I understand it correctly, to reproduce the issue your package should be removed during the transaction, but not installed at the same time (so reinstall won't work) - may be `dnf remove your-package`?
Do you happen to have dnf logs from the failed operation? It would be also helpful if you could attach the rpm with invalid file names so I could try to reproduce the issue and eventually fix this ugly traceback.
Also versions of dnf and libdnf used may help (rpm -q dnf libdnf).

Comment 2 Alain D D Williams 2020-11-11 20:20:28 UTC
Created attachment 1728503 [details]
tar file of dnf logs

extracts of dnf log file

Comment 3 Alain D D Williams 2020-11-12 10:38:40 UTC
I worked out why remove/reinstall did not repeat the message/error.

The files with the bad names were in a directory TestFiles ... but I had been playing around with putting more of these in /usr/share/doc/ and this increased the footprint, so the next release had them in a tar file.
These files are now in TestFiles.tar.gz - so the problem will not manifest itself again (for this package).

However: I expect that the bug is still there and will show its head when a RPM has files with 'strange' names.

Comment 4 Marek Blaha 2020-11-12 10:54:09 UTC
Is there a chance to get the rpm with files with the bad names (the older version before moving them into tar)? The bug is indeed still there and having package witch could reproduce the error would be great.
You can send it privately to my email mblaha
I looked into the logs and unfortunately did not find anything useful there.

Comment 5 Alain D D Williams 2020-11-12 12:08:15 UTC
That was a quick reply ... what time zone are you in ?

Unfortunately I purged a lot of stuff a couple of days ago. I will build a small RPM that contains the files with 'bad' names - easier for you and not much harder than going back through version control, etc, etc.

Comment 6 Alain D D Williams 2020-11-12 12:40:55 UTC
I have put the test RPM up here:

https://www.phcomp.co.uk/tmp/badfilename-1.13-0.0.noarch.rpm

Please let me know when you have grabbed it & I will zap it.

Unfortunately: installing/removing it does not give me nasty messages.

I will have to go and build the whole MicroEmacs package ... although I don't see if that will trigger it more ???

I am busy for a bit - stuff that I need to do :-(

Comment 7 Marek Blaha 2020-11-12 13:23:30 UTC
Cool, thanks for the rpm! Now I can reproduce the error (and eventually fix it):

# dnf install badfilename-1.13-0.0.noarch.rpm
# dnf remove -y badfilename
...
...
Traceback (most recent call last):
  File "/usr/bin/dnf", line 58, in <module>
    main.user_main(sys.argv[1:], exit_code=True)
  File "/usr/lib/python3.8/site-packages/dnf/cli/main.py", line 201, in user_main
    errcode = main(args)
  File "/usr/lib/python3.8/site-packages/dnf/cli/main.py", line 67, in main
    return _main(base, args, cli_class, option_parser_class)
  File "/usr/lib/python3.8/site-packages/dnf/cli/main.py", line 106, in _main
    return cli_run(cli, base)
  File "/usr/lib/python3.8/site-packages/dnf/cli/main.py", line 130, in cli_run
    ret = resolving(cli, base)
  File "/usr/lib/python3.8/site-packages/dnf/cli/main.py", line 176, in resolving
    base.do_transaction(display=displays)
  File "/usr/lib/python3.8/site-packages/dnf/cli/cli.py", line 244, in do_transaction
    tid = super(BaseCli, self).do_transaction(display)
  File "/usr/lib/python3.8/site-packages/dnf/base.py", line 886, in do_transaction
    self._plugins.unload_removed_plugins(self.transaction)
  File "/usr/lib/python3.8/site-packages/dnf/plugin.py", line 180, in unload_removed_plugins
    files_erased.update(pkg.files)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 46: invalid start byte

Comment 8 Alain D D Williams 2020-11-12 14:42:38 UTC
Interesting: I installed/removed it with the rpm command -> no errors; you did it with dnf and got errors.

Good luck squashing the bug.

Comment 15 errata-xmlrpc 2021-11-09 19:52:16 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: dnf security and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:4464