Bug 1332099

Summary: Installing packages produces a backtrace, installation goes ok
Product: [Fedora] Fedora Reporter: E Järvinen <ejarvinen>
Component: dnfAssignee: Martin Hatina <mhatina>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: low Docs Contact:
Priority: unspecified    
Version: 26CC: mhatina, mluscon, mmraka, ondrej.sojka, packaging-team-maint, pnemade, vmukhame
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: dnf-2.6.2-1.fc26 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-07-25 16:56:22 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description E Järvinen 2016-05-02 08:37:32 UTC
Description of problem:

Installing new packages (or updating, doesn't seem to matter) produces a backtrace as the last step.

Version-Release number of selected component (if applicable):

DNF version: 1.1.8, fully updated F23 on ARM (RPi 3)

How reproducible:

Just install anything with dnf or update the system.

Steps to Reproduce:
1. dnf install <anything installable here>
2. wait
3. backtrace follows

Actual results:

A backtrace

Expected results:

No backtrace

Additional info:

Traceback (most recent call last):
  File "/usr/bin/dnf", line 58, in <module>
    main.user_main(sys.argv[1:], exit_code=True)
  File "/usr/lib/python3.4/site-packages/dnf/cli/main.py", line 174, in user_main
    errcode = main(args)
  File "/usr/lib/python3.4/site-packages/dnf/cli/main.py", line 60, in main
    return _main(base, args)
  File "/usr/lib/python3.4/site-packages/dnf/base.py", line 100, in __exit__
    self.close()
  File "/usr/lib/python3.4/site-packages/dnf/base.py", line 287, in close
    self._tempfile_persistor.get_saved_tempfiles())
  File "/usr/lib/python3.4/site-packages/dnf/persistor.py", line 483, in get_saved_tempfiles
    return self._get_json_db(self.db_path)
  File "/usr/lib/python3.4/site-packages/dnf/persistor.py", line 425, in _get_json_db
    default = json.loads(content)
  File "/usr/lib/python3.4/json/__init__.py", line 318, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.4/json/decoder.py", line 343, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.4/json/decoder.py", line 359, in raw_decode
    obj, end = self.scan_once(s, idx)
ValueError: Expecting ',' delimiter: line 1 column 52935 (char 52934)

Comment 1 Honza Silhan 2016-05-02 11:46:53 UTC
To fix the problem:

# rm /var/cache/dnf/tempfiles.json

DNF should better handle this error and advice the user the fix. I'd rather not automatically delete the file by DNF, because other json file contain more valuable information i.e. groups metadata.

Comment 2 E Järvinen 2016-05-02 12:16:53 UTC
Just ran dnf update after removing the offending file, fix works, thank you!

FWIW, tempfiles.json was pretty old:

-rw-rw-r--. 1 root root 52934  2. 4. 16:47 /var/cache/dnf/tempfiles.json

(that's 2nd of April)

Comment 3 Fedora Admin XMLRPC Client 2016-07-08 09:39:54 UTC
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Comment 4 Fedora End Of Life 2016-11-25 07:31:50 UTC
This message is a reminder that Fedora 23 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 23. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '23'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 23 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 5 Michael Mráka 2016-11-28 15:05:36 UTC
Still valid in rawhide - dnf-2.0.0-0.rc1.4.fc26.noarch
Also present on x84_64 arch.
.

# sed -i 's/,/ /' /var/cache/dnf/tempfiles.json 
# dnf install zip
Last metadata expiration check: 0:43:13 ago on Mon Nov 28 15:19:42 2016 CET.
Dependencies resolved.
==========================================================================================================
 Package              Arch                    Version                      Repository                Size
==========================================================================================================
Installing:
 zip                  x86_64                  3.0-16.fc24                  rawhide                  271 k
...
Installed:
  zip.x86_64 3.0-16.fc24                                                                                  

Complete!
Traceback (most recent call last):
  File "/usr/bin/dnf", line 58, in <module>
    main.user_main(sys.argv[1:], exit_code=True)
  File "/usr/lib/python3.5/site-packages/dnf/cli/main.py", line 170, in user_main
    errcode = main(args)
  File "/usr/lib/python3.5/site-packages/dnf/cli/main.py", line 60, in main
    return _main(base, args)
  File "/usr/lib/python3.5/site-packages/dnf/base.py", line 100, in __exit__
    self.close()
  File "/usr/lib/python3.5/site-packages/dnf/base.py", line 306, in close
    self._tempfile_persistor.get_saved_tempfiles())
  File "/usr/lib/python3.5/site-packages/dnf/persistor.py", line 486, in get_saved_tempfiles
    return self._get_json_db(self.db_path)
  File "/usr/lib/python3.5/site-packages/dnf/persistor.py", line 428, in _get_json_db
    default = json.loads(content)
  File "/usr/lib64/python3.5/json/__init__.py", line 319, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python3.5/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib64/python3.5/json/decoder.py", line 355, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 1 column 96 (char 95)

Comment 6 Fedora End Of Life 2017-02-28 09:57:48 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 26 development cycle.
Changing version to '26'.

Comment 7 Ondřej Sojka 2017-07-20 13:16:31 UTC
What should be the right fix? Now, it doesn't print out the whole traceback, but catches the exception and prints just the short message ("Expecting ',' delimiter: line 1 column 96 (char 95)"). This was added in https://github.com/rpm-software-management/dnf/pull/859.

Then, it reinitializes the file to an empty DB. That happens with both expired_repos.json and tempfiles.json.

The short message is confusing to the user: user installs package, dnf prints out "Success!" and then spits out something like "Expecting ',' delimiter: line 1 column 96 (char 95)" without any context. The message should be moved to a lower loglevel (debug) or be printed with more context about what just happened.

Comment 8 Fedora Update System 2017-07-24 14:52:24 UTC
libdnf-0.9.3-1.fc26 dnf-plugins-core-2.1.3-1.fc26 dnf-2.6.2-1.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-6f4c06b2d7

Comment 9 Fedora Update System 2017-07-25 04:27:25 UTC
dnf-2.6.2-1.fc26, dnf-plugins-core-2.1.3-1.fc26, libdnf-0.9.3-1.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-6f4c06b2d7

Comment 10 Fedora Update System 2017-07-25 16:56:22 UTC
dnf-2.6.2-1.fc26, dnf-plugins-core-2.1.3-1.fc26, libdnf-0.9.3-1.fc26 has been pushed to the Fedora 26 stable repository. If problems still persist, please make note of it in this bug report.