Bug 1669824 - “database disk image is malformed”
Summary: “database disk image is malformed”
Keywords:
Status: CLOSED EOL
Alias: None
Product: Fedora
Classification: Fedora
Component: dnf
Version: 30
Hardware: x86_64
OS: Linux
high
unspecified
Target Milestone: ---
Assignee: rpm-software-management
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-01-27 11:14 UTC by Martin Ueding
Modified: 2021-06-07 10:52 UTC (History)
14 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-05-26 18:19:03 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Martin Ueding 2019-01-27 11:14:49 UTC
Description of problem:

    I might have run out of space on the `/` drive at some point causing a
    problem with the `/var/lib/dnf/history.sqlite`. At some point I could not
    install any new updates because the SQLite database has been corrupted:

        # env LC_ALL=C dnf update
    Last metadata expiration check: 0:18:57 ago on Wed Jan 23 16:16:14 2019.
    Traceback (most recent call last):
      File "/usr/bin/dnf", line 58, in <module>
        main.user_main(sys.argv[1:], exit_code=True)
      File "/usr/lib/python3.7/site-packages/dnf/cli/main.py", line 179, in user_main
        errcode = main(args)
      File "/usr/lib/python3.7/site-packages/dnf/cli/main.py", line 64, in main
        return _main(base, args, cli_class, option_parser_class)
      File "/usr/lib/python3.7/site-packages/dnf/cli/main.py", line 99, in _main
        return cli_run(cli, base)
      File "/usr/lib/python3.7/site-packages/dnf/cli/main.py", line 123, in cli_run
        ret = resolving(cli, base)
      File "/usr/lib/python3.7/site-packages/dnf/cli/main.py", line 146, in resolving
        base.resolve(cli.demands.allow_erasing)
      File "/usr/lib/python3.7/site-packages/dnf/base.py", line 760, in resolve
        self._transaction = self._goal2transaction(goal)
      File "/usr/lib/python3.7/site-packages/dnf/base.py", line 681, in _goal2transaction
        ts.add_upgrade(pkg, upgraded, obs)
      File "/usr/lib/python3.7/site-packages/dnf/db/group.py", line 269, in add_upgrade
        ti_new = self.new(new, libdnf.transaction.TransactionItemAction_UPGRADE)
      File "/usr/lib/python3.7/site-packages/dnf/db/group.py", line 222, in new
        reason = self.get_reason(pkg)
      File "/usr/lib/python3.7/site-packages/dnf/db/group.py", line 237, in get_reason
        return self.history.swdb.resolveRPMTransactionItemReason(pkg.name, pkg.arch, -1)
      File "/usr/lib64/python3.7/site-packages/libdnf/transaction.py", line 788, in resolveRPMTransactionItemReason
        return _transaction.Swdb_resolveRPMTransactionItemReason(self, name, arch, maxTransactionId)
    RuntimeError: Step: database disk image is malformed in

            SELECT
                ti.action as action,
                ti.reason as reason
            FROM
                trans_item ti
            JOIN
                trans t ON ti.trans_id = t.id
            JOIN
                rpm i USING (item_id)
            WHERE
                t.state = 1
                /* see comment in TransactionItem.hpp - TransactionItemAction */
                AND ti.action not in (3, 5, 7, 10)
                AND i.name = 'python2-rpkg'
                AND i.arch = 'noarch'
            ORDER BY
                ti.trans_id DESC
            LIMIT 1

    I have tried running a few commands:

    - dnf clean all
    - dnf makecache
    - rpm --buildddb

    In general errors with SQLite databases might be cured with the following,
    so I have tried that.

        sqlite3 history.sqlite ".dump" | sqlite3 history-new.db

    This did not have the desired effect at first, I presume that SELinux
    labels were defunkt. I used `restorecon` to relabel the files. After a
    reboot I dumped the history database into a new file and replaced the old
    one with that. Then it started to work again.

    I asked for help at
    https://unix.stackexchange.com/questions/496248/database-disk-image-is-malformed-with-dnf-on-fedora-29?noredirect=1#comment914249_496248
    but was just asked to file a bug here.

Version-Release number of selected component (if applicable):

    Fedora 29
    dnf 4.0.9

How reproducible:

    I don't know, unfortunately.

Comment 1 Daniel Mach 2019-01-28 07:38:53 UTC
I don't think it's selinux related (restorecon).
It's more likely a corrupted sqlite database
and you fixed it by dump & restore.
Running out of space in / looks like the root cause.
System can get into an undefined state pretty easily.

If yous still have it, could you attach the broken
/var/lib/dnf/history.sqlite file to this bug?
If it's too big or if you don't want to attach it to a public bugzilla,
you can either email me (dmach) the file or link to it.

Comment 2 Martin Ueding 2019-01-28 15:45:44 UTC
I have just re-checked that this is the broken version of the `history.sqlite` file and uploaded it here:

https://uni-bonn.sciebo.de/s/2IyonVck9y2leGp

The link should be good until 2019-12-31.

Comment 3 Daniel Mach 2019-03-01 07:27:36 UTC
I ran:
PRAGMA integrity_check;
row 50351 missing from index rpm_name
row 52357 missing from index rpm_name
row 52739 missing from index rpm_name
row 56528 missing from index rpm_name
row 61235 missing from index rpm_name
row 67482 missing from index rpm_name

To fix that, I ran:
REINDEX;

This might help us to recover from this state.
Still not sure what the root cause is.

We need to read through https://www.sqlite.org/howtocorrupt.html
and make sure we don't do any of those steps in libdnf or in combination with PackageKit.

Comment 4 space88man 2019-03-08 02:44:55 UTC
Data point: encountered the same issue with a different error message

SQLite version 3.26.0 2018-12-01 12:34:55
Enter ".help" for usage hints.
sqlite> PRAGMA integrity_check;
*** in database main ***
On tree page 2887 cell 223: invalid page number 21365001
Multiple uses for byte 2492 of page 2887
Error: database disk image is malformed
sqlite> 

dumping and undumping fixed the issue

Comment 5 Lumír Balhar 2019-04-06 15:37:16 UTC
I encountered the same issue and the reason was probably full filesystem on /.

I solved it by dump/restore of /usr/lib/dnf/history.sqlite.

Comment 6 Dr. David Alan Gilbert 2019-04-13 11:52:34 UTC
Just hit this on f30, I've not knowingly filled up my /

[dg@major ~]$ sudo dnf remove v8-devel
terminate called after throwing an instance of 'SQLite3::LibException'
  what():  Step: database disk image is malformed in

        SELECT
            ti.action as action,
            ti.reason as reason
        FROM
            trans_item ti
        JOIN
            trans t ON ti.trans_id = t.id
        JOIN
            rpm i USING (item_id)
        WHERE
            t.state = 1
            /* see comment in TransactionItem.hpp - TransactionItemAction */
            AND ti.action not in (3, 5, 7, 10)
            AND i.name = 'grub2-pc-modules'
            AND i.arch = 'noarch'
        ORDER BY
            ti.trans_id DESC
        LIMIT 1
    
Aborted

Comment 7 Peter Bieringer 2019-05-03 05:53:24 UTC
Hit me also on F30 during update of packages

INSERT OR REPLACE INTO item_replaced_by VALUES (30730, 30729)

Removed all files

rm -rf /var/cache/dnf/* /var/lib/dnf/* /var/lib/yum/*

and run again

dnf update

This solved the issue

Comment 9 Ben Cotton 2020-04-30 20:27:03 UTC
This message is a reminder that Fedora 30 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 30 on 2020-05-26.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '30'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 30 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 10 Ben Cotton 2020-05-26 18:19:03 UTC
Fedora 30 changed to end-of-life (EOL) status on 2020-05-26. Fedora 30 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.

Comment 11 José Luís 2021-06-07 08:59:47 UTC
@Ben I'm having the same issue right now in fedora 34, I run dnf update after installation and got this error, now every time I tried to install or run an update shows this error:

Total                                                                                                                                   246 kB/s | 1.3 MB     00:05     
Running transaction check
error: SELECT DISTINCT key FROM 'Requirename' ORDER BY key: 11: database disk image is malformed
...

  Installing       : newt-0.52.21-9.fc34.x86_64                                                                                                                     2/5 
error: INSERT INTO 'Requirename' VALUES(?, ?, ?): 11: database disk image is malformed
error: INSERT INTO 'Requirename' VALUES(?, ?, ?): 11: database disk image is malformed
error: INSERT INTO 'Requirename' VALUES(?, ?, ?): 11: database disk image is malformed
error: INSERT INTO 'Requirename' VALUES(?, ?, ?): 11: database disk image is malformed

----------------------------
dnf log file shows:

  File "/usr/lib/python3.9/site-packages/dnf/cli/main.py", line 130, in cli_run
    ret = resolving(cli, base)
  File "/usr/lib/python3.9/site-packages/dnf/cli/main.py", line 176, in resolving
    base.do_transaction(display=displays)
  File "/usr/lib/python3.9/site-packages/dnf/cli/cli.py", line 264, in do_transaction
    raise dnf.exceptions.Error(_('Transaction failed'))
dnf.exceptions.Error: Transaction failed
2021-06-07T09:48:31+0100 CRITICAL Error: Transaction failed

Comment 12 Daniel Mach 2021-06-07 10:52:42 UTC
José,
the issue you have is a different one than this bug.
It is most likely related to RPM database rather than anything in DNF.
If you have more details than what you have reported or ideally a reproducer, could you file a new bug on RPM?


Note You need to log in before you can comment on or make changes to this bug.