Bug 1315016

Summary: dnf destroy RPM DB when package installed unsuccessfuly
Product: [Fedora] Fedora Reporter: Frantisek Hanzlik <franta>
Component: rpmAssignee: Packaging Maintenance Team <packaging-team-maint>
Status: CLOSED INSUFFICIENT_DATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 23CC: ferdnyc, franta, jsilhan, jzeleny, lkardos, mluscon, novyjindrich, packaging-team-maint, pknirsch, pnemade, vmukhame
Target Milestone: ---Flags: franta: needinfo-
Target Release: ---   
Hardware: i686   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-05-09 08:39:42 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Frantisek Hanzlik 2016-03-05 17:58:52 UTC
Description of problem:
on just installed F23/i686 I tried to install some additional packages. Installation one of them was unsuccessful, and after that, it seem as RPM DB is destroyed and unusable (size of original /var/lib/rpm/ was ~ 175 MB, after this incident was approx. 20 MB).

Version-Release number of selected component (if applicable):
dnf-1.1.6-2.fc23.noarch

What I was doing (sorry, it is in Czech, I not expect that dnf/rpm libs is broken so much):

[root@linux ~]# dnf install seamonkey thunderbird
Last metadata expiration check performed 0:56:53 ago on Sat Mar  5 16:51:43 2016.
Závislosti vyřešeny.
====================================================================================================================================================================
 Package                                  Arch                              Verze                                        Repozitář                             Vel.
====================================================================================================================================================================
Instalování:
 seamonkey                                i686                              2.39-1.fc23                                  updates                               63 M
 thunderbird                              i686                              38.6.0-3.fc23                                updates                               62 M

Shrnutí transakce
====================================================================================================================================================================
Instalovat  2 Packages

Celková velikost ke stahování: 125 M
Nainstalovaná velikost: 256 M
V pořádku [a/N]: a
Stahování balíčků:
(1/2): thunderbird-38.6.0-3.fc23.i686.rpm                                                                                           1.4 MB/s |  62 MB     00:45    
(2/2): seamonkey-2.39-1.fc23.i686.rpm                                                                                               1.3 MB/s |  63 MB     00:48    
--------------------------------------------------------------------------------------------------------------------------------------------------------------------
Celkem                                                                                                                              2.5 MB/s | 125 MB     00:50     
Spouštím kontrolu transakce
Kontrola transakce byla úspěšná
Probíhá test transakce
Test transakce byl úspěšný.
Transakce běží
  Instalování  : seamonkey-2.39-1.fc23.i686                                                                                                                     1/2 
  Instalování  : thunderbird-38.6.0-3.fc23.i686                                                                                                                 2/2 
Error unpacking rpm package thunderbird-38.6.0-3.fc23.i686
Error unpacking rpm package thunderbird-38.6.0-3.fc23.i686
error: unpacking of archive failed on file /usr/lib/thunderbird/libmozsqlite3.so;56db0e53: cpio: read
error: thunderbird-38.6.0-3.fc23.i686: install failed
  Ověřuji      : seamonkey-2.39-1.fc23.i686                                                                                                                     1/2 
thunderbird-38.6.0-3.fc23.i686 by mělo být nainstalováno, ale není!
  Ověřuji      : thunderbird-38.6.0-3.fc23.i686                                                                                                                 2/2 

Nainstalováno:
  seamonkey.i686 2.39-1.fc23                                                                                                                                        

Selhalo:
  thunderbird.i686 38.6.0-3.fc23                                                                                                                                    

Hotovo!

[root@linux ~]# dnf install thunderbird
Traceback (most recent call last):
  File "/usr/bin/dnf", line 57, in <module>
    main.user_main(sys.argv[1:], exit_code=True)
  File "/usr/lib/python3.4/site-packages/dnf/cli/main.py", line 174, in user_main
    errcode = main(args)
  File "/usr/lib/python3.4/site-packages/dnf/cli/main.py", line 60, in main
    return _main(base, args)
  File "/usr/lib/python3.4/site-packages/dnf/cli/main.py", line 112, in _main
    cli.run()
  File "/usr/lib/python3.4/site-packages/dnf/cli/cli.py", line 1094, in run
    self._process_demands()
  File "/usr/lib/python3.4/site-packages/dnf/cli/cli.py", line 810, in _process_demands
    load_available_repos=lar)
  File "/usr/lib/python3.4/site-packages/dnf/base.py", line 256, in fill_sack
    self._add_repo_to_sack(r.id)
  File "/usr/lib/python3.4/site-packages/dnf/base.py", line 134, in _add_repo_to_sack
    load_updateinfo=True)
AssertionError: Can not load repo.

[root@linux ~]# rpm --rebuilddb
chyba: rpmdbNextIterator: přeskakuji h#    1492 SHA1 digest v hlavičce:: BAD Expected(f2b9ac00df6a51b86ee17a2ef5be28dc0bac307e) != (0e3bd3b3fb5d5b5f6a5030bdb7327d76386d63b5)
chyba: rpmdb: BDB0058 page 21781: illegal page type or format
chyba: rpmdb: BDB0061 PANIC: Nepřípustný argument
chyba: rpmdb: BDB3016 Packages: pgin failed for page 21781
chyba: db5 error(-30973) from dbcursor->c_get: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
chyba: rpmdb: BDB0060 PANIC: fatal region error detected; run recovery
chyba: db5 error(-30973) from dbcursor->c_close: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
chyba: rpmdb: BDB0060 PANIC: fatal region error detected; run recovery
chyba: db5 error(-30973) from db->close: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
chyba: rpmdb: BDB1581 File handles still open at environment close
chyba: rpmdb: BDB1582 Open file handle: /var/lib/rpm/Packages
chyba: rpmdb: BDB0060 PANIC: fatal region error detected; run recovery
chyba: db5 error(-30973) from dbenv->close: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery


Additional info:
System info:
AMD Sempron 3300+ CPU
1GB DDR1 RAM (895 MB usable)
SiS 761GX graphic
2GB swap
28 GB system partition (all but /home), 11GB free
Selinux enabled

Comment 1 Michal Luscon 2016-03-14 12:39:38 UTC
RPM guys are much more smarter - they will find the root cause of the unsuccessful transaction.

Comment 2 Ľuboš Kardoš 2016-04-06 11:13:01 UTC
Is this somehow reproducible?

Comment 3 "FeRD" (Frank Dana) 2016-04-28 07:27:22 UTC
This stuff:

Error unpacking rpm package thunderbird-38.6.0-3.fc23.i686
error: unpacking of archive failed on file /usr/lib/thunderbird/libmozsqlite3.so;56db0e53: cpio: read error

...indicates that rpm failed to process the thunderbird-38.6.0-3.fc23.i686.rpm file that was just downloaded, due to a read error. That error wouldn't be caused by any rpmdb problems, AFAIK cpio is only used to process the .rpm file contents and never touches the rpmdb.) The cause of the cpio read error would be local file corruption.

Since dnf reported no errors downloading the .rpm, it seems like a strong possibility that the local filesystem is damaged, either it's become corrupted or there are underlying disk errors.

I would strongly recommend scanning that 28GB system partition and the disk it's stored on for errors. If the filesystem's damaged, it's possible that was the cause of the /var/lib/rpmdb corruption as well, not anything dnf or rpm did.

Comment 4 Ľuboš Kardoš 2016-05-09 08:39:42 UTC
No response to comment 2 (asked a month ago) and I agree with comment 3 that this seems more like file system damage. I am closing the bug. Feel free to reopen the bug if you have some new information how to reproduce this bug.