Bug 2175539

Summary: rpmdb_migrate crashes on every boot
Product: [Fedora] Fedora Reporter: Michael Catanzaro <mcatanza>
Component: rpmAssignee: Packaging Maintenance Team <packaging-team-maint>
Status: CLOSED DUPLICATE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 37CC: igor.raits, mdomonko, ngompa13, packaging-team-maint, pmatilai, vmukhame
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-03-07 20:52:45 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Michael Catanzaro 2023-03-05 17:31:14 UTC
Description of problem: After upgrading from F37 to F38, I see the rpmdb-migrate script crashes once on every boot:

$ systemctl status rpmdb-migrate
× rpmdb-migrate.service - RPM database migration to /usr
     Loaded: loaded (/usr/lib/systemd/system/rpmdb-migrate.service; enabled; preset: enabled)
     Active: failed (Result: signal) since Sun 2023-03-05 11:12:02 CST; 1min 23s ago
    Process: 1516 ExecStart=/usr/lib/rpm/rpmdb_migrate (code=killed, signal=SEGV)
   Main PID: 1516 (code=killed, signal=SEGV)
        CPU: 1ms

Mar 05 11:12:02 chargestone-cave systemd[1]: Starting rpmdb-migrate.service - RPM database migration to /usr...
Mar 05 11:12:02 chargestone-cave systemd[1]: rpmdb-migrate.service: Main process exited, code=killed, status=11/SEGV
Mar 05 11:12:02 chargestone-cave systemd[1]: rpmdb-migrate.service: Failed with result 'signal'.
Mar 05 11:12:02 chargestone-cave systemd[1]: Failed to start rpmdb-migrate.service - RPM database migration to /usr.


Version-Release number of selected component (if applicable): rpm-4.18.0-10.fc38


How reproducible: Always


Steps to Reproduce:
1. Boot system

Actual results: 'systemctl status rpmdb-migrate.service' reports the crash


Expected results: No crash


Additional info: My rpmdb still exists in /var/lib/rpm even though 'rpm --eval %_dbpath' returns /usr/lib/sysimage/rpm

For some reason, this process does not leave a core dump when it crashes, so I don't know how to get a backtrace. At first I thought there was no backtrace in coredumpctl because it happens in early boot, but even when running manually it does not produce a core dump in coredumpctl:

$ sudo -i
# /usr/lib/rpm/rpmdb_migrate
Segmentation fault

To try to debug further, I ran:

# gdb rpm
(gdb) r --verbose --rebuilddb

expecting this to crash, but instead the migration succeeded, my /var/lib/rpm is gone, and now I guess I'll be unable to reproduce the bug again going forward.

Comment 1 Michael Catanzaro 2023-03-05 17:36:55 UTC
Oh one more thing: I also tried:

$ sudo /usr/lib/rpm/rpmdb_migrate

and this did crash, but the backtrace was useless as it pointed to sudo killing itself with SIGSEGV to ensure it dies with the same signal as the script that it was running. :(

Comment 2 Neal Gompa 2023-03-06 15:14:57 UTC
rpmdb_migrate isn't very complex: https://src.fedoraproject.org/rpms/rpm/blob/rawhide/f/rpmdb_migrate

There are two potential points to crash:

* rpm --eval '%_dbpath'
* rpm --verbose --rebuilddb

Other than that, it does nothing special and crashes are out of my realm of understanding. :/

Comment 3 Panu Matilainen 2023-03-07 12:44:22 UTC
Sure, I only CC'd you for awareness. The peculiar thing is it didn't crash when rebuilding outside that script.

I doubt it has anything to do with this, but the "canonical" way of rebuilding the rpmdb is "rpmdb --rebuilddb" (note the command). This *may* matter with SELinux transitions and whatnot, that's a complicated and fragile puzzle from what I've seen.

Comment 4 Michael Catanzaro 2023-03-07 20:52:45 UTC

*** This bug has been marked as a duplicate of bug 2164752 ***