Bug 1875000

Summary:	dnf upgrade left system with important libs symlinked to 0 length new libraries
Product:	[Fedora] Fedora	Reporter:	Zdenek Kabelac <zkabelac>
Component:	rpm	Assignee:	Packaging Maintenance Team <packaging-team-maint>
Status:	CLOSED INSUFFICIENT_DATA	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	rawhide	CC:	dmach, igor.raits, jmracek, jrohel, mblaha, mhatina, mjw, packaging-team-maint, pkratoch, pmatilai, pmoravco, rpm-software-management, vmukhame
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2021-02-05 14:10:53 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Zdenek Kabelac 2020-09-02 17:42:03 UTC

I've been doing upgrade from rawhide around fc32 to rawhide (as of 2020/08/29)

The upgrade process end with disaster which is likely worth of creation of this BZ.

Basically - the 'upgrade'  stopped at some point - and after 'naive' reboot I've been left with unbootable system ending in kernel panic due to unmountable rootfs.

I'd been able to recover system with init=/bin/sh and lot of other tricks.

Exploring the system shown some important libraries (i.e. openssl)  were symlinked to 0 length new libraries  (with suffix 'g', while existing old libs has suffix 'c')

After fixing these symlinks (about 8 libraries I believe) - I had to also manage to start networking and download 'cryptsetup' package which was IMHO mandatory for the whole upgrade as lots of systemd referenced new symbol available in the latest version of this package.

So this gives me the thinking the starting trouble could have been upgrade of some systemd packages which stopped working and broke whole rest of transaction.

But also presents a question how is it possible to 'switch' to library with 0 length new versions - is there any check new files are correct from upgrade rpm unpackaging  - assuming if there is some out-of-disk space condition in the middle of transaction - the process correctly stops ??

Comment 1 Panu Matilainen 2020-09-07 11:35:02 UTC

What happened there is anybody's guess.

If it's just library symlinks that are wrecked then I'd maybe eyeball ldconfig first. 

Rpm verifies file contents on unpacking, always has, but errors do not abort the entire transaction at once because there's no means to go back either.

Comment 2 Panu Matilainen 2021-02-05 14:10:53 UTC

As noted above, insufficient data to even begin to guess. Closing.

Comment 3 Zdenek Kabelac 2021-02-05 15:24:07 UTC

This should have been quite easy to reproduce in your testing setup.

I'm not going to demolish my working machine - as it's been highly problematic to restore it back.

So this is IMHO foolish way to paper over  real major bug in the upgrading system.

(Mainly the clear fact that rpm is not verifying that uncompression was all without errors)

Comment 4 Panu Matilainen 2021-02-08 07:57:41 UTC

There's nothing resembling a reproducer here. Upgrade to rawhide at some random day going belly up, once, for a single person, could be caused by any number of things. 

As I already said but you seem to have ignored: rpm verifies the payload as it goes always, unless you explicitly tell it not to. If something later overwrites those files, there's nothing rpm can do about it.