Bug 1338921

Summary: dnf removes sqlite-libs because plexmediaserver pretends it provides them
Product: [Fedora] Fedora Reporter: Stephen Herr <sherr>
Component: dnfAssignee: Packaging Maintenance Team <packaging-team-maint>
Status: CLOSED DUPLICATE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 23CC: jsilhan, mluscon, mmraka, orion, packaging-team-maint, pnemade, sherr, vmukhame, wolfram
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-05-24 18:34:36 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1516045    

Description Stephen Herr 2016-05-23 16:16:47 UTC
Description of problem:
dnf is not doing proper dependency resolution on remove, causing it to leave behind broken dependencies.

Version-Release number of selected component (if applicable):
dnf-1.1.9-1.fc23.noarch

How reproducible:
Always

Steps to Reproduce:
1. dnf remove gstreamer1-libav # for example, from rpmfusion

Actual results:
May 23 11:02:05 DEBUG --> Starting dependency resolution
May 23 11:02:05 DEBUG --> Finding unneeded leftover dependencies
May 23 11:02:05 DEBUG ---> Package gstreamer1-libav.x86_64 1.6.2-1.fc23 will be erased
May 23 11:02:05 DEBUG ---> Package sqlite-libs.x86_64 3.11.0-3.fc23 will be erased
May 23 11:02:05 DEBUG --> Finished dependency resolution
May 23 11:02:05 INFO Dependencies resolved.
May 23 11:02:05 INFO ================================================================================
 Package               Arch        Version             Repository          Size
================================================================================
Removing:
 gstreamer1-libav      x86_64      1.6.2-1.fc23        @@commandline      649 k
 sqlite-libs           x86_64      3.11.0-3.fc23       @updates           879 k

Transaction Summary
================================================================================
Remove  2 Packages


Expected results:
If sqlite-libs is being removed then everything that depends on it should be removed too. This is obviously not happening as multiple things still installed depended on sqlite-libs:

$ rpm -q --whatrequires sqlite-libs
sqlite-3.11.0-3.fc23.x86_64
$ rpm -q --whatrequires libsqlite3.so.0
nss-softokn-3.23.0-1.0.fc23.i686
libsoup-2.52.2-2.fc23.i686


Additional info:
sqlite-libs is being uninstalled in this case because of bug 1227066. This is really bad behavior, you absolutely should not default that option to True.

These two bugs in concert are extremely dangerous, and mean that removing anything that depends on a system-ish thing can absolutely wreck your installation. Removing sqlite-libs for example breaks both gnome-shell and dnf, leaving the user in a world of hurt. This has happened to other people too, not just me: https://ask.fedoraproject.org/en/question/75684/how-to-recover-yumdnfrpm-after-uninstall-sqlite/

An expert user that has another working computer can figure out how to recover from this, but an average user probably not. This bug in concert with bug 1227066 can easily wreck a user's Fedora install depending on what they happen to be removing and what it happens to depend on. As such I'm setting severity to urgent.

Comment 1 Parag Nemade 2016-05-23 16:53:33 UTC
When I tried to reproduce this on Fedora 24, I see 2 things
1)  $ sudo  dnf remove gstreamer1-libav
Dependencies resolved.
================================================================================
 Package           Arch    Version       Repository                        Size
================================================================================
Removing:
 gstreamer1-libav  x86_64  1.6.3-1.fc23  @rpmfusion-free-updates-testing  650 k

Transaction Summary
================================================================================
Remove  1 Package

Installed size: 650 k
Is this ok [y/N]: n
Operation aborted.

2) Removal of sqlite-libs is not allowed. See
$ sudo  dnf remove sqlite-libs
Dependencies resolved.
Error: The operation would result in removing the following protected packages: dnf.

Do you have python3-dnf-plugins-core package installed that have protected plugin that prevents removal of sqlite-libs package?

Comment 2 Orion Poplawski 2016-05-23 17:07:52 UTC
Possibly related: bug #1338946

Comment 3 Michael Mráka 2016-05-24 06:18:27 UTC
Most likely there another package on your system providing libsqlite3.so.
What's the output of
rpm -q --whatprovides 'libsqlite3.so.0()(64bit)' ?

Comment 4 Stephen Herr 2016-05-24 13:54:45 UTC
(In reply to Michael Mráka from comment #3)
> Most likely there another package on your system providing libsqlite3.so.
> What's the output of
> rpm -q --whatprovides 'libsqlite3.so.0()(64bit)' ?

I will get the exact command output for you later, but I can tell you already that's wrong. 1) because 'sqlite' depended on 'sqlite-libs', and was not removed, so we already know there's a problem with dependency resolution. 2) because nss was definitely broken, which was causing a bunch of other things to fail.

Comment 5 Stephen Herr 2016-05-24 14:11:03 UTC
(In reply to Parag Nemade from comment #1)
> When I tried to reproduce this on Fedora 24, I see 2 things
> 1)  $ sudo  dnf remove gstreamer1-libav
> Dependencies resolved.
> =============================================================================
> ===
>  Package           Arch    Version       Repository                       
> Size
> =============================================================================
> ===
> Removing:
>  gstreamer1-libav  x86_64  1.6.3-1.fc23  @rpmfusion-free-updates-testing 
> 650 k
> 
> Transaction Summary
> =============================================================================
> ===
> Remove  1 Package
> 
> Installed size: 650 k
> Is this ok [y/N]: n
> Operation aborted.
> 
> 2) Removal of sqlite-libs is not allowed. See
> $ sudo  dnf remove sqlite-libs
> Dependencies resolved.
> Error: The operation would result in removing the following protected
> packages: dnf.
> 
> Do you have python3-dnf-plugins-core package installed that have protected
> plugin that prevents removal of sqlite-libs package?

I do:
$ rpm -q python3-dnf-plugins-core
python3-dnf-plugins-core-0.1.21-1.fc23.noarch

It sounds to me like the problem is fixed in F24 then, dependency resolution is happening properly so dnf does not try to remove sqlite-libs because it's not "unneeded".

I get the same result when I try to remove sqlite-libs directly, so in that case it is correctly detecting the dnf dependency and preventing the removal.

Comment 6 Stephen Herr 2016-05-24 14:14:11 UTC
(In reply to Stephen Herr from comment #4)
> (In reply to Michael Mráka from comment #3)
> > Most likely there another package on your system providing libsqlite3.so.
> > What's the output of
> > rpm -q --whatprovides 'libsqlite3.so.0()(64bit)' ?
> 
> I will get the exact command output for you later, but I can tell you
> already that's wrong. 1) because 'sqlite' depended on 'sqlite-libs', and was
> not removed, so we already know there's a problem with dependency
> resolution. 2) because nss was definitely broken, which was causing a bunch
> of other things to fail.

Hmm, so it looks like there is something else that provides that library:

$ rpm -q --whatprovides 'libsqlite3.so.0()(64bit)'
plexmediaserver-0.9.15.2.1663-7efd046.x86_64
sqlite-libs-3.11.0-3.fc23.x86_64

That doesn't answer the question about why sqlite was not removed however. And nss was definitely broken, but maybe plexmediaserver's sqlite library is just broken? I would hope not... but maybe...

Comment 7 Michael Mráka 2016-05-24 18:34:36 UTC
$ rpm -ql plexmediaserver-0.9.15.2.1663-7efd046.x86_64 |grep sqlite
/usr/lib/plexmediaserver/Resources/Python/lib/python2.7/sqlite3
/usr/lib/plexmediaserver/Resources/Python/lib/python2.7/sqlite3/__init__.py
/usr/lib/plexmediaserver/Resources/Python/lib/python2.7/sqlite3/dbapi2.py
/usr/lib/plexmediaserver/Resources/Python/lib/python2.7/sqlite3/dump.py
/usr/lib/plexmediaserver/Resources/Python/lib/python2.7/sqlite3/test
/usr/lib/plexmediaserver/Resources/Python/lib/python2.7/sqlite3/test/__init__.py
/usr/lib/plexmediaserver/Resources/Python/lib/python2.7/sqlite3/test/dbapi.py
/usr/lib/plexmediaserver/Resources/Python/lib/python2.7/sqlite3/test/dump.py
/usr/lib/plexmediaserver/Resources/Python/lib/python2.7/sqlite3/test/factory.py
/usr/lib/plexmediaserver/Resources/Python/lib/python2.7/sqlite3/test/hooks.py
/usr/lib/plexmediaserver/Resources/Python/lib/python2.7/sqlite3/test/py25tests.py
/usr/lib/plexmediaserver/Resources/Python/lib/python2.7/sqlite3/test/regression.py
/usr/lib/plexmediaserver/Resources/Python/lib/python2.7/sqlite3/test/transactions.py
/usr/lib/plexmediaserver/Resources/Python/lib/python2.7/sqlite3/test/types.py
/usr/lib/plexmediaserver/Resources/Python/lib/python2.7/sqlite3/test/userfunctions.py
/usr/lib/plexmediaserver/libgnsdk_storage_sqlite.so.3.07.7
/usr/lib/plexmediaserver/libsoci_sqlite3.so.3.0.0
/usr/lib/plexmediaserver/libsqlite3.so.0

So it tells it provides 'libsqlite3.so.0()(64bit)' but the library is not in standard library path where linker looks for it.

Dnf have really no way to prevent removing important stuff when broken third party packages pretend they provide (system) libraries when they don't.

*** This bug has been marked as a duplicate of bug 1284349 ***

Comment 8 Stephen Herr 2016-05-24 18:59:37 UTC
And for the record I've answered my own question about the 'sqlite' package:

$ rpm -q --whatprovides 'sqlite-libs'
sqlite-libs-3.11.0-3.fc23.i686
sqlite-libs-3.11.0-3.fc23.x86_64

I have two sqlite-libs packages installed, so there was something else that provided sqlite-libs when the x86_64 version was removed and 'sqlite' was perfectly happy remaining installed. I don't know if sqlite actually still worked or not, but if not that would be a problem with sqlite's dependency declaration, not dnf.

So I agree that this is not a dnf bug, with one giant exception: bug 1227066 really really needs to be fixed. A nice-to-have feature like automatic leaf dependency cleanup is much less important than not blowing up people's systems in the presence of sub-optimal packaging.

Furthermore I strongly suspect that plex's 'Provides' entry for 'libsqlite3.so.0()(64bit)' is auto-generated by the script that detects that kind of thing. If so, it *is* a bug that that script detects libraries that the system linker cannot find. But I don't know if that would be a bug on dnf or rpm or what.

Can you please look into that possibility Michael?

Comment 9 Michael Mráka 2016-05-26 10:25:35 UTC
Yes, it's an autogenerated provide.

And as for the libraries that the system linker cannot find - sometimes it's correct e.g. in scl packages becuase they modify library path, sometimes it isn't. And it's hard to find a 100% correct common pattern among them. So unless packages bundle own copies of system libraries (which is forbidden in Fedora guidlines) current implementation works good.

Comment 10 Michael Mráka 2016-05-26 10:45:56 UTC
It's rather workaround then a proper fix - make dnf depend on system libraries
https://github.com/rpm-software-management/dnf/pull/502

Comment 11 Michael Mráka 2016-06-23 14:31:11 UTC
*** Bug 1348653 has been marked as a duplicate of this bug. ***