Bug 910093

Summary: yum: dependency bloat when installing Java packages
Product: [Fedora] Fedora Reporter: Mikolaj Izdebski <mizdebsk>
Component: yumAssignee: Packaging Maintenance Team <packaging-team-maint>
Status: CLOSED NOTABUG QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: 19CC: admiller, dbhole, ffesti, james.antill, jzeleny, packaging-team-maint, sochotni, tim.lauridsen
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-04-01 15:18:44 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
log from installing maven
none
log from erasing libgcj
none
log autoremoving unneeded packages
none
log from verifying package dependencies none

Description Mikolaj Izdebski 2013-02-11 19:15:25 UTC 
Description of problem:
YUM installs many more packages then necessary to satisfy dependencies of requested package. Total installed size of unnecessary dependencies is 159 MB. That's a lot given that size of upstream Macen after unpacking is less than 6 MB (including all of its depencencies except Java Virtual Machine).

This problem is not specific to maven package (or its dependencies), but generic problem with YUM. The same problem can be seen when installing different Java applications like Eclipse Platform, or JBoss Application Server.

Version-Release number of selected component (if applicable):
yum-3.4.3-49.fc18.noarch

How reproducible:
Easily. Can also be reproduced on Koji.

Steps to Reproduce:
1. Install maven package
yum -y -d 10 -e 10 --nogpgcheck --noplugins --releasever rawhide --installroot /chroot install maven

2. Erase libgcj
yum -y -d 10 -e 10 --nogpgcheck --noplugins --releasever rawhide --installroot /chroot erase libgcj
(this steps removes 3 packages, 65 MB)

3. Autoremove all unneeded packages
yum -y -d 10 -e 10 --nogpgcheck --noplugins --releasever rawhide --installroot /chroot autoremove
(this steps removes 66 packages, 94 MB)

4. Verify that maven is still installed
rpm -qa maven --root /chroot

5. Verify that all dependencies are fine
rpm -Vavv --nofiles --root /chroot

Actual results:
YUM in step 1. installs at least 69 packages more then necessary to satisfy dependencies of maven. 159 MB of unneeded dependencies.

Expected results:
Only minimal set of packages required to satisfy maven dependencies is installed.

Additional info:
Maven (or its transitive dependencies) require java, which is a virtual package provided by java-1.5.0-gcj and java-1.7.0-openjdk.

Maven requires java >= 1.5. To satisfy this dependency YUM picks java-1.5.0-gcj because according to compare_providers() java-1.5.0-gcj is default provider for java:
requires minimal: 15
 Loser(with 44): 1:java-1.7.0-openjdk-1.7.0.9-2.3.5.3.fc19.x86_64
 Winner: java-1.5.0-gcj-1.5.0.0-40.fc18.x86_64

Some dependencies of maven (for example apache-commons-io) require java >= 1:1.6, but this dependency can be satisfied only by java-1.7.0-openjdk. In turn both java-1.5.0-gcj and java-1.7.0-openjdk end up being installed, while java-1.7.0-openjdk alone would suffice to satisfy all dependencies.

Logs of above steps 1-3,5 will be attached.
Comment 1 Mikolaj Izdebski 2013-02-11 19:17:35 UTC 
Created attachment 696252 [details]
log from installing maven

Log from: yum -y -d 10 -e 10 --nogpgcheck --noplugins --releasever rawhide --installroot /chroot install maven
Comment 2 Mikolaj Izdebski 2013-02-11 19:18:44 UTC 
Created attachment 696253 [details]
log from erasing libgcj

Log from:
yum -y -d 10 -e 10 --nogpgcheck --noplugins --releasever rawhide --installroot /chroot erase libgcj
Comment 3 Mikolaj Izdebski 2013-02-11 19:19:42 UTC 
Created attachment 696254 [details]
log autoremoving unneeded packages

Log from:
yum -y -d 10 -e 10 --nogpgcheck --noplugins --releasever rawhide --installroot /chroot autoremove
Comment 4 Mikolaj Izdebski 2013-02-11 19:21:08 UTC 
Created attachment 696255 [details]
log from verifying package dependencies

Log from:
rpm -Vavv --nofiles --root /chroot
Comment 5 Fedora End Of Life 2013-04-03 16:44:13 UTC 
This bug appears to have been reported against 'rawhide' during the Fedora 19 development cycle.
Changing version to '19'.

(As we did not run this process for some time, it could affect also pre-Fedora 19 development
cycle bugs. We are very sorry. It will help us with cleanup during Fedora 19 End Of Life. Thank you.)

More information and reason for this action is here:
https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Fedora19
Comment 6 Mikolaj Izdebski 2014-03-27 13:33:12 UTC 
Ping, is there any progress on that bug?
Comment 8 James Antill 2014-04-01 15:18:44 UTC 
> Some dependencies of maven (for example apache-commons-io) require java >= 1:1.6, but this dependency can be satisfied only by java-1.7.0-openjdk. In turn both java-1.5.0-gcj and java-1.7.0-openjdk end up being installed, while java-1.7.0-openjdk alone would suffice to satisfy all dependencies.

 yum doesn't have a backtracking depsolver, the maven packaging could be tweaked so that compare providers picks the correct java ... or something else, like not depping on java at all (as we know it's deps. do) ... or directly depping on the java you want.
 Apart from that there is nothing yum can do.
Comment 9 Mikolaj Izdebski 2014-04-01 16:54:02 UTC 
Installing packages for which user didn't request and which are not required to satisfy dependencies is considered as not a bug?  Is this intended feature then?

(In reply to James Antill from comment #8)
> the maven packaging could be
> tweaked so that compare providers picks the correct java ... or something
> else, like not depping on java at all (as we know it's deps. do) ... or
> directly depping on the java you want.

That would only be a workaround for the real bug in YUM.

>  Apart from that there is nothing yum can do.

I'm sure there's a lot YUM could do, for example remove superflous packages from set of packages to install.
Comment 10 Tim Lauridsen 2014-04-01 17:10:18 UTC 
(In reply to Mikolaj Izdebski from comment #9)
> Installing packages for which user didn't request and which are not required
> to satisfy dependencies is considered as not a bug?  Is this intended
> feature then?
> 

If you want maven to pull in Java 7, then maven should require java 7.

It is not up to yum to have psychic powers to guess what the user want and dont want.

It is the packages job, to make sure that there packages pull in the right requirements.
Comment 11 Mikolaj Izdebski 2014-04-01 19:04:23 UTC 
(In reply to Tim Lauridsen from comment #10)
> (In reply to Mikolaj Izdebski from comment #9)
> > Installing packages for which user didn't request and which are not required
> > to satisfy dependencies is considered as not a bug?  Is this intended
> > feature then?
> > 
> 
> If you want maven to pull in Java 7, then maven should require java 7.

And Maven does require Java 7 -- transitively.

> It is not up to yum to have psychic powers to guess what the user want and
> dont want.

Of course.

> It is the packages job, to make sure that there packages pull in the right
> requirements.

It is packagers job to specify correct dependencies.  It is YUM job to install correct package sets.  It is YUM maintainers job to fix YUM if it does not work as intended.
Comment 12 Tim Lauridsen 2014-04-02 04:56:41 UTC 
(In reply to Mikolaj Izdebski from comment #11)

> 
> > It is the packages job, to make sure that there packages pull in the right
> > requirements.
> 
> It is packagers job to specify correct dependencies.  It is YUM job to
> install correct package sets.  It is YUM maintainers job to fix YUM if it
> does not work as intended.

Yum has to make so choices base on definded requirements and the available providers, and sometimes these choices are not the ones the uses expects.

but the problem is when a package requires ex. java and there is multiple packages there is multiple packages providing java,it have to choose one.
I does not matter that a another requiment has a requirement to java >= 7
yum dont know that a the point of time it selects java-gcj.

So if you want other choices you need to and require: jave >= 7 or remove require: java, because you know that one of your other requirement will pull java >= 7 in.
Comment 13 James Antill 2014-04-07 20:38:08 UTC 
(In reply to Mikolaj Izdebski from comment #9)
> Installing packages for which user didn't request and which are not required
> to satisfy dependencies is considered as not a bug?  Is this intended
> feature then?

If DNF didn't exist then there would be some merit to trying to solve these problems in yum. As it is you are going to have a completely new depsolver in a few releases. So your best options are to live with it until then, or work around it via. changing the packaging.