Bug 1012198

Summary: libvirt-daemon-lxc kind of heavy on the dependencies.
Product: [Fedora] Fedora Reporter: Matthew Miller <mattdm>
Component: libvirtAssignee: Libvirt Maintainers <libvirt-maint>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 20CC: alexl, berrange, clalancette, crobinso, itamar, jforbes, jpoimboe, jyang, laine, libvirt-maint, veillard, virt-maint
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: libvirt-1.1.3.1-1.fc20 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1024393 (view as bug list) Environment:
Last Closed: 2013-11-12 00:35:38 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Remove bogus deps from libvirt-daemon RPM none

Description Matthew Miller 2013-09-26 02:04:16 UTC
From the basically-minimal Fedora 20 Cloud image, installing libvirt-daemon-lxc pulls in.... kind of a lot of stuff. Could we reduce this? It would be nice for the purpose of having a cloud image which runs containers. The more lightweight the "outer shell" can be, the more room there is for the actual contents.


I'd prefer the advantages of libvirt lxc over the lxc tools, but, this is literally 150x bigger.

List of deps follows.


================================================================================
 Package                         Arch   Version           Repository       Size
================================================================================
Installing:
 libvirt-daemon-lxc              x86_64 1.1.2-3.fc20      updates-testing  34 k
Installing for dependencies:
 augeas-libs                     x86_64 1.1.0-2.fc20      fedora          323 k
 boost-system                    x86_64 1.54.0-5.fc20     fedora           38 k
 boost-thread                    x86_64 1.54.0-5.fc20     fedora           57 k
 bzip2                           x86_64 1.0.6-9.fc20      fedora           52 k
 ceph-libs                       x86_64 0.67.3-1.fc20     updates-testing 1.7 M
 corosync                        x86_64 2.3.2-1.fc20      fedora          185 k
 corosynclib                     x86_64 2.3.2-1.fc20      fedora          110 k
 cryptopp                        x86_64 5.6.2-3.fc20      fedora          1.0 M
 cyrus-sasl                      x86_64 2.1.26-12.fc20    updates-testing  87 k
 cyrus-sasl-md5                  x86_64 2.1.26-12.fc20    updates-testing  55 k
 device-mapper-event             x86_64 1.02.81-1.fc20    updates-testing 140 k
 device-mapper-event-libs        x86_64 1.02.81-1.fc20    updates-testing 134 k
 device-mapper-persistent-data   x86_64 0.2.7-1.fc20      updates-testing 362 k
 ebtables                        x86_64 2.0.10-11.fc20    fedora          120 k
 fuse-libs                       x86_64 2.9.3-2.fc20      fedora           92 k
 gettext                         x86_64 0.18.3.1-1.fc20   fedora          969 k
 gettext-libs                    x86_64 0.18.3.1-1.fc20   fedora          258 k
 glusterfs                       x86_64 3.4.0-8.fc20      fedora          850 k
 glusterfs-api                   x86_64 3.4.0-8.fc20      fedora           49 k
 glusterfs-fuse                  x86_64 3.4.0-8.fc20      fedora           91 k
 glusterfs-libs                  x86_64 3.4.0-8.fc20      fedora          225 k
 gnutls-dane                     x86_64 3.1.13-2.fc20     fedora           49 k
 gnutls-utils                    x86_64 3.1.13-2.fc20     fedora          267 k
 iscsi-initiator-utils           x86_64 6.2.0.873-12.fc20 fedora          459 k
 keyutils                        x86_64 1.5.6-2.fc20      updates-testing  53 k
 ldns                            x86_64 1.6.16-6.fc20     fedora          469 k
 leveldb                         x86_64 1.12.0-5.fc20     fedora          160 k
 libaio                          x86_64 0.3.109-8.fc20    fedora           23 k
 libcroco                        x86_64 0.6.8-3.fc20      fedora          103 k
 libevent                        x86_64 2.0.21-3.fc20     fedora          210 k
 libibverbs                      x86_64 1.1.7-3.fc20      fedora           48 k
 libiscsi                        x86_64 1.9.0-4.fc20      fedora           59 k
 libnfsidmap                     x86_64 0.25-7.fc20       fedora           45 k
 libqb                           x86_64 0.16.0-1.fc20     fedora           92 k
 librdmacm                       x86_64 1.0.17-2.fc20     fedora           58 k
 libtirpc                        x86_64 0.2.3-4.fc20      fedora           82 k
 libunistring                    x86_64 0.9.3-9.fc20      fedora          291 k
 libvirt-client                  x86_64 1.1.2-3.fc20      updates-testing 5.4 M
 libvirt-daemon                  x86_64 1.1.2-3.fc20      updates-testing 2.2 M
 libvirt-daemon-driver-interface x86_64 1.1.2-3.fc20      updates-testing  75 k
 libvirt-daemon-driver-lxc       x86_64 1.1.2-3.fc20      updates-testing 117 k
 libvirt-daemon-driver-network   x86_64 1.1.2-3.fc20      updates-testing  87 k
 libvirt-daemon-driver-nodedev   x86_64 1.1.2-3.fc20      updates-testing  75 k
 libvirt-daemon-driver-nwfilter  x86_64 1.1.2-3.fc20      updates-testing 100 k
 libvirt-daemon-driver-secret    x86_64 1.1.2-3.fc20      updates-testing  70 k
 libvirt-daemon-driver-storage   x86_64 1.1.2-3.fc20      updates-testing 115 k
 libwsman1                       x86_64 2.3.6-8.fc20      fedora          121 k
 libxslt                         x86_64 1.1.28-5.fc20     fedora          241 k
 lvm2                            x86_64 2.02.102-1.fc20   updates-testing 755 k
 lvm2-libs                       x86_64 2.02.102-1.fc20   updates-testing 625 k
 lzo                             x86_64 2.06-5.fc20       fedora           58 k
 lzop                            x86_64 1.03-9.fc20       fedora           54 k
 net-snmp-libs                   x86_64 1:5.7.2-15.fc20   fedora          745 k
 netcf-libs                      x86_64 0.2.3-5.fc20      fedora           65 k
 nfs-utils                       x86_64 1:1.2.8-4.1.fc20  fedora          356 k
 nmap-ncat                       x86_64 2:6.40-2.fc20     fedora          123 k
 numactl-libs                    x86_64 2.0.8-4.fc20      fedora           28 k
 numad                           x86_64 0.5-12.20130814git.fc20
                                                          fedora           30 k
 pm-utils                        x86_64 1.4.1-26.fc20     fedora          139 k
 qemu-img                        x86_64 2:1.6.0-7.fc20    fedora          479 k
 quota                           x86_64 1:4.01-10.fc20    fedora          177 k
 quota-nls                       noarch 1:4.01-10.fc20    fedora           89 k
 radvd                           x86_64 1.9.2-4.fc20      fedora           85 k
 rpcbind                         x86_64 0.2.1-0.1.fc20    fedora           56 k
 sheepdog                        x86_64 0.3.0-5.fc20      fedora           95 k
 snappy                          x86_64 1.1.0-2.fc20      fedora           40 k
 socat                           x86_64 1.7.2.2-3.fc20    fedora          264 k
 tcp_wrappers                    x86_64 7.6-76.fc20       fedora           80 k
 unbound-libs                    x86_64 1.4.20-19.fc20    fedora          293 k
 yajl                            x86_64 2.0.4-3.fc20      fedora           38 k

Transaction Summary
================================================================================
Install  1 Package (+70 Dependent packages)

Total download size: 22 M
Installed size: 79 M

Comment 1 Alexander Larsson 2013-09-26 08:44:08 UTC
 libvirt-client                  x86_64 1.1.2-3.fc20      updates-testing 5.4 M

This has a 1.5 meg ChangeLog.gz, and 15 megs of translations. That seems a bit large...

 ceph-libs                       x86_64 0.67.3-1.fc20     updates-testing 1.7 M
Also kind of large, does libvirt really depend on Ceph?

Comment 2 Daniel Berrangé 2013-09-26 08:53:37 UTC
(In reply to Alexander Larsson from comment #1)
>  libvirt-client                  x86_64 1.1.2-3.fc20      updates-testing
> 5.4 M
> 
> This has a 1.5 meg ChangeLog.gz, and 15 megs of translations. That seems a
> bit large...

The changelog is moved into -devel in current builds so won't impact. There's not really anything we can do about translations unless we want to screw over non-english speakers.

> 
>  ceph-libs                       x86_64 0.67.3-1.fc20     updates-testing
> 1.7 M
> Also kind of large, does libvirt really depend on Ceph?

Yes, the storage management features use ceph.

Comment 3 Matthew Miller 2013-10-04 19:31:29 UTC
Update: something is now pulling in Avahi. I *really* don't want avahi on the cloud image.

Comment 4 Alexander Larsson 2013-10-08 08:31:52 UTC
avahi daemon, or avahi libs?

Comment 5 Daniel Berrangé 2013-10-08 09:01:08 UTC
libvirt only depends on avahi-libs, so if something is pulling in the daemon, it isn't libvirt directly.

Comment 6 Matthew Miller 2013-10-08 11:56:57 UTC
avahi-libs now pulls in avahi daemon -- that's bug #913168, apparently.

Comment 7 Daniel Berrangé 2013-10-08 14:02:40 UTC
Oh actually, I just noticed you are installing the high level libvirt lxc package:

> Installing:
>   libvirt-daemon-lxc              x86_64 1.1.2-3.fc20      updates-testing  34 

At the cost of breaking many libvirt APIs you could switch to installing the finer grained package  'libvirt-daemon-driver-lxc'. That would pull in just the lxc hypervisor APIs, which is closer in terms of functionality to the lxc tools. 

Of course if the app using libvirt needs to use non-hypervisor APIs, you're still doomed because it'll need to pull in the other libvirt-daemon-driver-* packages anyway.  It also might lead to bug reports from users who find that various parts of libvirt are non-functional, so not sure whether its a good idea for the cloud image or not really.

Comment 8 Matthew Miller 2013-10-08 19:41:03 UTC
I was excited about that suggestion for a minute, but it turns out that if I just do `yum install libvirt-daemon-driver-lxc`, everything is pulled in except

  hwdata
  libpciaccess
  libvirt-daemon-driver-interface
  libvirt-daemon-driver-nodedev
  libvirt-daemon-driver-nwfilter
  libvirt-daemon-driver-secret
  libvirt-daemon-driver-storage
  libvirt-daemon-lxc

So basically it looks like everything is pulled in _except_ the APIs. This saves about 5MB -- two megabytes compressed -- and doesn't really address the security-respin concern. So I think we're better off with the higher-level unless we can somehow make that have a significant impact.

Thanks for the idea, though!

Comment 9 Daniel Berrangé 2013-10-09 09:11:12 UTC
(In reply to Matthew Miller from comment #8)
> I was excited about that suggestion for a minute, but it turns out that if I
> just do `yum install libvirt-daemon-driver-lxc`, everything is pulled in
> except
> 
>   hwdata
>   libpciaccess
>   libvirt-daemon-driver-interface
>   libvirt-daemon-driver-nodedev
>   libvirt-daemon-driver-nwfilter
>   libvirt-daemon-driver-secret
>   libvirt-daemon-driver-storage
>   libvirt-daemon-lxc
> 
> So basically it looks like everything is pulled in _except_ the APIs. This
> saves about 5MB -- two megabytes compressed -- and doesn't really address
> the security-respin concern. So I think we're better off with the
> higher-level unless we can somehow make that have a significant impact.

Smells like something is wrong there then. All those storage related deps like ceph/gluster/etc should only be required by the libvirt-daemon-driver-storage sub-RPM.

Comment 10 Matthew Miller 2013-10-09 12:02:46 UTC
(In reply to Daniel Berrange from comment #9)
> Smells like something is wrong there then. All those storage related deps
> like ceph/gluster/etc should only be required by the
> libvirt-daemon-driver-storage sub-RPM.

libvirt-daemon-driver-lxc requires libvirt-daemon, and then that pulls in all the other stuff.

Comment 11 Daniel Berrangé 2013-10-09 12:30:16 UTC
Ok, that definitely sounds broken then. The libvirt-daemon package itself should be pretty minimal on deps since it only really contains RPC code.

Comment 12 Daniel Berrangé 2013-10-29 11:56:02 UTC
Created attachment 817058 [details]
Remove bogus deps from libvirt-daemon RPM

This patch removes the many bogus deps from the libvirt-daemon RPM, pushing them down to the libvirt-daemon-driver-XXXX RPMs which actually require them.

On a Fedora 19 host, I installed a minimal container using

 # yum -y --releasever=19 --nogpg --installroot=/var/lib/libvirt/filesystems/libvirtsize1 \
          --disablerepo='*' --enablerepo=fedora install \
          systemd passwd yum fedora-release vim-minimal openssh-server procps-ng

And then installed just 'libvirt-daemon-driver-lxc' - this pulls in libvirt-daemon-driver-network, libvirt-daemon and libvirt-client.

With the existing packaging I see this size usage

# du -h -c -s /var/lib/libvirt/filesystems/libvirtsize1
453M	/var/lib/libvirt/filesystems/libvirtsize1
453M	total

With the patch applied, we save 41 MB:

# du -h -c -s /var/lib/libvirt/filesystems/libvirtsize2
412M	/var/lib/libvirt/filesystems/libvirtsize2
412M	total

The following packages are now, no longer required in the minimal install

augeas-libs-1.0.0-2.fc19.x86_64
boost-system-1.53.0-6.fc19.x86_64
boost-thread-1.53.0-6.fc19.x86_64
bzip2-1.0.6-8.fc19.x86_64
ceph-libs-0.56.4-1.fc19.x86_64
corosync-2.3.0-3.fc19.x86_64
corosynclib-2.3.0-3.fc19.x86_64
cryptopp-5.6.2-2.fc19.x86_64
device-mapper-event-1.02.77-9.fc19.x86_64
device-mapper-event-libs-1.02.77-9.fc19.x86_64
device-mapper-persistent-data-0.1.4-3.fc19.x86_64
e2fsprogs-libs-1.42.7-2.fc19.x86_64
ebtables-2.0.10-8.fc19.x86_64
fuse-2.9.2-3.fc19.x86_64
glusterfs-3.4.0-0.5.beta2.fc19.x86_64
glusterfs-api-3.4.0-0.5.beta2.fc19.x86_64
glusterfs-fuse-3.4.0-0.5.beta2.fc19.x86_64
iscsi-initiator-utils-6.2.0.873-6.fc19.x86_64
keyutils-1.5.5-4.fc19.x86_64
leveldb-1.9.0-1.fc19.x86_64
libaio-0.3.109-7.fc19.x86_64
libibverbs-1.1.6-6.fc19.x86_64
libiscsi-1.7.0-4.fc19.x86_64
libnfsidmap-0.25-5.fc19.x86_64
libqb-0.14.4-2.fc19.x86_64
librdmacm-1.0.17-1.fc19.x86_64
libtirpc-0.2.3-2.fc19.x86_64
libxslt-1.1.28-3.fc19.x86_64
lvm2-2.02.98-9.fc19.x86_64
lvm2-libs-2.02.98-9.fc19.x86_64
lzo-2.06-4.fc19.x86_64
lzop-1.03-8.fc19.x86_64
netcf-libs-0.2.3-4.fc19.x86_64
net-snmp-libs-5.7.2-11.fc19.x86_64
nfs-utils-1.2.8-2.0.fc19.x86_64
qemu-img-1.4.2-3.fc19.x86_64
quota-4.01-6.fc19.x86_64
quota-nls-4.01-6.fc19.noarch
rpcbind-0.2.0-21.fc19.x86_64
sheepdog-0.3.0-4.fc19.x86_64
snappy-1.1.0-1.fc19.x86_64
tcp_wrappers-7.6-73.fc19.x86_64
which-2.20-5.fc19.x86_64
xz-5.1.2-4alpha.fc19.x86_64


I don't think we can get much smaller than this without either major code changes to libvirt, or improvements to RPM to allow exclusion of things like translations without breaking upgrades

Comment 13 Daniel Berrangé 2013-10-29 11:57:41 UTC
Oh, note that since I used F19 for this test, it still has the 1.5 MB changelog.gz file present, that is gone in rawhide/f20.

Comment 14 Fedora Update System 2013-11-06 18:38:25 UTC
libvirt-1.1.3.1-1.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/libvirt-1.1.3.1-1.fc20

Comment 15 Fedora Update System 2013-11-07 19:08:51 UTC
Package libvirt-1.1.3.1-1.fc20:
* should fix your issue,
* was pushed to the Fedora 20 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing libvirt-1.1.3.1-1.fc20'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-20869/libvirt-1.1.3.1-1.fc20
then log in and leave karma (feedback).

Comment 16 Fedora Update System 2013-11-12 00:35:38 UTC
libvirt-1.1.3.1-1.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.