Bug 1393688 - [RFE] Ship extra block drivers in QEMU as separate packages
Summary: [RFE] Ship extra block drivers in QEMU as separate packages
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: qemu
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Fedora Virtualization Maintainers
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-11-10 07:30 UTC by Fam Zheng
Modified: 2016-11-14 14:39 UTC (History)
11 users (show)

Fixed In Version: qemu-2.7.0-8.fc26
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-11-14 14:39:33 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
qemu.spec patch (3.88 KB, patch)
2016-11-10 16:34 UTC, Richard W.M. Jones
no flags Details | Diff
qemu.spec patch (4.21 KB, patch)
2016-11-10 17:56 UTC, Richard W.M. Jones
no flags Details | Diff
0001-Create-subpackages-for-modularized-qemu-block-driver.patch (23.93 KB, patch)
2016-11-11 10:34 UTC, Richard W.M. Jones
no flags Details | Diff
0001-Create-subpackages-for-modularized-qemu-block-driver.patch (24.73 KB, patch)
2016-11-11 16:24 UTC, Richard W.M. Jones
no flags Details | Diff

Description Fam Zheng 2016-11-10 07:30:41 UTC
The "$QEMU_SRC/configure --enable-modules" option of the build system allows building a few block drivers as DSO objects (e.g. block-iscsi.so, block-glusterfs.so). It can be used to trim dependent packages of the main qemu packages, and make installing of ceph, gluster and iscsi client libraries optional. We should achieve this in Fedora too. See also:

https://www.archlinux.org/packages/extra/x86_64/qemu-arch-extra/
https://packages.debian.org/sid/qemu-block-extra
http://packages.ubuntu.com/xenial/qemu-block-extra

Commands:

$ cd $QEMU_SRC
$ ./configure --enable-modules --target-list=x86_64-softmmu && make -j8
<snip>
$ ls *.so
block-curl.so  block-dmg-bz2.so  block-iscsi.so  block-ssh.so

Comment 1 Richard W.M. Jones 2016-11-10 16:34:49 UTC
Created attachment 1219459 [details]
qemu.spec patch

The basic plan could be as simple as the attached specfile patch.

Unfortunately the dependencies aren't quite right in this version.
RPM doesn't consider the plugins when calculating dependencies so
although the dependencies are dropped from the qemu-system-* packages,
they are NOT added to the qemu-block-* packages.  Still examining
this.

There is a problem on the upgrade path.  If the user has the 'qemu'
metapackage installed, then they will automatically get all the block
drivers on upgrade.  If the user only has (eg) 'qemu-system-x86' then
on upgrade they will lose all the block drivers in modules.

Comment 2 Daniel Berrangé 2016-11-10 16:39:31 UTC
(In reply to Richard W.M. Jones from comment #1)
>  If the user only has (eg) 'qemu-system-x86' then
> on upgrade they will lose all the block drivers in modules.

That's not only a problem on upgrade - we've encouraged applications to have "Requires: qemu-system-x86" or "Requires: qemu-kvm" as their primary dependancy and we don't want them to all suddenly loose the block drivers on their installs.

Comment 3 Richard W.M. Jones 2016-11-10 17:56:37 UTC
Created attachment 1219469 [details]
qemu.spec patch

Improved patch which fixes the problem with dependencies.

Comment 4 Richard W.M. Jones 2016-11-10 17:57:00 UTC
(In reply to Daniel Berrange from comment #2)
> (In reply to Richard W.M. Jones from comment #1)
> >  If the user only has (eg) 'qemu-system-x86' then
> > on upgrade they will lose all the block drivers in modules.
> 
> That's not only a problem on upgrade - we've encouraged applications to have
> "Requires: qemu-system-x86" or "Requires: qemu-kvm" as their primary
> dependancy and we don't want them to all suddenly loose the block drivers on
> their installs.

Agreed, but I don't have any better ideas at the moment.

Comment 5 Daniel Berrangé 2016-11-10 18:17:15 UTC
I think the key thing is that everyone should continue to get exactly the same stuff they have today. Users should have to make an explicit change to *not* get some of the modules.

To achieve this we need to use the same trick we did with libvirt when we modularized libvirtd

Change all the 'qemu-system-$ARCH' sub-RPMs into virtual packages which just have some deps

  Requires: qemu-system-$ARCH-emul
  Requires: qemu-module-block-curl
  Requires: qemu-module-block-rbd
  Requires: qemu-module-block-gluster
  Requires: qemu-module-block-dmg
  Requires: qemu-module-block-ssh


All the files that used to be in qemu-system-$ARCH now go into a new qemu-system-$ARCH-emul instead.

That way, people who want to have a super cut-down install of QEMU would install qemu-system-x86-emul only. Everyone else continues to get the same set of block modules they have today.

Comment 6 Richard W.M. Jones 2016-11-10 18:42:51 UTC
I agree that would work.

Can I propose another idea: We add the requires to qemu-common
but make them "Recommends".  This allows minimal environments
which don't want the extra dependencies to remove the functionality
while not making the spec too complicated or the list of packages too
long [in your scenario above, what should happen for a user who
just installs qemu-system-x86-emul?]

The only disadvantage would be that we couldn't implement this
in RHEL 7.

Comment 7 Daniel Berrangé 2016-11-11 09:20:10 UTC
(In reply to Richard W.M. Jones from comment #6)
> I agree that would work.
> 
> Can I propose another idea: We add the requires to qemu-common
> but make them "Recommends".  This allows minimal environments
> which don't want the extra dependencies to remove the functionality

I don't think that Recommends as a concept works very well. It means you get every module installed by default and then have to manually uninstall pieces you don't want. This means if you have an application that wants to install only the RBD module, there's no way to express this in deps from your package as AFAICT, there's no way to say "don't install this particular Recommended package" as an RPM dep. With my sub-package proposal, apps can explicitly dep on the exact subset of bits they wish to have.

> while not making the spec too complicated or the list of packages too
> long [in your scenario above, what should happen for a user who
> just installs qemu-system-x86-emul?]

If you only install qemu-system-x86-emul you get just the built-in functionality of QEMU, none of the modules, so the smallest possible install.

> The only disadvantage would be that we couldn't implement this
> in RHEL 7.

Comment 8 Richard W.M. Jones 2016-11-11 09:44:14 UTC
I didn't realize that it was intended that qemu-system-*-emul
was a valid installation choice.  In that case we should call it
something more descriptive, such as qemu-system-*-core.

I'll try to modify the patch to do that now.

Comment 9 Richard W.M. Jones 2016-11-11 10:34:58 UTC
Created attachment 1219699 [details]
0001-Create-subpackages-for-modularized-qemu-block-driver.patch

Updated patch which implements Dan's plan from comment 5
except the core subpackages are called qemu-system-*-core.

A scratch build is building:
http://koji.fedoraproject.org/koji/taskinfo?taskID=16401212

Comment 10 Daniel Berrangé 2016-11-11 10:37:30 UTC
That looks reasonable to me.

Comment 11 Richard W.M. Jones 2016-11-11 13:57:27 UTC
I had a play with the scratch build (see comment 9).

Upgrades work, automatically installing qemu-block-* and
qemu-system-*-core.

You cannot practically uninstall qemu-block-* without removing
quite a lot of other stuff.  I believe this is because of the
libvirt-daemon-qemu -> qemu dependency (and libguestfs for example
depends on libvirt-daemon-qemu).  So perhaps further subpackaging
of libvirt will be needed also.

Currently there is no "qemu-kvm-core".  Should there be?  The
idea is that qemu-kvm-core would depend on qemu-%{kvm_package}-core
(eg. qemu-system-x86-core on x86_64).

Comment 12 Richard W.M. Jones 2016-11-11 16:24:44 UTC
Created attachment 1219828 [details]
0001-Create-subpackages-for-modularized-qemu-block-driver.patch

Updated patch with qemu-kvm-core package.

Comment 13 Richard W.M. Jones 2016-11-11 21:57:26 UTC
This is a scratch build if anyone wishes to test it out:

http://koji.fedoraproject.org/koji/taskinfo?taskID=16404695

Comment 14 Daniel Berrangé 2016-11-14 13:38:24 UTC
I tested upgrade path of that scratch build and everything worked ok.

I notice that qemu-kvm does not depend on qemu-kvm-core. I understand why this is not required, but it was a little surprising. No big deal though - if we ever get a reason to have such a dep we can add it later without trouble.

So I think this change is fine to push to rawhide.

Comment 15 Richard W.M. Jones 2016-11-14 14:39:33 UTC
Final package will be available here soon:

http://koji.fedoraproject.org/koji/taskinfo?taskID=16449155


Note You need to log in before you can comment on or make changes to this bug.