Bug 825820

Summary: Libvirt is missing important hooks
Product: Red Hat Enterprise Linux 6 Reporter: Simo Sorce <ssorce>
Component: libvirtAssignee: Laine Stump <laine>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 6.3CC: acathrow, bili, dallan, dyasny, dyuan, fschwarz, gsun, mzhan, rwu
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: libvirt-0.9.13-3.el6 Doc Type: Bug Fix
Doc Text:
Cause: libvirt had hooks for calling a user-written script when a guest was started or stopped, but had no hook to call a script for each guest when libvirtd itself was restarted. Consequence: Certain custom setups that required extra operations not directly provided by libvirt would fail when libvirtd was restarted. For example, packet forwarding rules installed to redirect incoming connections to a particular guest would be overridden by libvirt's "refresh" of its own iptables packet forwarding rules, breaking the connection forwarding that had been setup (libvirtd refreshes all iptables rules that it has installed each time it is restarted). (See http://wiki.libvirt.org/page/Networking#Forwarding_Incoming_Connections for an example of setting up iptables rules to forward incoming connections to a guest) Fix: libvirt now has a new "reconnect" hook; the qemu hook script is called with a type of "reconnect" each time libvirtd is restarted. Result: user supplied hook scripts can be written to recognize the "reconnect" event, and reload the user-supplied iptables forwarding rules (for example) when it occurs. The result is that incoming connections continue to be forwarded correctly, even when libvirtd is restarted.
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-02-21 02:15:50 EST Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Description Simo Sorce 2012-05-28 10:53:19 EDT
in oreder to be able to set custom iptables rules the only way to override the std libvirtd rules it to add a qemu hook so that these rules can be applied when the VM starts.
However the hooks are not run when libvirtd is restarted and so rules cannot be properly reset after libvirt re-adds the standard rules.

The fix seem to have hit upstream now: https://www.redhat.com/archives/libvir-list/2012-May/msg01270.html
Comment 2 Laine Stump 2012-06-05 17:02:14 EDT
I've verified that the patches are adequate to perform the task Simo is requesting, and they've been ACKed upstream.
Comment 4 Laine Stump 2012-06-14 02:32:37 EDT
Dan's patch has been pushed upstream:

commit 6510c97bf5920044355d22646a91fb0a8fb9bf68
Author: Daniel P. Berrange <berrange@redhat.com>
Date:   Mon May 28 15:04:31 2012 +0100

    Add some missing hook functions
    
    A core use case of the hook scripts is to be able to do things
    to a guest's network configuration. It is possible to hook into
    the 'start' operation for a QEMU guest which runs just before
    the guest is started. The TAP devices will exist at this point,
    but the QEMU process will not. It can be desirable to have a
    'started' hook too, which runs once QEMU has started.
    
    If libvirtd is restarted it will re-populate firewall rules,
    but there is no QEMU hook to trigger for existing domains.
    This is solved with a 'reconnect' hook.
    
    Finally, if attaching to an external QEMU process there needs
    to be an 'attach' hook script.
    
    This all also applies to the LXC driver
    
    * docs/hooks.html.in: Document new operations
    * src/util/hooks.c, src/util/hooks.c: Add 'started', 'reconnect'
      and 'attach' operations for QEMU. Add 'prepare', 'started',
      'release' and 'reconnect' operations for LXC
    * src/lxc/lxc_driver.c: Add hooks for 'prepare', 'started',
      'release' and 'reconnect' operations
    * src/qemu/qemu_process.c: Add hooks for 'started', 'reconnect'
      and 'reconnect' operations
Comment 7 Wayne Sun 2012-07-24 03:18:46 EDT
pkgs:
libvirt-0.9.13-3.el6.x86_64
qemu-kvm-rhev-0.12.1.2-2.297.el6_3.x86_64
kernel-2.6.32-280.el6.x86_64

Prepare a domain
# virsh list --all
 Id    Name                           State
----------------------------------------------------
 -     rhel6u2                        shut off

Prepare a script
# cat /etc/libvirt/hooks/qemu
#! /bin/bash
echo "$0" "$@" >> /tmp/qemu.log
exit 0

Add x mod
# chmod +x /etc/libvirt/hooks/qemu

steps:
1. start the domain
# virsh start rhel6u2
Domain rhel6u2 started

check in log:
# cat /tmp/qemu.log
/etc/libvirt/hooks/qemu rhel6u2 prepare begin -
/etc/libvirt/hooks/qemu rhel6u2 start begin -
/etc/libvirt/hooks/qemu rhel6u2 started begin -

2. restart libvirtd
# service libvirtd restart
Stopping libvirtd daemon:                                  [  OK  ]
Starting libvirtd daemon:                                  [  OK  ]

check in log:
...
/etc/libvirt/hooks/qemu rhel6u2 reconnect begin -

3. destroy domain
# virsh destroy rhel6u2
Domain rhel6u2 destroyed

check in log:
...
/etc/libvirt/hooks/qemu rhel6u2 stopped end -
/etc/libvirt/hooks/qemu rhel6u2 release end -

4. attach domain
# qemu-img create /var/lib/libvirt/images/foo.img 1G
Formatting '/var/lib/libvirt/images/foo.img', fmt=raw size=1073741824 

# /usr/libexec/qemu-kvm -cdrom /var/lib/libvirt/images/foo.img -monitor unix:/tmp/demo,server,nowait -name foo -uuid cece4f9f-dff0-575d-0e8e-01fe380f12ea  &
[1] 1688

# virsh qemu-attach 1688
Domain foo attached to pid 1688

# virsh list --all
 Id    Name                           State
----------------------------------------------------
 2     foo                            running
 -     rhel6u2                        shut off

# cat /tmp/qemu.log 
...
/etc/libvirt/hooks/qemu foo attach begin -


5. lxc hooks

Now define lxc domain get this error:
# virsh define xml1
error: Failed to define domain from xml1
error: internal error unexpected domain type lxc, expecting one of these: qemu, kqemu, kvm, xen

So this is bacially blocked.
Comment 8 Wayne Sun 2012-07-25 04:33:34 EDT
> 
> 5. lxc hooks
> 
> Now define lxc domain get this error:
> # virsh define xml1
> error: Failed to define domain from xml1
> error: internal error unexpected domain type lxc, expecting one of these:
> qemu, kqemu, kvm, xen
> 
> So this is bacially blocked.

Make a mistake here, define should be:
# virsh -c lxc:/// define xml1
Domain toy defined from xml1

# cat /etc/libvirt/hooks/lxc
#! /bin/bash
echo "$0" "$@" >> /tmp/lxc.log
exit 0


# virsh -c lxc:/// start toy
error: Failed to start domain toy
error: internal error guest failed to start: PATH=/bin:/sbin TERM=linux container=lxc-libvirt container_uuid=d1f4798b-bebf-d93c-1d97-fe1c1cb7c780 LIBVIRT_LXC_UUID=d1f4798b-bebf-d93c-1d97-fe1c1cb7c780 LIBVIRT_LXC_NAME=toy /bin/sh
2012-07-25 07:38:51.688+0000: 1: info : libvirt version: 0.9.13, package: 3.el6 (Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>, 2012-07-20-03:24:32, x86-010.build.bos.redhat.com)
2012-07-25 07:38:51.688+0000: 1: error : lxcContainerIdentifyCGroups:1282 : Unable to read directory /sys/fs/cgroup: No such file or directory
2012-07-25 07:38:51.691+0000: 6548: info : libvirt version: 0.9.13, package: 3.el6 (Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>, 2012-07-20-03:24:32, x86-010.build.bos.redhat.com)
2012-07-25 07:38:51.691+0000: 6548: error : virCommandWait:2314 : internal error Child process (ip link set veth1 netns 6549) status unexpected: exit status 2

^^Another bug is filed to track this:
https://bugzilla.redhat.com/show_bug.cgi?id=842979

check the log:
# cat /tmp/lxc.log 
/etc/libvirt/hooks/lxc toy prepare begin -
/etc/libvirt/hooks/lxc toy start begin -
/etc/libvirt/hooks/lxc toy stopped end -
/etc/libvirt/hooks/lxc toy release end -

Eventhough domain fail to start, but the hook is working as expected.
So, this is fixed.
Comment 9 errata-xmlrpc 2013-02-21 02:15:50 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2013-0276.html