Bug 1745007 - [OSP-13] Configure log filter to capture interactions between libvirt and QEMU
Summary: [OSP-13] Configure log filter to capture interactions between libvirt and QEMU
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 13.0 (Queens)
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: z9
: 13.0 (Queens)
Assignee: Martin Schuppert
QA Contact: Sasha Smolyak
URL:
Whiteboard:
Depends On: 1717359 1745003 1745005
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-08-23 13:04 UTC by Martin Schuppert
Modified: 2019-12-18 10:45 UTC (History)
10 users (show)

Fixed In Version: puppet-nova-12.4.0-24.el7ost openstack-tripleo-heat-templates-8.3.1-81.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1745005
Environment:
Last Closed: 2019-12-18 10:45:48 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1840140 0 None None None 2019-08-23 13:04:15 UTC
OpenStack gerrit 678213 0 None MERGED Adds log_filters parameter to nova::compute::libvirt 2020-10-02 13:05:11 UTC

Description Martin Schuppert 2019-08-23 13:04:15 UTC
+++ This bug was initially created as a clone of Bug #1745005 +++

+++ This bug was initially created as a clone of Bug #1745003 +++

+++ This bug was initially created as a clone of Bug #1717359 +++

What?
-----

On all compute nodes, by default, enable the libvirt log filter
("1:qemu") to capture the QMP (QEMU Machine Protocol) traffic.  This
will show the QMP request and response flow between libvirt and QEMU.


Why?
----

Often times, when debugging issues that involve interactions between
Nova, libvirt and QEMU, we need to understand the precise requests that
libvirt is sending to QEMU, and the corresponding responses from QEMU.
Since we don't capture these interactions by default, we request them[*]
_after_ a bug is reported, at which point it might not be possible to
capture them for various reasons, or worse, the machine might no longer
be available.

Having this log filter enabled can be extremely useful in debugging a
range of scenarios: live migration (with shared and non-shared storage),
live snapshots, "volume migration", and anything that involves libvirt
sending QMP commands to QEMU.

Note that we're only selectively enabling a targetted log filter
("1:qemu").  It will slightly increase the log file size, but nothing
crazy.  This slight increase in log file size is acceptable, given the
benefits.


How?
----

When deploying compute nodes, make sure that TripleO sets the below two
configuration attributes in `/etc/libvirt/libvirtd.conf`.  The "1:qemu" 
log filter will capture the QMP traffic between libvirt and QEMU in the
specified log file, `libvirtd.log`:

    ...
    log_filters="1:qemu"
    log_outputs="1:file:/var/log/libvirt/libvirtd.log"
    ...


[*] https://kashyapc.fedorapeople.org/virt/openstack/request-nova-libvirt-qemu-debug-logs.txt

Comment 2 Lon Hohberger 2019-09-25 10:44:43 UTC
According to our records, this should be resolved by openstack-tripleo-heat-templates-8.3.1-87.el7ost.  This build is available now.


Note You need to log in before you can comment on or make changes to this bug.