Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1745005

Summary: [OSP-14] Configure log filter to capture interactions between libvirt and QEMU
Product: Red Hat OpenStack Reporter: Martin Schuppert <mschuppe>
Component: openstack-tripleo-heat-templatesAssignee: Martin Schuppert <mschuppe>
Status: CLOSED ERRATA QA Contact: Sasha Smolyak <ssmolyak>
Severity: medium Docs Contact:
Priority: medium    
Version: 14.0 (Rocky)CC: gveitmic, jhakimra, kchamart, mbooth, mburns, mschuppe, ratailor, ssmolyak
Target Milestone: z4Keywords: Triaged, ZStream
Target Release: 14.0 (Rocky)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: puppet-nova-13.3.2-0.20190426043944.d968cc2.el7ost openstack-tripleo-heat-templates-9.3.1-0.20190513171760.el7ost Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1745003
: 1745007 (view as bug list) Environment:
Last Closed: 2019-11-06 16:48:54 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1717359, 1745003    
Bug Blocks: 1745007    

Description Martin Schuppert 2019-08-23 13:03:09 UTC 
+++ This bug was initially created as a clone of Bug #1745003 +++

+++ This bug was initially created as a clone of Bug #1717359 +++

What?
-----

On all compute nodes, by default, enable the libvirt log filter
("1:qemu") to capture the QMP (QEMU Machine Protocol) traffic.  This
will show the QMP request and response flow between libvirt and QEMU.


Why?
----

Often times, when debugging issues that involve interactions between
Nova, libvirt and QEMU, we need to understand the precise requests that
libvirt is sending to QEMU, and the corresponding responses from QEMU.
Since we don't capture these interactions by default, we request them[*]
_after_ a bug is reported, at which point it might not be possible to
capture them for various reasons, or worse, the machine might no longer
be available.

Having this log filter enabled can be extremely useful in debugging a
range of scenarios: live migration (with shared and non-shared storage),
live snapshots, "volume migration", and anything that involves libvirt
sending QMP commands to QEMU.

Note that we're only selectively enabling a targetted log filter
("1:qemu").  It will slightly increase the log file size, but nothing
crazy.  This slight increase in log file size is acceptable, given the
benefits.


How?
----

When deploying compute nodes, make sure that TripleO sets the below two
configuration attributes in `/etc/libvirt/libvirtd.conf`.  The "1:qemu" 
log filter will capture the QMP traffic between libvirt and QEMU in the
specified log file, `libvirtd.log`:

    ...
    log_filters="1:qemu"
    log_outputs="1:file:/var/log/libvirt/libvirtd.log"
    ...


[*] https://kashyapc.fedorapeople.org/virt/openstack/request-nova-libvirt-qemu-debug-logs.txt
Comment 4 errata-xmlrpc 2019-11-06 16:48:54 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:3745