Bug 1376009

Summary: [RFE] allow setting of log level for gluster driver (libvirt)
Product: Red Hat Enterprise Linux 7 Reporter: Sahina Bose <sabose>
Component: libvirtAssignee: Peter Krempa <pkrempa>
Status: CLOSED ERRATA QA Contact: lijuan men <lmen>
Severity: medium Docs Contact:
Priority: high    
Version: 7.3CC: dyuan, jcody, jdenemar, jsuchane, mtessun, pkrempa, rbalakri, xuzhang
Target Milestone: rcKeywords: FutureFeature
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: libvirt-2.5.0-1.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-08-01 17:16:43 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1151859, 1360301    
Bug Blocks:    

Description Sahina Bose 2016-09-14 12:50:39 UTC
Description of problem:

When guest images are accessed via gluster driver (libgfapi), we need a way to specify the log level and log file. This is needed for supportability of product

Version-Release number of selected component (if applicable):


How reproducible:
NA

Steps to Reproduce:
NA

Actual results:


Expected results:


Additional info:

Comment 1 Daniel Berrangé 2016-09-14 13:07:30 UTC
Using the gluster logfile facility is not an option in libvirt. We cannot trust QEMU (and by implication libfgapi) to do logging safely. Having the libgfapi logs to to stderr allows them to be captured by default with other QEMU console output, which is now managed by virtlogd to provided guarantee file rotation and size limiting to prevent a DOS attack by QEMU.  At most we should consider making it possible to configure the log level via /etc/libvirt/qemu.conf, and leave the output on stderr.

Comment 3 Prasanna Kumar Kalever 2016-09-15 08:53:09 UTC
Thanks for chiming here Dan, the info is really helpful +1 for that.

We can find the logs of qemu+gfapi sent to stderr in one among the following files 'usr/local/var/log/libvirt/qemu/GuestName.log' or /var/log/libvirt/qemu/GuestName.log

But as Dan said rightly, we may need Log/Debug Level tuning from the libvirt side.

I shall try to send a first cut of patch to libvirt some time today.

Comment 4 Prasanna Kumar Kalever 2016-09-15 10:04:28 UTC
please-review:
https://www.redhat.com/archives/libvir-list/2016-September/msg00485.html

Comment 5 Jaroslav Suchanek 2016-09-19 08:00:18 UTC
(In reply to Prasanna Kumar Kalever from comment #4)
> please-review:
> https://www.redhat.com/archives/libvir-list/2016-September/msg00485.html

Please move bug to POST only if the patch is pushed to master and was done before rebase deadline. After rebase move the bug to POST after you posted a downstream backport of the fix. Thanks.

Comment 6 Peter Krempa 2016-11-09 16:02:40 UTC
The feature was added upstream:

commit e66603539b9b8b24c737b40805b4bbd99369b6b8
Author: Prasanna Kumar Kalever <prasanna.kalever>
Date:   Fri Oct 14 05:37:45 2016 +0200

    qemu: command: Add debug option for gluster volumes
    
    Propagate the selected or default level to qemu if it's supported.
    
    Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1376009
    
    Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever>
    Signed-off-by: Peter Krempa <pkrempa>

commit a944bd925902d9ecce81e08900ad6a1adee06c6c
Author: Prasanna Kumar Kalever <prasanna.kalever>
Date:   Thu Sep 22 01:04:18 2016 +0530

    qemu: conf: add option for tuning debug logging level
    
    This helps in selecting log level of the gluster gfapi, output to stderr.
    The option is 'gluster_debug_level', can be tuned by editing
    '/etc/libvirt/qemu.conf'
    
    Debug levels ranges 0-9, with 9 being the most verbose, and 0
    representing no debugging output.  The default is the same as it was
    before, which is a level of 4.  The current logging levels defined in
    the gluster gfapi are:
    
        0 - None
        1 - Emergency
        2 - Alert
        3 - Critical
        4 - Error
        5 - Warning
        6 - Notice
        7 - Info
        8 - Debug
        9 - Trace
    
    Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever>
    Signed-off-by: Peter Krempa <pkrempa>

Comment 8 lijuan men 2017-02-23 06:52:59 UTC
I have tested 4 scenarios.scenario1-2 are normal,scenario 3-4 are not normal,I think.

summary for the 4 scenarios:

scenario1: keep gluster_debug_level as the default value(4),start a guest with gluster disk,look at the qemu command. file.debug=4 in the qemu command,it is right.

scenario2:set gluster_debug_level as one number in 0-9(such as 9),start a guest with gluster disk,look at the qemu command. file.debug=9(if I set it as 9) in the qemu command,it is right.

scenario3:set gluster_debug_level as one number **out of** 0-9(such as 20),start a guest with gluster disk,look at the qemu command. file.debug=20(if I set it as 20) in the qemu command. But as it is said in /etc/libvirt/qemu.conf,"Qemu gluster libgfapi log level, debug levels are 0-9".So I think,setting it as 20(more than 9) is not legal.Can we prevent this kind of setting?

scenario4:set gluster_debug_level as a character(such as 'a')or negative number(such as -1),restart libvirtd,then libvirtd will die.As scenario3,can we prevent this kind of setting?



detailed steps for the 4 scenarios:

version:
libvirt-3.0.0-1.el7.x86_64
qemu-kvm-rhev-2.8.0-3.el7.x86_64

scenario1:
1.don't set the value of gluster_debug_level,keep it as the default value.
#cat /etc/libvirt/qemu.conf
...
#gluster_debug_level = 4
...

2.restart the libvirtd
# systemctl restart libvirtd

3.start a guest with xml:
...
<disk type='network' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source protocol='gluster' name='test/test.qcow2'>
        <host name='10.66.70.107'/>
      </source>
      <target dev='vda' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/>
    </disk>
...

# virsh start qq

4.look at the qemu command

# ps -ef | grep qq
...
-drive file=gluster://10.66.70.107/test/test.qcow2,***file.debug=4***,format=qcow2,if=none,id=drive-virtio-disk0,cache=none 
...


scenario2:
1.set gluster_debug_level as 9 in /etc/libvirt/qemu.conf
#cat /etc/libvirt/qemu.conf
...
gluster_debug_level = 9
...

2.restart libvirtd
# systemctl restart libvirtd

3.restart the guest
# virsh destroy qq
# virsh start qq

4.look at the qemu command
-drive file=gluster://10.66.70.107/test/test.qcow2,***file.debug=9***,format=qcow2,if=none,id=drive-virtio-disk0,cache=none


scenario3:
1.set gluster_debug_level as **20(more than 9)** in /etc/libvirt/qemu.conf
#cat /etc/libvirt/qemu.conf
...
gluster_debug_level = 20
...

2.restart libvirtd
# systemctl restart libvirtd

3.restart the guest
# virsh destroy qq
# virsh start qq

4.look at the qemu command
-drive file=gluster://10.66.70.107/test/test.qcow2,***file.debug=20***,format=qcow2,if=none,id=drive-virtio-disk0,cache=none


scenario4:
1.set gluster_debug_level as 'a'(or -1 0.1) in /etc/libvirt/qemu.conf
#cat /etc/libvirt/qemu.conf
...
gluster_debug_level = a
...

2.restart libvirtd

# systemctl restart libvirtd

# systemctl status libvirtd
● libvirtd.service - Virtualization daemon
   Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor preset: enabled)
   Active: ***inactive (dead)**** since Thu 2017-02-23 01:12:47 EST; 3s ago
     Docs: man:libvirtd(8)

Comment 9 Peter Krempa 2017-02-24 14:23:32 UTC
Scenario 3:
I don't think it's worthwhile adding a range check here. The only option how to set it documents the values which are accepted and this config option is for debugging purposes only mostly.

Scenario 4:
Using anything besides a signed integer is checked since it's parsed into an unsigned variable. It's expected that the daemon does not start in such case since the config file can't be parsed.

Comment 10 lijuan men 2017-04-13 06:35:25 UTC
(In reply to Peter Krempa from comment #9)
> Scenario 3:
> I don't think it's worthwhile adding a range check here. The only option how
> to set it documents the values which are accepted and this config option is
> for debugging purposes only mostly.
> 
> Scenario 4:
> Using anything besides a signed integer is checked since it's parsed into an
> unsigned variable. It's expected that the daemon does not start in such case
> since the config file can't be parsed.

ok
thanks for your reply
mark the bug as verified

Comment 11 errata-xmlrpc 2017-08-01 17:16:43 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1846

Comment 12 errata-xmlrpc 2017-08-01 23:57:38 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1846