RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 640202 - [libvirt] [logs] libvirt logs should be improved
Summary: [libvirt] [logs] libvirt logs should be improved
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: libvirt
Version: 6.1
Hardware: All
OS: Linux
urgent
medium
Target Milestone: rc
: ---
Assignee: Osier Yang
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks: 711206
TreeView+ depends on / blocked
 
Reported: 2010-10-05 08:08 UTC by Haim
Modified: 2014-01-13 00:47 UTC (History)
16 users (show)

Fixed In Version: libvirt-0.8.7-3.el6
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 711206 (view as bug list)
Environment:
Last Closed: 2011-05-19 13:22:32 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2011:0596 0 normal SHIPPED_LIVE libvirt bug fix and enhancement update 2011-05-18 17:56:36 UTC

Comment 2 Haim 2010-10-05 08:38:37 UTC
libvirt logs should be improved in order to allow proper analysis of daily issues. 
I would like to points for the following cases: 

1) threading - in order to tail a problem, one needs some kind of 'tagging' for single session (thread), in such, it will help us follow the issue from the beginning, till the end, and pin point the scenario\issue. 

2) failed actions does not refer to faulty domains (vm), and such, when we see an error, we cannot refer it to the correspondence domain, which makes debugging problem problematic. 

examples: 


10:38:19.638: error : qemudDomainBlockStats:9738 : Requested operation is not valid: domain is not running 

that explains the above for both vm identification and threading.

Comment 3 Daniel Berrangé 2010-11-10 23:36:47 UTC
It isn't practical to include the VM name/uuid/etc in every error message. We do however generate a log line at the entry to each public API call. This could be made to include the VM uuid/name. If we included a thread ID in every log line, you could then correlate the error log message, with the API entry point log message and identify the VM

Comment 4 Dan Kenigsberg 2010-11-11 09:29:03 UTC
comment 3's solution seems workable, isn't it, Haim?

btw, 

debug : virDomainLookupByUUID:2045 : conn=0x7f9eb0003300, uuid=<9F><FE>(<B6>a4K^^<88>^D^Q<85><F4><9C>Co

should better log the ASCII form of the domain uuid.

Comment 5 Haim 2010-11-14 22:00:09 UTC
(In reply to comment #4)
> comment 3's solution seems workable, isn't it, Haim?
> 
> btw, 
> 
> debug : virDomainLookupByUUID:2045 : conn=0x7f9eb0003300,
> uuid=<9F><FE>(<B6>a4K^^<88>^D^Q<85><F4><9C>Co
> 
> should better log the ASCII form of the domain uuid.

sounds good.

Comment 6 Daniel Berrangé 2010-11-23 11:04:20 UTC
One part of the fix is posted, to include a thread ID.

http://www.redhat.com/archives/libvir-list/2010-November/msg00947.html

More to come...

Comment 9 Osier Yang 2011-01-09 02:07:23 UTC
patch got pushed upstream: a98d8f0d274372d49915bf82dd09c82b156e5200

Comment 11 Vivian Bian 2011-01-21 12:23:35 UTC
tried the following things , but I'm not sure they are correct to get any conclusion to verify this bug :

1. set value of "log_level" in "/etc/libvirt/libvirtd.conf" as 4
2. set value of "log_outputs" in "/etc/libvirt/libvirtd.conf" as "1:stderr"
3. if libvirtd process is running, kill it
    # kill `pidof libvirtd`
4. start it in foreground:
   # libvirtd

   
A: with the old libvirt libvirt-0.8.7-1.el6.x86_64.rpm
   a. virsh destroy guest
   b. virsh start guest
20:31:16.079: 7838: warning : qemudDispatchSignalEvent:403 : Shutting down on signal 2
20:32:51.897: 8217: warning : qemudStartVMDaemon:3233 : Executing /usr/libexec/qemu-kvm
20:32:51.908: 8217: warning : qemudStartVMDaemon:3246 : Executing done /usr/libexec/qemu-kvm


B: with the new libvirt libvirt-0.8.7-3.el6
   a. virsh destroy guest
   b. virsh start guest
20:25:16.079: 7838: warning : qemudDispatchSignalEvent:403 : Shutting down on signal 2
20:25:16.403: 6453: warning : qemudStartVMDaemon:3239 : Executing /usr/libexec/qemu-kvm
20:25:16.413: 6453: warning : qemudStartVMDaemon:3249 : Executing done /usr/libexec/qemu-kvm


Seems there are no differences from two libvirtd.log . So would you please give me some suggestion on how to verify this bug ?

Comment 12 Osier Yang 2011-01-21 14:59:24 UTC
you should set log_level to "1", and check debug log.

Comment 13 Vivian Bian 2011-01-24 04:27:37 UTC
tried the following things , but I'm not sure they are correct to get any
conclusion to verify this bug :

1. set value of "log_level" in "/etc/libvirt/libvirtd.conf" as 1
2. set value of "log_outputs" in "/etc/libvirt/libvirtd.conf" as "1:file:libvirtd.log"
3. if libvirtd process is running, kill it
    # kill `pidof libvirtd`
4. start it in foreground:
   # libvirtd


A: with the old libvirt libvirt-0.8.7-1.el6.x86_64.rpm
   a. virsh destroy guest
   b. virsh start guest
   c. grep domain -i libvirtd.log

06:02:20.040: 4930: debug : virDomainLookupByUUID:2103 : conn=0x7f2218000bd0, uuid=<removed>


B: with the new libvirt libvirt-0.8.7-3.el6
   a. virsh destroy guest
   b. virsh start guest
   c. grep domain -i libvirtd.log

05:59:37.044: 4500: debug : virDomainFree:2294 : dom=0x7f3e8c0008e0, (VM: name=foo, uuid=79ce5c1a-4f45-4531-99e3-fc8a3a479072), 
05:59:37.046: 4497: debug : virDomainLookupByUUID:2124 : conn=0x7f3ea0000bd0, uuid=79ce5c1a-4f45-4531-99e3-fc8a3a479072
05:59:37.046: 4497: debug : virDomainFree:2294 : dom=0x7f3e9c000920, (VM: name=foo, uuid=79ce5c1a-4f45-4531-99e3-fc8a3a479072), 

from the above we can see the new libvirt rpm includes the VM name/uuid/etc in domain messages and resolved the garbled words problem . 
So set the bug status to VERIFIED

Comment 19 errata-xmlrpc 2011-05-19 13:22:32 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2011-0596.html

Comment 20 Eric Blake 2011-06-06 20:08:17 UTC
This patch introduced a regression, although it crashes the client and not libvirtd:
https://bugzilla.redhat.com/show_bug.cgi?id=711206


Note You need to log in before you can comment on or make changes to this bug.