RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1771489 - libvirt log is not created on a RHEL8.1 host
Summary: libvirt log is not created on a RHEL8.1 host
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: libvirt
Version: 8.1
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: rc
: ---
Assignee: Michal Privoznik
QA Contact: Yanqiu Zhang
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-11-12 13:23 UTC by Avihai
Modified: 2020-01-23 16:30 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-01-23 16:30:54 UTC
Type: Bug
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)
messages and vdsm logs (618.31 KB, application/gzip)
2019-11-12 13:23 UTC, Avihai 		no flags Details
View All

Description Avihai 2019-11-12 13:23:06 UTC 
Created attachment 1635343 [details]
messages and vdsm logs

Description of problem:
RHEL8.1 ,I tried to get the log after seeing libvirt error in Bug 1771454 but no libvirt log exist under /var/log/libvirt.

Version-Release number of selected component (if applicable):
ovirt-engine-4.4.0-0.4.master.el7.noarch
vdsm-4.40.0-127.gitc628cce.el8ev.x86_64
python3-libvirt-5.0.0-5.module+el8.0.1+3755+6782b0ed.x86_64
qemu-kvm-3.1.0-30.module+el8.0.1+3755+6782b0ed.x86_64

[root@storage-ge4-vdsm1 ~]# cat /etc/redhat-release 
Red Hat Enterprise Linux release 8.1 (Ootpa)
[root@storage-ge4-vdsm1 ~]# uname -a
Linux storage-ge4-vdsm1.scl.lab.tlv.redhat.com 4.18.0-147.el8.x86_64 #1 SMP Thu Sep 26 15:52:44 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux


How reproducible:
100%

Steps to Reproduce:
1.try to find libvirt log exist under /var/log/libvirt

Actual results:
libvirt log does not exist under /var/log/libvirt

Expected results:
libvirt log should exist under /var/log/libvirt

Additional info:
At messages log I only see the following:
Nov 12 15:01:37 storage-ge4-vdsm1 journal[21161]: libvirt version: 5.0.0, package: 12.module+el8.0.1+3755+6782b0ed (Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>, 2019-07-25-00:51:52, )
Nov 12 15:01:37 storage-ge4-vdsm1 journal[21161]: hostname: storage-ge4-vdsm1.scl.lab.tlv.redhat.com
Nov 12 15:01:37 storage-ge4-vdsm1 journal[21161]: Domain id=1 name='vm_test1' uuid=4ba3afac-d81d-4433-b85f-5e4362b15681 is tainted: custom-ga-command
Nov 12 15:01:37 storage-ge4-vdsm1 journal[13943]: Cannot open log file: '/var/log/libvirt/qemu/vm_test1.log': Device or resource busy
Nov 12 15:01:37 storage-ge4-vdsm1 journal[21161]: Cannot open log file: '/var/log/libvirt/qemu/vm_test1.log': Device or resource busy


# Also the service status details:
[root@storage-ge4-vdsm1 ~]# systemctl status libvirtd 
● libvirtd.service - Virtualization daemon
   Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/libvirtd.service.d
           └─unlimited-core.conf
   Active: active (running) since Tue 2019-11-12 14:56:15 IST; 24min ago
     Docs: man:libvirtd(8)
           https://libvirt.org
 Main PID: 21161 (libvirtd)
    Tasks: 17 (limit: 32768)
   Memory: 14.5M
   CGroup: /system.slice/libvirtd.service
           └─21161 /usr/sbin/libvirtd --listen

Nov 12 14:56:12 storage-ge4-vdsm1.scl.lab.tlv.redhat.com systemd[1]: Stopped Virtualization daemon.
Nov 12 14:56:12 storage-ge4-vdsm1.scl.lab.tlv.redhat.com systemd[1]: Starting Virtualization daemon...
Nov 12 14:56:15 storage-ge4-vdsm1.scl.lab.tlv.redhat.com systemd[1]: Started Virtualization daemon.
Nov 12 15:01:37 storage-ge4-vdsm1.scl.lab.tlv.redhat.com libvirtd[21161]: libvirt version: 5.0.0, package: 12.module+el8.0.1+3755+6782b0ed (Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>, 2019-07-25-00:51>
Nov 12 15:01:37 storage-ge4-vdsm1.scl.lab.tlv.redhat.com libvirtd[21161]: hostname: storage-ge4-vdsm1.scl.lab.tlv.redhat.com
Nov 12 15:01:37 storage-ge4-vdsm1.scl.lab.tlv.redhat.com libvirtd[21161]: Domain id=1 name='vm_test1' uuid=4ba3afac-d81d-4433-b85f-5e4362b15681 is tainted: custom-ga-command
Nov 12 15:01:37 storage-ge4-vdsm1.scl.lab.tlv.redhat.com libvirtd[21161]: Cannot open log file: '/var/log/libvirt/qemu/vm_test1.log': Device or resource busy
Nov 12 15:01:37 storage-ge4-vdsm1.scl.lab.tlv.redhat.com libvirtd[21161]: Unable to open domainlog
Comment 1 Ryan Barry 2019-11-13 00:40:20 UTC 
Michal, is this a known problem?
Comment 2 Michal Privoznik 2019-11-13 15:52:39 UTC 
(In reply to Ryan Barry from comment #1)
> Michal, is this a known problem?

Smells like bug 1718707. I'm switching over to libvirt for investigation. Avihai, can you please share libvirtd debug log?
Comment 3 Avihai 2019-11-24 16:14:56 UTC 
(In reply to Michal Privoznik from comment #2)
> (In reply to Ryan Barry from comment #1)
> > Michal, is this a known problem?
> 
> Smells like bug 1718707. I'm switching over to libvirt for investigation.
> Avihai, can you please share libvirtd debug log?

This is all I can see on my hosts under /var/log/libvirt , no libvirtd debug log is seen :

[root@storage-ge4-vdsm1 ~]# cd /var/log/libvirt/
[root@storage-ge4-vdsm1 libvirt]# ls -ltr
total 12
drwx------. 2 root root 12288 Nov 24 03:12 qemu


sorry for the dumb Q, as I do not have libvirt.log due to this bug where can I find libvirtd debug log?

Looking at the service itself I see log output is incomplete or unavailable:
[root@storage-ge4-vdsm1 libvirt]# systemctl status libvirtd
● libvirtd.service - Virtualization daemon
   Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/libvirtd.service.d
           └─unlimited-core.conf
   Active: active (running) since Tue 2019-11-12 14:56:15 IST; 1 weeks 5 days ago
     Docs: man:libvirtd(8)
           https://libvirt.org
 Main PID: 21161 (libvirtd)
    Tasks: 18 (limit: 32768)
   Memory: 31.6M
   CGroup: /system.slice/libvirtd.service
           └─21161 /usr/sbin/libvirtd --listen

Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable.
Comment 4 Michal Privoznik 2019-11-25 09:47:00 UTC 
(In reply to Avihai from comment #3)
> (In reply to Michal Privoznik from comment #2)
> > (In reply to Ryan Barry from comment #1)
> > > Michal, is this a known problem?
> > 
> > Smells like bug 1718707. I'm switching over to libvirt for investigation.
> > Avihai, can you please share libvirtd debug log?
> 
> This is all I can see on my hosts under /var/log/libvirt , no libvirtd debug
> log is seen :
> 
> [root@storage-ge4-vdsm1 ~]# cd /var/log/libvirt/
> [root@storage-ge4-vdsm1 libvirt]# ls -ltr
> total 12
> drwx------. 2 root root 12288 Nov 24 03:12 qemu
> 
> 
> sorry for the dumb Q, as I do not have libvirt.log due to this bug where can
> I find libvirtd debug log?

You need to set it up:

https://wiki.libvirt.org/page/DebugLogs

> 
> Looking at the service itself I see log output is incomplete or unavailable:
> [root@storage-ge4-vdsm1 libvirt]# systemctl status libvirtd
> ● libvirtd.service - Virtualization daemon
>    Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor
> preset: enabled)
>   Drop-In: /etc/systemd/system/libvirtd.service.d
>            └─unlimited-core.conf
>    Active: active (running) since Tue 2019-11-12 14:56:15 IST; 1 weeks 5
> days ago
>      Docs: man:libvirtd(8)
>            https://libvirt.org
>  Main PID: 21161 (libvirtd)
>     Tasks: 18 (limit: 32768)
>    Memory: 31.6M
>    CGroup: /system.slice/libvirtd.service
>            └─21161 /usr/sbin/libvirtd --listen
> 
> Warning: Journal has been rotated since unit was started. Log output is
> incomplete or unavailable.

D'oh! By default libvirt will not collect any debug logs, and errors/warnings will go to journal. Well, I'll try to fix the referenced bug and see if it fixes this one too.
Comment 5 Michal Privoznik 2020-01-14 10:51:20 UTC 
So the referenced bug is fixed now. However, this looks like a misconfiguration bug to me. By default, debug logs are not collected hence there are no files under /var/log/libvirt/ and all errors/warnings are logged through journald. Avihai, can you confirm that when you enable debug logs (see comment 4) then the log file is actually created?
Comment 6 Avihai 2020-01-23 07:28:00 UTC 
(In reply to Michal Privoznik from comment #5)
> So the referenced bug is fixed now. However, this looks like a
> misconfiguration bug to me. By default, debug logs are not collected hence
> there are no files under /var/log/libvirt/ and all errors/warnings are
> logged through journald. Avihai, can you confirm that when you enable debug
> logs (see comment 4) then the log file is actually created?

Indeed the workaround to make libvirt log to appear in /var/log/libvirt/ was to enable debug logs.
Feel free to close this bug as NOTABUG if it's misconfiguration .

However I wonder why libvirt log is different from other logs that do appear without any needed additional configurations(at lease in info level or a short log to save only errors), maybe RFE is needed here?
Comment 7 Michal Privoznik 2020-01-23 16:30:54 UTC 
(In reply to Avihai from comment #6)
> (In reply to Michal Privoznik from comment #5)
> > So the referenced bug is fixed now. However, this looks like a
> > misconfiguration bug to me. By default, debug logs are not collected hence
> > there are no files under /var/log/libvirt/ and all errors/warnings are
> > logged through journald. Avihai, can you confirm that when you enable debug
> > logs (see comment 4) then the log file is actually created?
> 
> Indeed the workaround to make libvirt log to appear in /var/log/libvirt/ was
> to enable debug logs.
> Feel free to close this bug as NOTABUG if it's misconfiguration .
> 
> However I wonder why libvirt log is different from other logs that do appear
> without any needed additional configurations(at lease in info level or a
> short log to save only errors), maybe RFE is needed here?

Errors will appear in journal and so will warnings. But by default, debug logs are discarded because libvirt produces a lot of them.
Note You need to log in before you can comment on or make changes to this bug.