RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2174446 - Logs are spammed with ' Domain id=7 is tainted: custom-ga-command' message
Summary: Logs are spammed with ' Domain id=7 is tainted: custom-ga-command' message
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 9
Classification: Red Hat
Component: libvirt
Version: 9.0
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Peter Krempa
QA Contact: Lili Zhu
URL:
Whiteboard:
Depends On:
Blocks: 2174447 2176924
TreeView+ depends on / blocked
 
Reported: 2023-03-01 16:18 UTC by Peter Krempa
Modified: 2023-05-09 08:13 UTC (History)
7 users (show)

Fixed In Version: libvirt-9.0.0-8.el9_2
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 2174447 2176924 (view as bug list)
Environment:
Last Closed: 2023-05-09 07:27:59 UTC
Type: Bug
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHELPLAN-150321 0 None None None 2023-03-01 16:19:52 UTC
Red Hat Product Errata RHBA-2023:2171 0 None None None 2023-05-09 07:28:37 UTC

Description Peter Krempa 2023-03-01 16:18:25 UTC 
Description of problem:
When a management APP is using agent command passhthrough the logs are spammed with:

 Domain id=7 is tainted: custom-ga-command

over and over again.

Version-Release number of selected component (if applicable):
introduced in libvirt 7.1

How reproducible:
always

Steps to Reproduce:
1. Start VM
2. repeatedly invoke command such as:
$ virsh qemu-agent-command ha '{"execute":"guest-get-vcpus"}'

3. observe logs being spammed

Actual results:


Expected results:


Additional info:
Comment 1 Peter Krempa 2023-03-06 13:53:31 UTC 
Fixed upstream:

commit 9134b40d0b43a5e1a9928b0a0d948205941d9807
Author: Peter Krempa <pkrempa>
Date:   Wed Mar 1 17:09:42 2023 +0100

    qemu: domain: Fix logic when tainting domain
    
    Originally the code was skipping all repeated taints with the same taint
    flag but a logic bug introduced in commit 30626ed15b239c424ae inverted
    the condition. This caused that actually the first occurence was NOT
    logged but any subsequent was.
    
    This was noticed when going through oVirt logs as they use custom guest
    agent commands and the logs are totally spammed with this message.
    
    Fixes: 30626ed15b239c424ae891f096057a696eadd715
    Signed-off-by: Peter Krempa <pkrempa>
    Reviewed-by: Laine Stump <laine>

v9.1.0-11-g9134b40d0b
Comment 8 Lili Zhu 2023-03-11 07:22:10 UTC 
Rreproduced this bug with:
libvirt-9.0.0-7.el9.x86_64

1. prepare a guest with guest agent
# virsh domtime vm1
Time: 1678509372

2. check the some info about guest using qemu-agent-command
# virsh qemu-agent-command vm1 '{"execute":"guest-get-fsinfo"}'
{"return":[{"name":"vda1","total-bytes":627875840,...

3.check the log
# grep -ni tainted /var/log/libvirt/libvirtd.log 
(no output)

4. repeat step 2, check the log again
# grep -ni tainted /var/log/libvirt/libvirtd.log 
48494:2023-03-11 07:17:05.341+0000: 40743: warning : qemuDomainObjTaintMsg:6618 : Domain id=1 name='vm1' uuid=479df013-c11c-4109-98bf-461bb6a525bb is tainted: custom-ga-command
48496:2023-03-11 07:17:05.341+0000: 40743: debug : qemuDomainLogAppendMessage:6990 : Append log message (vm='vm1' message='2023-03-11 07:17:05.341+0000: Domain id=1 is tainted: custom-ga-command

5. repat step2, and check the log again
# grep -ni tainted /var/log/libvirt/libvirtd.log 
48494:2023-03-11 07:17:05.341+0000: 40743: warning : qemuDomainObjTaintMsg:6618 : Domain id=1 name='vm1' uuid=479df013-c11c-4109-98bf-461bb6a525bb is tainted: custom-ga-command
48496:2023-03-11 07:17:05.341+0000: 40743: debug : qemuDomainLogAppendMessage:6990 : Append log message (vm='vm1' message='2023-03-11 07:17:05.341+0000: Domain id=1 is tainted: custom-ga-command
[root@dell-per640-07 ~]# grep -ni tainted /var/log/libvirt/libvirtd.log 
48494:2023-03-11 07:17:05.341+0000: 40743: warning : qemuDomainObjTaintMsg:6618 : Domain id=1 name='vm1' uuid=479df013-c11c-4109-98bf-461bb6a525bb is tainted: custom-ga-command
48496:2023-03-11 07:17:05.341+0000: 40743: debug : qemuDomainLogAppendMessage:6990 : Append log message (vm='vm1' message='2023-03-11 07:17:05.341+0000: Domain id=1 is tainted: custom-ga-command
49475:2023-03-11 07:18:17.349+0000: 40740: warning : qemuDomainObjTaintMsg:6618 : Domain id=1 name='vm1' uuid=479df013-c11c-4109-98bf-461bb6a525bb is tainted: custom-ga-command
49477:2023-03-11 07:18:17.349+0000: 40740: debug : qemuDomainLogAppendMessage:6990 : Append log message (vm='vm1' message='2023-03-11 07:18:17.349+0000: Domain id=1 is tainted: custom-ga-command

8. repeat step2 for several times and check the log
# grep -ni tainted /var/log/libvirt/libvirtd.log 
48494:2023-03-11 07:17:05.341+0000: 40743: warning : qemuDomainObjTaintMsg:6618 : Domain id=1 name='vm1' uuid=479df013-c11c-4109-98bf-461bb6a525bb is tainted: custom-ga-command
48496:2023-03-11 07:17:05.341+0000: 40743: debug : qemuDomainLogAppendMessage:6990 : Append log message (vm='vm1' message='2023-03-11 07:17:05.341+0000: Domain id=1 is tainted: custom-ga-command
49475:2023-03-11 07:18:17.349+0000: 40740: warning : qemuDomainObjTaintMsg:6618 : Domain id=1 name='vm1' uuid=479df013-c11c-4109-98bf-461bb6a525bb is tainted: custom-ga-command
49477:2023-03-11 07:18:17.349+0000: 40740: debug : qemuDomainLogAppendMessage:6990 : Append log message (vm='vm1' message='2023-03-11 07:18:17.349+0000: Domain id=1 is tainted: custom-ga-command
50456:2023-03-11 07:19:35.177+0000: 40739: warning : qemuDomainObjTaintMsg:6618 : Domain id=1 name='vm1' uuid=479df013-c11c-4109-98bf-461bb6a525bb is tainted: custom-ga-command
50458:2023-03-11 07:19:35.177+0000: 40739: debug : qemuDomainLogAppendMessage:6990 : Append log message (vm='vm1' message='2023-03-11 07:19:35.177+0000: Domain id=1 is tainted: custom-ga-command
51437:2023-03-11 07:20:07.119+0000: 40741: warning : qemuDomainObjTaintMsg:6618 : Domain id=1 name='vm1' uuid=479df013-c11c-4109-98bf-461bb6a525bb is tainted: custom-ga-command
51439:2023-03-11 07:20:07.119+0000: 40741: debug : qemuDomainLogAppendMessage:6990 : Append log message (vm='vm1' message='2023-03-11 07:20:07.119+0000: Domain id=1 is tainted: custom-ga-command
52418:2023-03-11 07:20:21.750+0000: 40742: warning : qemuDomainObjTaintMsg:6618 : Domain id=1 name='vm1' uuid=479df013-c11c-4109-98bf-461bb6a525bb is tainted: custom-ga-command
52420:2023-03-11 07:20:21.750+0000: 40742: debug : qemuDomainLogAppendMessage:6990 : Append log message (vm='vm1' message='2023-03-11 07:20:21.750+0000: Domain id=1 is tainted: custom-ga-command
53399:2023-03-11 07:20:26.791+0000: 40743: warning : qemuDomainObjTaintMsg:6618 : Domain id=1 name='vm1' uuid=479df013-c11c-4109-98bf-461bb6a525bb is tainted: custom-ga-command
53401:2023-03-11 07:20:26.791+0000: 40743: debug : qemuDomainLogAppendMessage:6990 : Append log message (vm='vm1' message='2023-03-11 07:20:26.791+0000: Domain id=1 is tainted: custom-ga-command

(logs being spammed)
Comment 9 Lili Zhu 2023-03-11 07:56:27 UTC 
Verify this bug with:
libvirt-9.0.0-8.el9_2.x86_64

1. prepare a guest with guest agent
# virsh domtime vm1
Time: 1678521142

2. check the some info about guest using qemu-agent-command
# virsh qemu-agent-command vm1 '{"execute":"guest-get-fsinfo"}'
{"return":[{"name":"vda1","total-bytes":627875840,...

3.check the log
# grep -ni tainted /var/log/libvirt/libvirtd.log 
549:2023-03-11 07:52:36.284+0000: 42237: warning : qemuDomainObjTaintMsg:6618 : Domain id=1 name='vm1' uuid=2fa63f27-2c87-4125-beaf-46d5d087c4e3 is tainted: custom-ga-command
551:2023-03-11 07:52:36.284+0000: 42237: debug : qemuDomainLogAppendMessage:6990 : Append log message (vm='vm1' message='2023-03-11 07:52:36.284+0000: Domain id=1 is tainted: custom-ga-command

4. repeat step 2 for 100 times
# for i in {1..100}; do virsh qemu-agent-command vm1 '{"execute":"guest-get-fsinfo"}'; done
....

5. check the log
# grep -ni tainted /var/log/libvirt/libvirtd.log 
549:2023-03-11 07:52:36.284+0000: 42237: warning : qemuDomainObjTaintMsg:6618 : Domain id=1 name='vm1' uuid=2fa63f27-2c87-4125-beaf-46d5d087c4e3 is tainted: custom-ga-command
551:2023-03-11 07:52:36.284+0000: 42237: debug : qemuDomainLogAppendMessage:6990 : Append log message (vm='vm1' message='2023-03-11 07:52:36.284+0000: Domain id=1 is tainted: custom-ga-command

(no new logs about tainted messages, logs are not spammed)
Comment 12 Lili Zhu 2023-03-23 03:18:24 UTC 
According to Comment #9, mark the bug as verified.
Comment 14 errata-xmlrpc 2023-05-09 07:27:59 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (libvirt bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:2171
Note You need to log in before you can comment on or make changes to this bug.