Bug 2218644

Summary: query-stats QMP command interrupts vcpus, the Max Latencies could be more than 100us (rhel 9.3.0 clone)
Product: Red Hat Enterprise Linux 9 Reporter: Marcelo Tosatti <mtosatti>
Component: qemu-kvmAssignee: Marcelo Tosatti <mtosatti>
qemu-kvm sub component: General QA Contact: Gu Nini <ngu>
Status: VERIFIED --- Docs Contact:
Severity: high    
Priority: high CC: alougovs, chayang, coli, jinzhao, junzhao, juzhang, lijin, ngu, pbonzini, pezhang, virt-maint, ymankad
Version: 9.3Keywords: CustomerScenariosInitiative, Triaged, ZStream
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Unspecified   
Whiteboard:
Fixed In Version: qemu-kvm-8.0.0-7.el9 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 2214884
: 2221219 (view as bug list) Environment:
Last Closed: Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 2214884    
Bug Blocks: 2221219    

Description Marcelo Tosatti 2023-06-29 17:53:56 UTC 
RHEL 9.3.0 / CentOS clone

+++ This bug was initially created as a clone of Bug #2214884 +++

Description of problem:
Under ocp4.13/ocp-v 4.13.1-188 env, the real time worker node is installed with RHCOS based on RHEL9.2, it's found "# Max Latencies: 00090 00096 00141" when run 1 hour cyclictest, while a maximum of 50us is acceptable. From Marcelo, it's caused by "query-stats QMP command interrupts vcpus.", "[PATCH] kvm: reuse per-vcpu stats fd to avoid vcpu interruption" was submitted.

Version-Release number of selected component (if applicable):
Host kernel: Scratch kernel 5.14.0-284.13.1.rt14.298kvmrtv1.el9_2.x86_64 
Guest kernel: Scratch kernel 5.14.0-284.13.1.rt14.298kvmrtv1.el9_2.x86_64
Qemu: qemu-kvm-core-7.2.0-14.el9_2.x86_64

How reproducible:
100%

Steps to Reproduce:
1. 
2.
3.

Actual results:

Expected results:

Additional info:
Comment 2 John Ferlan 2023-06-30 15:07:20 UTC 
Adding zstream and ZTR=9.2.0, see bug 2214884 for details, especially https://bugzilla.redhat.com/show_bug.cgi?id=2214884#c2
Comment 4 Miroslav Rezanina 2023-07-03 07:25:33 UTC 
*** Bug 2214884 has been marked as a duplicate of this bug. ***
Comment 9 Marcelo Tosatti 2023-07-10 11:19:30 UTC 
Test steps to verify bug:

1) Run qemu-kvm in parallel with rt-trace-bpf, with isolated vcpu.

2) Run query-stats command on QMP 
(https://patchew.org/Libvirt/20220624081449.70085-1-natto@weirdnatto.in/20220624081449.70085-3-natto@weirdnatto.in/), should see
IPIs to pcpu where isolated vcpu runs.

3) with fixed qemu-kvm, should be able to run query-stats QMP command and not see the IPIs.
Comment 10 Yanan Fu 2023-07-11 09:11:35 UTC 
QE bot(pre verify): Set 'Verified:Tested,SanityOnly' as gating/tier1 test pass.