Bug 1556724
| Summary: | tuned cpu-partitioning profile, disable cstates with processor.max_cstate=0 intel_idle.max_cstate=0 | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Andreas Karis <akaris> |
| Component: | tuned | Assignee: | Jaroslav Škarvada <jskarvad> |
| Status: | CLOSED WONTFIX | QA Contact: | qe-baseos-daemons |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 7.4 | CC: | akaris, chorn, jeder, jraju, jskarvad, marjones, olysonek, thozza |
| Target Milestone: | rc | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2018-06-21 12:28:27 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 1549614 | ||
|
Description
Andreas Karis
2018-03-15 06:57:49 UTC
(In reply to Andreas Karis from comment #0) > Description of problem: > tuned cpu-partitioning profile, disable cstates with processor.max_cstate=0 > intel_idle.max_cstate=0 > We currently lock the CPU to C1, not to C0 according to the feedback from the performance team, btw it also seems that the kernel driver sets the C state to the C1, even if you request C0 from the kernel command line (processor_idle.c). Tuned sets the C state through the /dev/cpu_dma_latency not through the kernel command line for greater flexibility. I.e. customer can unlock the CPU from the C1 state by simply switching to different Tuned profile and save power, she/he doesn't need to reboot the machine, for details see e.g.: https://jeremyeder.com/2012/11/14/processor-max_cstate-intel_idle-max_cstate-and-devcpu_dma_latency/ So I don't think we need to change it. Hello, The cpu-partitioning profile lets CPUs go into C1 state, and that pretty often. E.g. from a customer system with tuned cpu-partitioning: ~~~ 10.009961 sec Package Core CPU Avg_MHz Busy% Bzy_MHz TSC_MHz IRQ SMI CPU%c1 CPU%c3 CPU%c6 CPU%c7 CoreTmp PkgTmp Pkg%pc2 Pkg%pc3 Pkg%pc6 PkgWatt RAMWatt PKG_% RAM_% - - - 576 20.58 2800 2198 0 7040 79.42 0.00 0.00 0.00 77 77 0.00 0.00 0.00 175.39 31.82 0.00 0.00 0 0 0 2797 100.00 2800 2198 0 80 0.00 0.00 0.00 0.00 65 70 0.00 0.00 0.00 82.37 17.63 0.00 0.00 0 0 44 883 31.56 2800 2198 0 80 68.44 0 1 1 170 6.06 2800 2198 0 80 93.94 0.00 0.00 0.00 63 0 1 45 632 22.60 2800 2198 0 80 77.40 0 2 2 145 5.20 2800 2198 0 80 94.80 0.00 0.00 0.00 66 0 2 46 254 9.09 2800 2198 0 80 90.91 0 3 3 209 7.49 2800 2198 0 80 92.51 0.00 0.00 0.00 64 ~~~ Or from my lab: ~~~ root@overcloud-compute-0 ~]# turbostat Core CPU Avg_MHz Busy% Bzy_MHz TSC_MHz IRQ SMI CPU%c1 CPU%c3 CPU%c6 CPU%c7 CoreTmp PkgTmp PkgWatt RAMWatt PKG_% RAM_% - - 722 30.11 2400 2197 8207 0 69.89 0.00 0.00 0.00 69 69 78.53 9.39 0.00 0.00 0 0 20 0.85 2400 2197 5313 0 99.15 0.00 0.00 0.00 62 69 43.97 3.85 0.00 0.00 0 20 21 0.89 2400 2197 499 0 99.11 1 2 2397 100.00 2400 2197 9 0 0.00 0.00 0.00 0.00 66 1 22 2397 100.00 2400 2197 9 0 0.00 2 4 2397 100.00 2400 2197 9 0 0.00 0.00 0.00 0.00 69 2 24 2397 100.00 2400 2197 9 0 0.00 3 6 2397 100.00 2400 2197 9 0 0.00 0.00 0.00 0.00 68 3 26 2397 100.00 2400 2197 9 0 0.00 4 8 2397 100.00 2400 2197 9 0 0.00 0.00 0.00 0.00 67 4 28 2397 100.00 2400 2197 9 0 0.00 8 10 0 0.01 2400 2197 6 0 99.99 0.00 0.00 0.00 61 8 30 0 0.00 2400 2197 4 0 100.00 9 12 0 0.01 2400 2197 16 0 99.99 0.00 0.00 0.00 61 9 32 0 0.00 2400 2197 4 0 100.00 10 14 0 0.00 2399 2197 3 0 100.00 0.00 0.00 0.00 60 10 34 0 0.01 2400 2197 20 0 99.99 11 16 0 0.01 2400 2197 14 0 99.99 0.00 0.00 0.00 60 11 36 0 0.00 2400 2197 4 0 100.00 12 18 0 0.00 2400 2197 6 0 100.00 0.00 0.00 0.00 60 12 38 0 0.00 2401 2197 4 0 100.00 0 1 23 0.98 2400 2197 1419 0 99.02 0.00 0.00 0.00 61 64 34.56 5.54 0.00 0.00 0 21 37 1.53 2400 2197 687 0 98.47 1 3 2397 100.00 2400 2197 9 0 0.00 0.00 0.00 0.00 65 1 23 2397 100.00 2400 2197 9 0 0.00 2 5 2397 100.00 2400 2197 9 0 0.00 0.00 0.00 0.00 64 ~~~ ~~~ [root@overcloud-compute-0 ~]# cat /proc/cmdline BOOT_IMAGE=/boot/vmlinuz-3.10.0-693.17.1.el7.x86_64 root=UUID=fa414390-f78d-49d4-a164-54615a32977b ro console=tty0 console=ttyS0,115200n8 crashkernel=auto rhgb quiet default_hugepagesz=1GB hugepagesz=1G hugepages=32 iommu=pt intel_iommu=on isolcpus=2,4,6,8,10,12,14,16,18,22,24,26,28,30,32,34,36,38,3,5,7,9,11,13,15,17,19,23,25,27,29,31,33,35,37,39 nohz=on nohz_full=2,4,6,8,10,12,14,16,18,22,24,26,28,30,32,34,36,38,3,5,7,9,11,13,15,17,19,23,25,27,29,31,33,35,37,39 rcu_nocbs=2,4,6,8,10,12,14,16,18,22,24,26,28,30,32,34,36,38,3,5,7,9,11,13,15,17,19,23,25,27,29,31,33,35,37,39 tuned.non_isolcpus=00300003 intel_pstate=disable nosoftlockup [root@overcloud-compute-0 ~]# ~~~ Only the PMDs don't sleep (which is logical, they poll constantly). However, with the cpu-partitioning profile, none of the CPUs should ever go to C1, or the instance vCPUs won't be able to pull packets fast enough from the virtio ring and will lead to spurious drops on the vhost-user (vhu) ports in OVS. If the instance is using the standard virtio driver, that is. Agreed on locking to C1, I think that locking to C0 doesn't work, anyhow. Both https://access.redhat.com/solutions/202743 and https://access.redhat.com/articles/65410 recommend processor.max_cstates=1 https://access.redhat.com/articles/65410 additionally recommends idle=poll ~~~ A system may be prevented from entering power-saving states by booting with the processor.max_cstates=1 command line option. Additionally, the idle=poll option may be added for the fastest time out of the idle state. Unfortunately both of these options tend to cause power usage to spike. ~~~ https://www.kernel.org/doc/Documentation/admin-guide/kernel-parameters.txt list the 3 parameters: ~~~ intel_idle.max_cstate= [KNL,HW,ACPI,X86] 0 disables intel_idle and fall back on acpi_idle. 1 to 9 specify maximum depth of C-state. ~~~ ~~~ processor.max_cstate= [HW,ACPI] Limit processor to maximum C-state max_cstate=9 overrides any DMI blacklist limit. ~~~ ~~~ idle= [X86] Format: idle=poll, idle=halt, idle=nomwait Poll forces a polling idle loop that can slightly improve the performance of waking up a idle CPU, but will use a lot of power and make the system run hot. Not recommended. idle=halt: Halt is forced to be used for CPU idle. In such case C2/C3 won't be used again. idle=nomwait: Disable mwait for CPU C-states ~~~ I just tested out several variations: ~~~ processor.max_cstate=0 intel_idle.max_cstate=0 processor.max_cstate=0 intel_idle.max_cstate=0 processor.max_cstate=1 intel_idle.max_cstate=0 idle=poll processor.max_cstate=1 intel_idle.max_cstate=0 idle=poll ~~~ Only 2 yield C0 states only: ~~~ processor.max_cstate=0 intel_idle.max_cstate=0 idle=poll processor.max_cstate=1 intel_idle.max_cstate=0 idle=poll ~~~ The important point here is `idle=poll`. ~~~ [root@overcloud-compute-0 ~]# lscpu | grep 'Model name' Model name: Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz [root@overcloud-compute-0 ~]# cat /proc/cmdline BOOT_IMAGE=/boot/vmlinuz-3.10.0-693.17.1.el7.x86_64 root=UUID=fa414390-f78d-49d4-a164-54615a32977b ro console=tty0 console=ttyS0,115200n8 crashkernel=auto rhgb quiet default_hugepagesz=1GB hugepagesz=1G hugepages=32 iommu=pt intel_iommu=on isolcpus=2,4,6,8,10,12,14,16,18,22,24,26,28,30,32,34,36,38,3,5,7,9,11,13,15,17,19,23,25,27,29,31,33,35,37,39 processor.max_cstate=1 intel_idle.max_cstate=0 idle=poll nohz=on nohz_full=2,4,6,8,10,12,14,16,18,22,24,26,28,30,32,34,36,38,3,5,7,9,11,13,15,17,19,23,25,27,29,31,33,35,37,39 rcu_nocbs=2,4,6,8,10,12,14,16,18,22,24,26,28,30,32,34,36,38,3,5,7,9,11,13,15,17,19,23,25,27,29,31,33,35,37,39 tuned.non_isolcpus=00300003 intel_pstate=disable nosoftlockup [root@overcloud-compute-0 ~]# turbostat 1 0.023338 sec Core CPU Avg_MHz Busy% Bzy_MHz TSC_MHz IRQ SMI CPU%c1 CPU%c3 CPU%c6 CPU%c7 CoreTmp PkgTmp PkgWatt RAMWatt PKG_% RAM_% - - 2294 100.00 2400 2103 0 0 0.00 0.00 0.00 0.00 68 71 48.48 5.79 0.00 0.00 0 0 140 100.00 2400 128 0 0 0.00 0.00 0.00 0.00 64 71 4.19 0.33 0.00 0.00 0 20 140 100.00 2400 128 0 0 0.00 1 2 2326 100.00 2400 2132 0 0 0.00 0.00 0.00 0.00 66 1 22 2425 100.00 2400 2223 0 0 0.00 2 4 2424 100.00 2400 2222 0 0 0.00 0.00 0.00 0.00 67 2 24 2424 100.00 2400 2222 0 0 0.00 3 6 2422 100.00 2400 2220 0 0 0.00 0.00 0.00 0.00 68 3 26 2421 100.00 2400 2220 0 0 0.00 4 8 2420 100.00 2400 2218 0 0 0.00 0.00 0.00 0.00 66 4 28 2419 100.00 2400 2217 0 0 0.00 8 10 2419 100.00 2400 2218 0 0 0.00 0.00 0.00 0.00 65 8 30 2418 100.00 2400 2216 0 0 0.00 9 12 2416 100.00 2400 2215 0 0 0.00 0.00 0.00 0.00 65 9 32 2415 100.00 2400 2214 0 0 0.00 10 14 2414 100.00 2400 2213 0 0 0.00 0.00 0.00 0.00 66 10 34 2413 100.00 2400 2212 0 0 0.00 11 16 2412 100.00 2400 2211 0 0 0.00 0.00 0.00 0.00 65 11 36 2411 100.00 2400 2210 0 0 0.00 12 18 2409 100.00 2400 2209 0 0 0.00 0.00 0.00 0.00 64 12 38 2409 100.00 2400 2208 0 0 0.00 0 1 2408 100.00 2400 2207 0 0 0.00 0.00 0.00 0.00 67 70 44.29 5.45 0.00 0.00 0 21 2409 100.00 2400 2208 0 0 0.00 1 3 2408 100.00 2400 2207 0 0 0.00 0.00 0.00 0.00 68 1 23 2407 100.00 2400 2207 0 0 0.00 2 5 2406 100.00 2400 2205 0 0 0.00 0.00 0.00 0.00 68 2 25 2405 100.00 2400 2204 0 0 0.00 3 7 2404 100.00 2400 2204 0 0 0.00 0.00 0.00 0.00 66 3 27 2404 100.00 2400 2203 0 0 0.00 4 9 2403 100.00 2400 2203 0 0 0.00 0.00 0.00 0.00 67 4 29 2402 100.00 2400 2202 0 0 0.00 8 11 2402 100.00 2400 2202 0 0 0.00 0.00 0.00 0.00 66 8 31 2402 100.00 2400 2202 0 0 0.00 9 13 2401 100.00 2400 2201 0 0 0.00 0.00 0.00 0.00 66 9 33 2401 100.00 2400 2201 0 0 0.00 10 15 2400 100.00 2400 2200 0 0 0.00 0.00 0.00 0.00 66 10 35 2399 100.00 2400 2199 0 0 0.00 11 17 2399 100.00 2400 2199 0 0 0.00 0.00 0.00 0.00 67 11 37 2398 100.00 2400 2198 0 0 0.00 12 19 2397 100.00 2400 2197 0 0 0.00 0.00 0.00 0.00 66 12 39 2396 100.00 2400 2196 0 0 0.00 0.025176 sec Core CPU Avg_MHz Busy% Bzy_MHz TSC_MHz IRQ SMI CPU%c1 CPU%c3 CPU%c6 CPU%c7 CoreTmp PkgTmp PkgWatt RAMWatt PKG_% RAM_% - - 2412 100.00 2400 2211 0 0 0.00 0.00 0.00 0.00 68 71 94.11 9.51 0.00 0.00 0 0 2429 100.00 2400 2227 0 0 0.00 0.00 0.00 0.00 64 71 49.51 4.02 0.00 0.00 0 20 2429 100.00 2400 2227 0 0 0.00 1 2 2428 100.00 2400 2226 0 0 0.00 0.00 0.00 0.00 66 1 22 2427 100.00 2400 2225 0 0 0.00 2 4 2426 100.00 2400 2224 0 0 0.00 0.00 0.00 0.00 67 2 24 2425 100.00 2400 2223 0 0 0.00 3 6 2424 100.00 2400 2222 0 0 0.00 0.00 0.00 0.00 68 3 26 2422 100.00 2400 2220 0 0 0.00 4 8 2421 100.00 2400 2219 0 0 0.00 0.00 0.00 0.00 66 4 28 2420 100.00 2400 2219 0 0 0.00 8 10 2419 100.00 2400 2218 0 0 0.00 0.00 0.00 0.00 65 8 30 2418 100.00 2400 2216 0 0 0.00 9 12 2417 100.00 2400 2215 0 0 0.00 0.00 0.00 0.00 65 9 32 2416 100.00 2400 2215 0 0 0.00 10 14 2415 100.00 2400 2213 0 0 0.00 0.00 0.00 0.00 66 10 34 2414 100.00 2400 2213 0 0 0.00 11 16 2413 100.00 2400 2212 0 0 0.00 0.00 0.00 0.00 65 11 36 2412 100.00 2400 2211 0 0 0.00 12 18 2410 100.00 2400 2210 0 0 0.00 0.00 0.00 0.00 64 12 38 2410 100.00 2400 2209 0 0 0.00 0 1 2409 100.00 2400 2208 0 0 0.00 0.00 0.00 0.00 67 69 44.60 5.49 0.00 0.00 0 21 2409 100.00 2400 2208 0 0 0.00 1 3 2408 100.00 2400 2208 0 0 0.00 0.00 0.00 0.00 68 1 23 2408 100.00 2400 2207 0 0 0.00 2 5 2406 100.00 2400 2206 0 0 0.00 0.00 0.00 0.00 68 2 25 2406 100.00 2400 2205 0 0 0.00 3 7 2405 100.00 2400 2205 0 0 0.00 0.00 0.00 0.00 66 3 27 2405 100.00 2400 2204 0 0 0.00 4 9 2404 100.00 2400 2204 0 0 0.00 0.00 0.00 0.00 67 4 29 2404 100.00 2400 2204 0 0 0.00 8 11 2403 100.00 2400 2203 0 0 0.00 0.00 0.00 0.00 66 8 31 2403 100.00 2400 2203 0 0 0.00 9 13 2402 100.00 2400 2202 0 0 0.00 0.00 0.00 0.00 66 9 33 2402 100.00 2400 2202 0 0 0.00 10 15 2401 100.00 2400 2201 0 0 0.00 0.00 0.00 0.00 66 10 35 2401 100.00 2400 2201 0 0 0.00 11 17 2400 100.00 2400 2200 0 0 0.00 0.00 0.00 0.00 67 11 37 2399 100.00 2400 2199 0 0 0.00 12 19 2399 100.00 2400 2199 0 0 0.00 0.00 0.00 0.00 66 12 39 2397 100.00 2400 2198 0 0 0.00 [root@overcloud-compute-0 ~]# ~~~ ~~~ [root@overcloud-compute-0 ~]# cat /proc/cmdline BOOT_IMAGE=/boot/vmlinuz-3.10.0-693.17.1.el7.x86_64 root=UUID=fa414390-f78d-49d4-a164-54615a32977b ro console=tty0 console=ttyS0,115200n8 crashkernel=auto rhgb quiet default_hugepagesz=1GB hugepagesz=1G hugepages=32 iommu=pt intel_iommu=on isolcpus=2,4,6,8,10,12,14,16,18,22,24,26,28,30,32,34,36,38,3,5,7,9,11,13,15,17,19,23,25,27,29,31,33,35,37,39 processor.max_cstate=0 intel_idle.max_cstate=0 idle=poll nohz=on nohz_full=2,4,6,8,10,12,14,16,18,22,24,26,28,30,32,34,36,38,3,5,7,9,11,13,15,17,19,23,25,27,29,31,33,35,37,39 rcu_nocbs=2,4,6,8,10,12,14,16,18,22,24,26,28,30,32,34,36,38,3,5,7,9,11,13,15,17,19,23,25,27,29,31,33,35,37,39 tuned.non_isolcpus=00300003 intel_pstate=disable nosoftlockup [root@overcloud-compute-0 ~]# turbostat Core CPU Avg_MHz Busy% Bzy_MHz TSC_MHz IRQ SMI CPU%c1 CPU%c3 CPU%c6 CPU%c7 CoreTmp PkgTmp PkgWatt RAMWatt PKG_% RAM_% - - 2397 100.00 2400 2198 13460 0 0.00 0.00 0.00 0.00 66 69 92.05 9.46 0.00 0.00 0 0 2398 100.00 2400 2198 5147 0 0.00 0.00 0.00 0.00 63 69 48.47 3.90 0.00 0.00 0 20 2398 100.00 2400 2198 2987 0 0.00 1 2 2398 100.00 2400 2198 14 0 0.00 0.00 0.00 0.00 65 1 22 2398 100.00 2400 2198 14 0 0.00 2 4 2398 100.00 2400 2198 18 0 0.00 0.00 0.00 0.00 66 2 24 2398 100.00 2400 2198 40 0 0.00 3 6 2398 100.00 2400 2198 40 0 0.00 0.00 0.00 0.00 64 3 26 2398 100.00 2400 2198 40 0 0.00 4 8 2398 100.00 2400 2198 40 0 0.00 0.00 0.00 0.00 64 4 28 2398 100.00 2400 2198 40 0 0.00 8 10 2398 100.00 2400 2198 5 0 0.00 0.00 0.00 0.00 63 8 30 2398 100.00 2400 2198 5 0 0.00 9 12 2398 100.00 2400 2198 5 0 0.00 0.00 0.00 0.00 64 9 32 2398 100.00 2400 2198 5 0 0.00 10 14 2398 100.00 2400 2198 5 0 0.00 0.00 0.00 0.00 64 10 34 2398 100.00 2400 2198 5 0 0.00 11 16 2398 100.00 2400 2198 5 0 0.00 0.00 0.00 0.00 64 11 36 2397 100.00 2400 2198 5 0 0.00 12 18 2397 100.00 2400 2198 6 0 0.00 0.00 0.00 0.00 62 12 38 2397 100.00 2400 2198 5 0 0.00 0 1 2397 100.00 2400 2198 2262 0 0.00 0.00 0.00 0.00 63 66 43.58 5.56 0.00 0.00 0 21 2397 100.00 2400 2198 2660 0 0.00 1 3 2397 100.00 2400 2198 20 0 0.00 0.00 0.00 0.00 64 1 23 2397 100.00 2400 2198 5 0 0.00 2 5 2397 100.00 2400 2198 5 0 0.00 0.00 0.00 0.00 64 2 25 2397 100.00 2400 2198 5 0 0.00 3 7 2397 100.00 2400 2198 5 0 0.00 0.00 0.00 0.00 62 3 27 2397 100.00 2400 2198 5 0 0.00 4 9 2397 100.00 2400 2198 5 0 0.00 0.00 0.00 0.00 63 4 29 2397 100.00 2400 2198 5 0 0.00 8 11 2397 100.00 2400 2198 5 0 0.00 0.00 0.00 0.00 62 8 31 2397 100.00 2400 2198 5 0 0.00 9 13 2397 100.00 2400 2198 5 0 0.00 0.00 0.00 0.00 62 9 33 2397 100.00 2400 2198 5 0 0.00 10 15 2397 100.00 2400 2198 5 0 0.00 0.00 0.00 0.00 63 10 35 2397 100.00 2400 2197 5 0 0.00 11 17 2397 100.00 2400 2197 5 0 0.00 0.00 0.00 0.00 63 11 37 2397 100.00 2400 2197 6 0 0.00 12 19 2397 100.00 2400 2197 5 0 0.00 0.00 0.00 0.00 62 12 39 2397 100.00 2400 2197 6 0 0.00 Core CPU Avg_MHz Busy% Bzy_MHz TSC_MHz IRQ SMI CPU%c1 CPU%c3 CPU%c6 CPU%c7 CoreTmp PkgTmp PkgWatt RAMWatt PKG_% RAM_% - - 2397 100.00 2400 2197 13435 0 0.00 0.00 0.00 0.00 66 69 92.56 9.45 0.00 0.00 0 0 2397 100.00 2400 2197 5275 0 0.00 0.00 0.00 0.00 63 69 48.45 3.89 0.00 0.00 0 20 2397 100.00 2400 2197 2523 0 0.00 1 2 2397 100.00 2400 2197 14 0 0.00 0.00 0.00 0.00 65 1 22 2397 100.00 2400 2197 14 0 0.00 2 4 2397 100.00 2400 2197 14 0 0.00 0.00 0.00 0.00 66 2 24 2397 100.00 2400 2197 14 0 0.00 3 6 2397 100.00 2400 2197 14 0 0.00 0.00 0.00 0.00 65 3 26 2397 100.00 2400 2197 14 0 0.00 4 8 2397 100.00 2400 2197 14 0 0.00 0.00 0.00 0.00 64 4 28 2397 100.00 2400 2197 14 0 0.00 8 10 2397 100.00 2400 2197 2 0 0.00 0.00 0.00 0.00 64 8 30 2397 100.00 2400 2197 2 0 0.00 9 12 2397 100.00 2400 2197 2 0 0.00 0.00 0.00 0.00 64 9 32 2397 100.00 2400 2197 2 0 0.00 10 14 2397 100.00 2400 2197 2 0 0.00 0.00 0.00 0.00 64 10 34 2397 100.00 2400 2197 2 0 0.00 11 16 2397 100.00 2400 2197 2 0 0.00 0.00 0.00 0.00 64 11 36 2397 100.00 2400 2197 2 0 0.00 12 18 2397 100.00 2400 2197 2 0 0.00 0.00 0.00 0.00 62 12 38 2397 100.00 2400 2197 2 0 0.00 0 1 2397 100.00 2400 2197 2642 0 0.00 0.00 0.00 0.00 65 66 44.11 5.56 0.00 0.00 0 21 2397 100.00 2400 2197 2814 0 0.00 1 3 2397 100.00 2400 2197 10 0 0.00 0.00 0.00 0.00 65 1 23 2397 100.00 2400 2197 2 0 0.00 2 5 2397 100.00 2400 2197 2 0 0.00 0.00 0.00 0.00 65 2 25 2397 100.00 2400 2197 2 0 0.00 3 7 2397 100.00 2400 2197 3 0 0.00 0.00 0.00 0.00 63 3 27 2397 100.00 2400 2197 2 0 0.00 4 9 2397 100.00 2400 2197 2 0 0.00 0.00 0.00 0.00 64 4 29 2397 100.00 2400 2197 2 0 0.00 8 11 2397 100.00 2400 2197 2 0 0.00 0.00 0.00 0.00 62 8 31 2397 100.00 2400 2197 2 0 0.00 9 13 2397 100.00 2400 2197 2 0 0.00 0.00 0.00 0.00 63 9 33 2397 100.00 2400 2197 2 0 0.00 10 15 2397 100.00 2400 2197 2 0 0.00 0.00 0.00 0.00 63 10 35 2397 100.00 2400 2197 2 0 0.00 11 17 2397 100.00 2400 2197 2 0 0.00 0.00 0.00 0.00 64 11 37 2397 100.00 2400 2197 2 0 0.00 12 19 2397 100.00 2400 2197 2 0 0.00 0.00 0.00 0.00 63 12 39 2397 100.00 2400 2197 6 0 0.00 ^C [root@overcloud-compute-0 ~]# ~~~ The question hence is: * do we desire to lock only at C1, instead of heaving all CPUs remain at C0 constantly? Intel's recommendation: http://dpdk.org/doc/guides/linux_gsg/nic_perf_intel_platform.html#bios-settings Is Intel in the following recommending to disable C3 and C6 only, or all C-states? ~~~ Before starting consider resetting all BIOS settings to their default. Disable all power saving options such as: Power performance tuning, CPU P-State, CPU C3 Report and CPU C6 Report. Select Performance as the CPU Power and Performance policy. Disable Turbo Boost to ensure the performance scaling increases with the number of cores. Set memory frequency to the highest available number, NOT auto. Disable all virtualization options when you test the physical function of the NIC, and turn on VT-d if you wants to use VFIO. ~~~ If we allow C1, how can I make sure that I see no spurious TX drops in the following scenario: * instance with kernel virtio * OVS DPDK My understanding is that if the instance vCPUs go into C1, then they'll potentially cause packet drop on the vhu port. Hello, I followed https://jeremyeder.com/2012/11/14/processor-max_cstate-intel_idle-max_cstate-and-devcpu_dma_latency/ which leads to https://github.com/NetSys/NetBricks/blob/master/scripts/tuning/pmqos-static.py This script is part of tuned-utils! On compute-1, I executed: ~~~ yum install tuned-utils -y /usr/libexec/tuned/pmqos-static.py cpu_dma_latency=0 ~~~ And the CPU C-States now do look good: ~~~ [root@compute-1 ~]# /usr/libexec/tuned/pmqos-static.py cpu_dma_latency=0 [root@compute-1 ~]# turbostat Core CPU Avg_MHz Busy% Bzy_MHz TSC_MHz IRQ SMI CPU%c1 CPU%c3 CPU%c6 CPU%c7 CoreTmp PkgTmp PkgWatt RAMWatt PKG_% RAM_% - - 2198 100.00 2200 2198 32048 3608 0.00 0.00 0.00 0.00 63 65 178.67 31.67 0.00 0.00 0 0 2198 100.00 2200 2198 8894 41 0.00 0.00 0.00 0.00 58 62 88.24 17.62 0.00 0.00 0 44 2198 100.00 2200 2198 3293 41 0.00 1 1 2198 100.00 2200 2198 6261 41 0.00 0.00 0.00 0.00 60 1 45 2198 100.00 2200 2198 730 41 0.00 2 2 2198 100.00 2200 2198 5385 41 0.00 0.00 0.00 0.00 59 2 46 2198 100.00 2200 2198 627 41 0.00 3 3 2198 100.00 2200 2198 4142 41 0.00 0.00 0.00 0.00 59 3 47 2198 100.00 2200 2198 2253 41 0.00 4 4 2198 100.00 2200 2198 14 41 0.00 0.00 0.00 0.00 60 4 48 2198 100.00 2200 2198 14 41 0.00 5 5 2198 100.00 2200 2198 14 41 0.00 0.00 0.00 0.00 60 5 49 2198 100.00 2200 2198 15 41 0.00 8 6 2198 100.00 2200 2198 5 41 0.00 0.00 0.00 0.00 60 8 50 2198 100.00 2200 2198 4 41 0.00 9 7 2198 100.00 2200 2198 4 41 0.00 0.00 0.00 0.00 59 9 51 2198 100.00 2200 2198 4 41 0.00 10 8 2198 100.00 2200 2198 4 41 0.00 0.00 0.00 0.00 59 10 52 2198 100.00 2200 2198 4 41 0.00 11 9 2198 100.00 2200 2198 4 41 0.00 0.00 0.00 0.00 60 11 53 2198 100.00 2200 2198 4 41 0.00 12 10 2198 100.00 2200 2198 4 41 0.00 0.00 0.00 0.00 59 ~~~ If I follow Franck's recommendation, then we should not reach C1 for the cpu-partitioning profile. Or we should have a switch there that allows us to set /usr/libexec/tuned/pmqos-static.py cpu_dma_latency=0 Also note that the website which you referred to earlier explains why processor.max_cstate=1 and processor.max_cstate=0 are the same: ~~~ 1) When you set processor.max_cstate=0, the kernel actually silently sets it to 1. drivers/acpi/processor_idle.c:1086: 1086 if (max_cstate == 0) 1087 max_cstate = 1; ~~~ - Andreas I just checked, and none of the tuned profiles actually force C0. --- [root@overcloud-compute-0 ~]# for i in balanced cpu-partitioning desktop latency-performance network-latency network-throughput powersave throughput-performance virtual-guest virtual-host ; do echo "" ; echo "+++ $i +++" ; tuned-adm profile $i ; turbostat sleep 5 ; done +++ balanced +++ 5.002253 sec Core CPU Avg_MHz Busy% Bzy_MHz TSC_MHz IRQ SMI CPU%c1 CPU%c3 CPU%c6 CPU%c7 CoreTmp PkgTmp PkgWatt RAMWatt PKG_% RAM_% - - 538 19.46 2771 2198 0 0 2.47 0.03 78.03 0.00 64 66 54.65 9.44 0.00 0.00 0 0 32 1.19 2702 2198 0 0 27.38 0.02 71.40 0.00 59 66 36.83 3.85 0.00 0.00 0 20 6 0.21 2705 2198 0 0 28.37 1 2 2684 96.94 2772 2198 0 0 3.06 0.00 0.00 0.00 64 1 22 2685 96.95 2772 2198 0 0 3.05 2 4 2685 96.97 2772 2198 0 0 3.03 0.00 0.00 0.00 64 2 24 2686 96.98 2772 2198 0 0 3.02 3 6 2685 96.98 2772 2198 0 0 3.02 0.00 0.00 0.00 64 3 26 2686 96.99 2772 2198 0 0 3.01 4 8 2685 96.98 2772 2198 0 0 3.02 0.00 0.00 0.00 63 4 28 2686 96.99 2772 2198 0 0 3.01 8 10 0 0.00 2628 2198 0 0 0.01 0.00 99.98 0.00 57 8 30 0 0.00 2588 2198 0 0 0.01 9 12 0 0.00 2545 2198 0 0 0.01 0.00 99.99 0.00 56 9 32 0 0.00 2514 2198 0 0 0.01 10 14 0 0.00 2458 2198 0 0 0.01 0.00 99.99 0.00 55 10 34 0 0.00 2442 2198 0 0 0.01 11 16 0 0.00 2437 2198 0 0 0.01 0.00 99.99 0.00 56 11 36 0 0.00 2444 2198 0 0 0.01 12 18 0 0.00 2424 2198 0 0 0.01 0.01 99.97 0.00 55 12 38 0 0.00 2479 2198 0 0 0.01 0 1 13 0.70 1859 2198 0 0 9.18 0.62 89.50 0.00 52 59 17.82 5.59 0.00 0.00 0 21 7 0.42 1629 2198 0 0 9.45 1 3 0 0.00 1670 2198 0 0 0.01 0.00 99.99 0.00 52 1 23 0 0.00 2171 2198 0 0 0.01 2 5 0 0.00 1702 2197 0 0 0.01 0.00 99.99 0.00 51 2 25 0 0.00 2743 2197 0 0 0.01 3 7 0 0.00 2808 2197 0 0 0.01 0.01 99.98 0.00 51 3 27 0 0.00 2724 2197 0 0 0.01 4 9 0 0.00 2769 2197 0 0 0.01 0.00 99.98 0.00 52 4 29 0 0.00 2724 2197 0 0 0.01 8 11 0 0.00 2714 2197 0 0 0.01 0.00 99.99 0.00 52 8 31 0 0.00 2729 2197 0 0 0.01 9 13 0 0.00 2743 2197 0 0 0.01 0.00 99.99 0.00 51 9 33 0 0.00 2738 2197 0 0 0.01 10 15 0 0.00 2738 2197 0 0 0.01 0.00 99.99 0.00 52 10 35 0 0.00 2693 2197 0 0 0.01 11 17 0 0.00 2698 2197 0 0 0.01 0.01 99.97 0.00 51 11 37 0 0.00 2636 2197 0 0 0.01 12 19 0 0.00 2350 2197 0 0 0.03 0.00 99.96 0.00 52 12 39 0 0.01 2856 2197 0 0 0.02 +++ cpu-partitioning +++ 5.001604 sec Core CPU Avg_MHz Busy% Bzy_MHz TSC_MHz IRQ SMI CPU%c1 CPU%c3 CPU%c6 CPU%c7 CoreTmp PkgTmp PkgWatt RAMWatt PKG_% RAM_% - - 242 10.03 2400 2207 0 0 89.97 0.00 0.00 0.00 59 66 64.85 9.90 0.00 0.00 0 0 2422 100.00 2400 2220 0 0 0.00 0.00 0.00 0.00 59 66 35.24 4.31 0.00 0.00 0 20 2419 100.00 2400 2217 0 0 0.00 1 2 26 1.06 2400 2217 0 0 98.94 0.00 0.00 0.00 58 1 22 0 0.00 2399 2217 0 0 100.00 2 4 0 0.00 2399 2217 0 0 100.00 0.00 0.00 0.00 58 2 24 0 0.00 2404 2217 0 0 100.00 3 6 0 0.00 2401 2217 0 0 100.00 0.00 0.00 0.00 59 3 26 0 0.00 2400 2217 0 0 100.00 4 8 0 0.00 2400 2217 0 0 100.00 0.00 0.00 0.00 58 4 28 0 0.00 2400 2217 0 0 100.00 8 10 0 0.00 2401 2217 0 0 100.00 0.00 0.00 0.00 58 8 30 0 0.00 2401 2217 0 0 100.00 9 12 0 0.00 2401 2217 0 0 100.00 0.00 0.00 0.00 57 9 32 0 0.00 2401 2217 0 0 100.00 10 14 0 0.00 2401 2217 0 0 100.00 0.00 0.00 0.00 57 10 34 0 0.00 2402 2217 0 0 100.00 11 16 0 0.00 2401 2217 0 0 100.00 0.00 0.00 0.00 57 11 36 0 0.00 2401 2217 0 0 100.00 12 18 0 0.00 2402 2217 0 0 100.00 0.00 0.00 0.00 57 12 38 0 0.00 2401 2217 0 0 100.00 0 1 2398 100.00 2400 2198 0 0 0.00 0.00 0.00 0.00 58 60 29.61 5.59 0.00 0.00 0 21 2397 100.00 2400 2198 0 0 0.00 1 3 0 0.00 2402 2198 0 0 100.00 0.00 0.00 0.00 56 1 23 0 0.00 2402 2198 0 0 100.00 2 5 0 0.00 2401 2198 0 0 100.00 0.00 0.00 0.00 54 2 25 0 0.00 2401 2198 0 0 100.00 3 7 0 0.00 2401 2198 0 0 100.00 0.00 0.00 0.00 54 3 27 0 0.00 2401 2198 0 0 100.00 4 9 0 0.00 2401 2198 0 0 100.00 0.00 0.00 0.00 54 4 29 0 0.00 2401 2197 0 0 100.00 8 11 0 0.00 2401 2197 0 0 100.00 0.00 0.00 0.00 54 8 31 0 0.00 2403 2197 0 0 100.00 9 13 0 0.00 2402 2197 0 0 100.00 0.00 0.00 0.00 53 9 33 0 0.00 2401 2197 0 0 100.00 10 15 0 0.00 2401 2197 0 0 100.00 0.00 0.00 0.00 54 10 35 0 0.00 2401 2197 0 0 100.00 11 17 0 0.00 2401 2197 0 0 100.00 0.00 0.00 0.00 54 11 37 0 0.00 2401 2197 0 0 100.00 12 19 0 0.00 2402 2197 0 0 100.00 0.00 0.00 0.00 54 12 39 0 0.02 2400 2197 0 0 99.98 +++ desktop +++ 5.002544 sec Core CPU Avg_MHz Busy% Bzy_MHz TSC_MHz IRQ SMI CPU%c1 CPU%c3 CPU%c6 CPU%c7 CoreTmp PkgTmp PkgWatt RAMWatt PKG_% RAM_% - - 311 10.04 3100 2198 0 0 1.38 0.00 88.58 0.00 62 65 50.74 9.72 0.00 0.00 0 0 3100 99.99 3100 2200 0 0 0.01 0.00 0.00 0.00 62 65 28.28 3.98 0.00 0.00 0 20 3098 99.99 3100 2199 0 0 0.01 1 2 41 1.32 3101 2199 0 0 26.73 0.01 71.94 0.00 56 1 22 5 0.15 3106 2199 0 0 27.89 2 4 0 0.00 2809 2199 0 0 0.01 0.01 99.98 0.00 55 2 24 0 0.00 2758 2199 0 0 0.01 3 6 0 0.00 2739 2199 0 0 0.01 0.01 99.98 0.00 56 3 26 0 0.00 2685 2199 0 0 0.01 4 8 0 0.00 2655 2199 0 0 0.01 0.01 99.98 0.00 56 4 28 0 0.00 2635 2199 0 0 0.01 8 10 0 0.00 2636 2199 0 0 0.01 0.00 99.99 0.00 56 8 30 0 0.00 2603 2199 0 0 0.01 9 12 0 0.00 2660 2199 0 0 0.01 0.00 99.98 0.00 55 9 32 0 0.00 2592 2199 0 0 0.01 10 14 0 0.00 2630 2199 0 0 0.01 0.00 99.99 0.00 54 10 34 0 0.00 2663 2199 0 0 0.01 11 16 0 0.00 2764 2199 0 0 0.01 0.00 99.99 0.00 54 11 36 0 0.00 2799 2199 0 0 0.01 12 18 0 0.00 2774 2199 0 0 0.01 0.00 99.99 0.00 54 12 38 0 0.00 2795 2199 0 0 0.01 0 1 3097 99.99 3100 2198 0 0 0.01 0.00 0.00 0.00 59 59 22.46 5.74 0.00 0.00 0 21 3096 99.99 3100 2197 0 0 0.01 1 3 0 0.00 2990 2197 0 0 0.01 0.00 99.99 0.00 53 1 23 0 0.00 2855 2197 0 0 0.01 2 5 0 0.00 2797 2197 0 0 0.01 0.00 99.99 0.00 53 2 25 0 0.00 2733 2197 0 0 0.01 3 7 0 0.00 2717 2197 0 0 0.01 0.00 99.99 0.00 52 3 27 0 0.00 2666 2197 0 0 0.01 4 9 0 0.00 2654 2197 0 0 0.01 0.00 99.99 0.00 52 4 29 0 0.00 2638 2197 0 0 0.01 8 11 0 0.00 2664 2197 0 0 0.01 0.00 99.99 0.00 53 8 31 0 0.00 2648 2197 0 0 0.01 9 13 0 0.00 2565 2197 0 0 0.01 0.00 99.98 0.00 51 9 33 0 0.00 2597 2197 0 0 0.01 10 15 0 0.00 2620 2197 0 0 0.01 0.00 99.99 0.00 53 10 35 0 0.00 2564 2197 0 0 0.01 11 17 0 0.00 2679 2197 0 0 0.01 0.00 99.99 0.00 51 11 37 0 0.00 2794 2197 0 0 0.01 12 19 0 0.00 2834 2197 0 0 0.05 0.00 99.95 0.00 53 12 39 0 0.01 2803 2197 0 0 0.04 +++ latency-performance +++ 5.001753 sec Core CPU Avg_MHz Busy% Bzy_MHz TSC_MHz IRQ SMI CPU%c1 CPU%c3 CPU%c6 CPU%c7 CoreTmp PkgTmp PkgWatt RAMWatt PKG_% RAM_% - - 242 10.04 2400 2211 0 0 89.96 0.00 0.00 0.00 59 65 64.41 9.89 0.00 0.00 0 0 2434 100.00 2400 2232 0 0 0.00 0.00 0.00 0.00 59 65 34.95 4.30 0.00 0.00 0 20 2425 100.00 2400 2223 0 0 0.00 1 2 26 1.05 2400 2223 0 0 98.95 0.00 0.00 0.00 57 1 22 0 0.00 2400 2223 0 0 100.00 2 4 0 0.00 2401 2223 0 0 100.00 0.00 0.00 0.00 57 2 24 0 0.00 2400 2223 0 0 100.00 3 6 0 0.00 2402 2223 0 0 100.00 0.00 0.00 0.00 58 3 26 0 0.00 2402 2223 0 0 100.00 4 8 0 0.00 2401 2223 0 0 100.00 0.00 0.00 0.00 58 4 28 0 0.00 2402 2223 0 0 100.00 8 10 0 0.00 2402 2223 0 0 100.00 0.00 0.00 0.00 57 8 30 0 0.00 2403 2223 0 0 100.00 9 12 0 0.00 2398 2223 0 0 100.00 0.00 0.00 0.00 57 9 32 0 0.00 2401 2223 0 0 100.00 10 14 0 0.00 2402 2223 0 0 100.00 0.00 0.00 0.00 56 10 34 0 0.00 2402 2223 0 0 100.00 11 16 0 0.00 2401 2223 0 0 100.00 0.00 0.00 0.00 56 11 36 0 0.00 2402 2223 0 0 100.00 12 18 0 0.00 2401 2223 0 0 100.00 0.00 0.00 0.00 56 12 38 0 0.00 2401 2223 0 0 100.00 0 1 2406 100.00 2400 2205 0 0 0.00 0.00 0.00 0.00 57 60 29.45 5.59 0.00 0.00 0 21 2397 100.00 2400 2197 0 0 0.00 1 3 0 0.00 2402 2197 0 0 100.00 0.00 0.00 0.00 55 1 23 0 0.00 2401 2197 0 0 100.00 2 5 0 0.00 2401 2197 0 0 100.00 0.00 0.00 0.00 54 2 25 0 0.00 2402 2197 0 0 100.00 3 7 0 0.00 2401 2197 0 0 100.00 0.00 0.00 0.00 54 3 27 0 0.00 2401 2197 0 0 100.00 4 9 0 0.00 2400 2197 0 0 100.00 0.00 0.00 0.00 54 4 29 0 0.00 2401 2197 0 0 100.00 8 11 0 0.00 2401 2197 0 0 100.00 0.00 0.00 0.00 54 8 31 0 0.00 2403 2197 0 0 100.00 9 13 0 0.00 2401 2197 0 0 100.00 0.00 0.00 0.00 53 9 33 0 0.00 2402 2197 0 0 100.00 10 15 0 0.00 2402 2197 0 0 100.00 0.00 0.00 0.00 55 10 35 0 0.00 2401 2197 0 0 100.00 11 17 0 0.00 2401 2197 0 0 100.00 0.00 0.00 0.00 54 11 37 0 0.00 2401 2197 0 0 100.00 12 19 0 0.00 2400 2197 0 0 100.00 0.00 0.00 0.00 55 12 39 0 0.02 2400 2197 0 0 99.98 +++ network-latency +++ 5.001831 sec Core CPU Avg_MHz Busy% Bzy_MHz TSC_MHz IRQ SMI CPU%c1 CPU%c3 CPU%c6 CPU%c7 CoreTmp PkgTmp PkgWatt RAMWatt PKG_% RAM_% - - 242 10.04 2400 2208 0 0 89.96 0.00 0.00 0.00 59 65 64.71 9.80 0.00 0.00 0 0 2427 100.00 2400 2224 0 0 0.00 0.00 0.00 0.00 59 65 35.21 4.23 0.00 0.00 0 20 2420 100.00 2400 2219 0 0 0.00 1 2 26 1.07 2400 2219 0 0 98.93 0.00 0.00 0.00 57 1 22 0 0.00 2404 2219 0 0 100.00 2 4 0 0.00 2401 2219 0 0 100.00 0.00 0.00 0.00 57 2 24 0 0.00 2401 2219 0 0 100.00 3 6 0 0.00 2401 2219 0 0 100.00 0.00 0.00 0.00 58 3 26 0 0.00 2401 2219 0 0 100.00 4 8 0 0.00 2401 2219 0 0 100.00 0.00 0.00 0.00 58 4 28 0 0.00 2402 2219 0 0 100.00 8 10 0 0.00 2402 2219 0 0 100.00 0.00 0.00 0.00 57 8 30 0 0.00 2401 2219 0 0 100.00 9 12 0 0.00 2402 2219 0 0 100.00 0.00 0.00 0.00 56 9 32 0 0.00 2402 2219 0 0 100.00 10 14 0 0.00 2402 2219 0 0 100.00 0.00 0.00 0.00 57 10 34 0 0.00 2402 2219 0 0 100.00 11 16 0 0.00 2401 2219 0 0 100.00 0.00 0.00 0.00 56 11 36 0 0.00 2398 2219 0 0 100.00 12 18 0 0.00 2401 2219 0 0 100.00 0.00 0.00 0.00 56 12 38 0 0.00 2401 2219 0 0 100.00 0 1 2399 100.00 2400 2199 0 0 0.00 0.00 0.00 0.00 58 61 29.50 5.57 0.00 0.00 0 21 2397 100.00 2400 2197 0 0 0.00 1 3 0 0.00 2402 2197 0 0 100.00 0.00 0.00 0.00 56 1 23 0 0.00 2401 2197 0 0 100.00 2 5 0 0.00 2401 2197 0 0 100.00 0.00 0.00 0.00 55 2 25 0 0.00 2401 2197 0 0 100.00 3 7 0 0.00 2401 2197 0 0 100.00 0.00 0.00 0.00 55 3 27 0 0.00 2403 2197 0 0 100.00 4 9 0 0.00 2401 2197 0 0 100.00 0.00 0.00 0.00 55 4 29 0 0.00 2400 2197 0 0 100.00 8 11 0 0.00 2402 2197 0 0 100.00 0.00 0.00 0.00 55 8 31 0 0.00 2402 2197 0 0 100.00 9 13 0 0.00 2401 2197 0 0 100.00 0.00 0.00 0.00 54 9 33 0 0.00 2402 2197 0 0 100.00 10 15 0 0.00 2401 2197 0 0 100.00 0.00 0.00 0.00 55 10 35 0 0.00 2400 2197 0 0 100.00 11 17 0 0.00 2401 2197 0 0 100.00 0.00 0.00 0.00 54 11 37 0 0.00 2401 2197 0 0 100.00 12 19 0 0.00 2401 2197 0 0 100.00 0.00 0.00 0.00 55 12 39 0 0.02 2400 2197 0 0 99.98 +++ network-throughput +++ 5.040841 sec Core CPU Avg_MHz Busy% Bzy_MHz TSC_MHz IRQ SMI CPU%c1 CPU%c3 CPU%c6 CPU%c7 CoreTmp PkgTmp PkgWatt RAMWatt PKG_% RAM_% - - 311 10.04 3100 2197 0 0 1.38 0.00 88.58 0.00 62 64 50.76 9.86 0.00 0.00 0 0 3097 99.99 3100 2198 0 0 0.01 0.00 0.00 0.00 62 64 28.26 4.29 0.00 0.00 0 20 3094 99.99 3100 2196 0 0 0.01 1 2 41 1.32 3101 2196 0 0 26.71 0.00 71.97 0.00 56 1 22 5 0.15 3105 2196 0 0 27.87 2 4 0 0.00 2794 2196 0 0 0.01 0.00 99.99 0.00 55 2 24 0 0.00 2732 2196 0 0 0.01 3 6 0 0.00 2719 2196 0 0 0.01 0.00 99.99 0.00 55 3 26 0 0.00 2665 2196 0 0 0.01 4 8 0 0.00 2641 2196 0 0 0.01 0.00 99.99 0.00 56 4 28 0 0.00 2607 2196 0 0 0.01 8 10 0 0.00 2617 2196 0 0 0.01 0.00 99.99 0.00 56 8 30 0 0.00 2633 2196 0 0 0.01 9 12 0 0.00 2673 2196 0 0 0.01 0.00 99.99 0.00 54 9 32 0 0.00 2658 2196 0 0 0.01 10 14 0 0.00 2714 2196 0 0 0.01 0.00 99.99 0.00 54 10 34 0 0.00 2687 2196 0 0 0.01 11 16 0 0.00 2758 2196 0 0 0.01 0.00 99.99 0.00 54 11 36 0 0.00 2804 2196 0 0 0.01 12 18 0 0.00 2752 2196 0 0 0.01 0.00 99.99 0.00 54 12 38 0 0.00 2752 2196 0 0 0.01 0 1 3104 99.99 3100 2203 0 0 0.01 0.00 0.00 0.00 60 60 22.50 5.57 0.00 0.00 0 21 3096 99.99 3100 2198 0 0 0.01 1 3 0 0.00 2947 2198 0 0 0.01 0.00 99.99 0.00 54 1 23 0 0.00 2843 2197 0 0 0.01 2 5 0 0.00 2808 2197 0 0 0.01 0.00 99.99 0.00 53 2 25 0 0.00 2723 2197 0 0 0.01 3 7 0 0.00 2711 2197 0 0 0.01 0.00 99.99 0.00 52 3 27 0 0.00 2675 2197 0 0 0.01 4 9 0 0.00 2650 2197 0 0 0.01 0.00 99.99 0.00 53 4 29 0 0.00 2646 2197 0 0 0.01 8 11 0 0.00 2629 2197 0 0 0.01 0.00 99.99 0.00 53 8 31 0 0.00 2623 2197 0 0 0.01 9 13 0 0.00 2702 2197 0 0 0.01 0.00 99.98 0.00 52 9 33 0 0.00 2662 2197 0 0 0.01 10 15 0 0.00 2714 2197 0 0 0.01 0.00 99.99 0.00 53 10 35 0 0.00 2689 2197 0 0 0.01 11 17 0 0.00 2773 2197 0 0 0.01 0.00 99.99 0.00 53 11 37 0 0.00 2772 2197 0 0 0.01 12 19 0 0.00 2833 2197 0 0 0.05 0.01 99.95 0.00 53 12 39 0 0.01 2795 2197 0 0 0.03 +++ powersave +++ 5.003208 sec Core CPU Avg_MHz Busy% Bzy_MHz TSC_MHz IRQ SMI CPU%c1 CPU%c3 CPU%c6 CPU%c7 CoreTmp PkgTmp PkgWatt RAMWatt PKG_% RAM_% - - 311 10.04 3100 2199 0 0 1.38 0.00 88.58 0.00 62 64 50.83 9.88 0.00 0.00 0 0 3106 99.99 3100 2205 0 0 0.01 0.00 0.00 0.00 62 64 28.35 4.31 0.00 0.00 0 20 3098 99.99 3100 2199 0 0 0.01 1 2 41 1.32 3101 2199 0 0 26.70 0.00 71.98 0.00 55 1 22 5 0.15 3105 2199 0 0 27.87 2 4 0 0.00 2783 2199 0 0 0.01 0.00 99.99 0.00 55 2 24 0 0.01 2984 2199 0 0 0.01 3 6 0 0.00 2743 2199 0 0 0.01 0.00 99.99 0.00 55 3 26 0 0.00 2687 2199 0 0 0.01 4 8 0 0.00 2669 2199 0 0 0.01 0.00 99.99 0.00 55 4 28 0 0.00 2634 2199 0 0 0.01 8 10 0 0.00 2642 2199 0 0 0.01 0.00 99.99 0.00 55 8 30 0 0.00 2641 2199 0 0 0.01 9 12 0 0.00 2668 2199 0 0 0.01 0.00 99.99 0.00 54 9 32 0 0.00 2611 2199 0 0 0.01 10 14 0 0.00 2666 2199 0 0 0.01 0.00 99.99 0.00 54 10 34 0 0.00 2638 2199 0 0 0.01 11 16 0 0.00 2725 2199 0 0 0.01 0.00 99.99 0.00 54 11 36 0 0.00 2829 2199 0 0 0.02 12 18 0 0.00 2795 2199 0 0 0.01 0.00 99.99 0.00 54 12 38 0 0.00 2794 2199 0 0 0.01 0 1 3097 99.99 3100 2198 0 0 0.01 0.00 0.00 0.00 60 60 22.48 5.57 0.00 0.00 0 21 3097 99.99 3100 2198 0 0 0.01 1 3 0 0.00 3057 2198 0 0 0.01 0.00 99.99 0.00 54 1 23 0 0.00 3014 2198 0 0 0.01 2 5 0 0.00 2883 2198 0 0 0.01 0.00 99.99 0.00 53 2 25 0 0.00 2872 2198 0 0 0.01 3 7 0 0.01 2884 2198 0 0 0.01 0.01 99.97 0.00 53 3 27 0 0.00 2818 2198 0 0 0.01 4 9 0 0.00 2766 2198 0 0 0.01 0.00 99.99 0.00 53 4 29 0 0.00 2691 2198 0 0 0.01 8 11 0 0.00 2693 2198 0 0 0.01 0.00 99.98 0.00 54 8 31 0 0.00 2637 2198 0 0 0.01 9 13 0 0.00 2590 2198 0 0 0.01 0.00 99.99 0.00 52 9 33 0 0.00 2598 2198 0 0 0.01 10 15 0 0.00 2617 2198 0 0 0.01 0.00 99.99 0.00 53 10 35 0 0.00 2640 2198 0 0 0.01 11 17 0 0.00 2679 2198 0 0 0.01 0.00 99.99 0.00 52 11 37 0 0.00 2644 2197 0 0 0.01 12 19 0 0.00 2739 2197 0 0 0.05 0.00 99.95 0.00 53 12 39 0 0.01 2942 2197 0 0 0.03 +++ throughput-performance +++ 5.054865 sec Core CPU Avg_MHz Busy% Bzy_MHz TSC_MHz IRQ SMI CPU%c1 CPU%c3 CPU%c6 CPU%c7 CoreTmp PkgTmp PkgWatt RAMWatt PKG_% RAM_% - - 311 10.04 3100 2201 0 0 1.38 0.00 88.58 0.00 62 64 50.89 9.85 0.00 0.00 0 0 3109 99.99 3100 2207 0 0 0.01 0.00 0.00 0.00 62 64 28.38 4.30 0.00 0.00 0 20 3105 99.99 3100 2204 0 0 0.01 1 2 41 1.32 3101 2204 0 0 26.73 0.00 71.95 0.00 56 1 22 5 0.15 3105 2204 0 0 27.90 2 4 0 0.00 2783 2204 0 0 0.01 0.00 99.99 0.00 55 2 24 0 0.00 2707 2204 0 0 0.01 3 6 0 0.00 2710 2204 0 0 0.01 0.00 99.99 0.00 55 3 26 0 0.00 2658 2204 0 0 0.01 4 8 0 0.00 2642 2204 0 0 0.01 0.00 99.99 0.00 55 4 28 0 0.00 2608 2204 0 0 0.01 8 10 0 0.00 2608 2204 0 0 0.01 0.00 99.99 0.00 55 8 30 0 0.00 2644 2204 0 0 0.01 9 12 0 0.00 2670 2204 0 0 0.01 0.00 99.99 0.00 55 9 32 0 0.00 2640 2204 0 0 0.01 10 14 0 0.00 2724 2204 0 0 0.01 0.00 99.98 0.00 54 10 34 0 0.00 2696 2204 0 0 0.01 11 16 0 0.00 2760 2204 0 0 0.01 0.00 99.99 0.00 54 11 36 0 0.00 2801 2204 0 0 0.01 12 18 0 0.00 2772 2204 0 0 0.01 0.00 99.99 0.00 54 12 38 0 0.00 2765 2204 0 0 0.01 0 1 3097 99.99 3100 2198 0 0 0.01 0.00 0.00 0.00 60 60 22.51 5.56 0.00 0.00 0 21 3096 99.99 3100 2197 0 0 0.01 1 3 0 0.00 2951 2197 0 0 0.01 0.00 99.99 0.00 55 1 23 0 0.00 2843 2197 0 0 0.01 2 5 0 0.00 2811 2197 0 0 0.01 0.00 99.99 0.00 53 2 25 0 0.00 2722 2197 0 0 0.01 3 7 0 0.00 2724 2197 0 0 0.01 0.00 99.99 0.00 53 3 27 0 0.00 2671 2197 0 0 0.01 4 9 0 0.00 2646 2197 0 0 0.01 0.00 99.99 0.00 54 4 29 0 0.00 2645 2197 0 0 0.01 8 11 0 0.00 2624 2197 0 0 0.01 0.00 99.99 0.00 54 8 31 0 0.00 2641 2197 0 0 0.01 9 13 0 0.00 2679 2197 0 0 0.01 0.00 99.98 0.00 52 9 33 0 0.00 2664 2197 0 0 0.01 10 15 0 0.00 2754 2197 0 0 0.01 0.00 99.99 0.00 53 10 35 0 0.00 2776 2197 0 0 0.01 11 17 0 0.00 2761 2197 0 0 0.01 0.00 99.99 0.00 52 11 37 0 0.00 2781 2197 0 0 0.01 12 19 0 0.00 2813 2197 0 0 0.05 0.00 99.95 0.00 53 12 39 0 0.02 2842 2197 0 0 0.03 +++ virtual-guest +++ 5.055904 sec Core CPU Avg_MHz Busy% Bzy_MHz TSC_MHz IRQ SMI CPU%c1 CPU%c3 CPU%c6 CPU%c7 CoreTmp PkgTmp PkgWatt RAMWatt PKG_% RAM_% - - 310 10.04 3100 2194 0 0 1.39 0.00 88.58 0.00 62 64 50.66 9.79 0.00 0.00 0 0 3088 99.99 3100 2192 0 0 0.01 0.00 0.00 0.00 62 64 28.20 4.24 0.00 0.00 0 20 3086 99.99 3100 2190 0 0 0.01 1 2 41 1.33 3101 2190 0 0 26.95 0.00 71.72 0.00 56 1 22 5 0.15 3105 2190 0 0 28.13 2 4 0 0.00 2793 2190 0 0 0.01 0.00 99.99 0.00 55 2 24 0 0.00 2719 2190 0 0 0.01 3 6 0 0.00 2723 2190 0 0 0.01 0.00 99.99 0.00 55 3 26 0 0.00 2658 2190 0 0 0.01 4 8 0 0.00 2643 2190 0 0 0.01 0.00 99.99 0.00 56 4 28 0 0.00 2606 2190 0 0 0.01 8 10 0 0.00 2613 2190 0 0 0.01 0.00 99.99 0.00 55 8 30 0 0.00 2638 2190 0 0 0.01 9 12 0 0.00 2655 2190 0 0 0.01 0.00 99.98 0.00 54 9 32 0 0.00 2660 2190 0 0 0.01 10 14 0 0.00 2708 2190 0 0 0.01 0.00 99.99 0.00 54 10 34 0 0.00 2682 2190 0 0 0.01 11 16 0 0.00 2744 2190 0 0 0.01 0.00 99.99 0.00 54 11 36 0 0.00 2775 2190 0 0 0.01 12 18 0 0.00 2753 2190 0 0 0.01 0.00 99.99 0.00 54 12 38 0 0.00 2766 2190 0 0 0.01 0 1 3093 99.99 3100 2195 0 0 0.01 0.00 0.00 0.00 60 61 22.45 5.54 0.00 0.00 0 21 3096 99.99 3100 2197 0 0 0.01 1 3 0 0.00 2951 2197 0 0 0.01 0.00 99.99 0.00 54 1 23 0 0.00 2849 2197 0 0 0.01 2 5 0 0.00 2800 2197 0 0 0.01 0.00 99.99 0.00 53 2 25 0 0.00 2730 2197 0 0 0.01 3 7 0 0.00 2708 2197 0 0 0.01 0.00 99.99 0.00 53 3 27 0 0.00 2678 2197 0 0 0.01 4 9 0 0.00 2649 2197 0 0 0.01 0.00 99.99 0.00 54 4 29 0 0.00 2611 2197 0 0 0.01 8 11 0 0.00 2633 2197 0 0 0.01 0.00 99.99 0.00 54 8 31 0 0.00 2642 2197 0 0 0.01 9 13 0 0.00 2714 2197 0 0 0.01 0.00 99.98 0.00 52 9 33 0 0.00 2688 2197 0 0 0.01 10 15 0 0.00 2763 2197 0 0 0.01 0.00 99.99 0.00 53 10 35 0 0.00 2782 2197 0 0 0.01 11 17 0 0.00 2743 2197 0 0 0.01 0.00 99.99 0.00 52 11 37 0 0.00 2775 2197 0 0 0.01 12 19 0 0.00 2809 2197 0 0 0.05 0.00 99.95 0.00 53 12 39 0 0.01 2802 2197 0 0 0.04 +++ virtual-host +++ 5.038866 sec Core CPU Avg_MHz Busy% Bzy_MHz TSC_MHz IRQ SMI CPU%c1 CPU%c3 CPU%c6 CPU%c7 CoreTmp PkgTmp PkgWatt RAMWatt PKG_% RAM_% - - 311 10.04 3100 2198 0 0 1.38 0.00 88.58 0.00 62 64 50.83 9.81 0.00 0.00 0 0 3098 99.99 3100 2199 0 0 0.01 0.00 0.00 0.00 62 64 28.28 4.25 0.00 0.00 0 20 3097 99.99 3100 2198 0 0 0.01 1 2 41 1.32 3101 2198 0 0 26.69 0.00 71.99 0.00 55 1 22 5 0.15 3105 2198 0 0 27.86 2 4 0 0.00 2800 2198 0 0 0.01 0.00 99.99 0.00 55 2 24 0 0.00 2728 2198 0 0 0.01 3 6 0 0.00 2718 2198 0 0 0.01 0.00 99.99 0.00 55 3 26 0 0.00 2676 2198 0 0 0.01 4 8 0 0.00 2639 2198 0 0 0.01 0.00 99.99 0.00 55 4 28 0 0.00 2619 2198 0 0 0.01 8 10 0 0.00 2658 2198 0 0 0.01 0.00 99.99 0.00 55 8 30 0 0.00 2644 2198 0 0 0.01 9 12 0 0.00 2706 2198 0 0 0.01 0.00 99.98 0.00 54 9 32 0 0.00 2682 2198 0 0 0.01 10 14 0 0.00 2727 2198 0 0 0.01 0.00 99.99 0.00 54 10 34 0 0.00 2694 2198 0 0 0.01 11 16 0 0.00 2770 2198 0 0 0.01 0.00 99.99 0.00 54 11 36 0 0.00 2802 2198 0 0 0.01 12 18 0 0.00 2774 2198 0 0 0.01 0.00 99.99 0.00 53 12 38 0 0.00 2768 2198 0 0 0.01 0 1 3099 99.99 3100 2199 0 0 0.01 0.00 0.00 0.00 60 60 22.55 5.56 0.00 0.00 0 21 3096 99.99 3100 2197 0 0 0.01 1 3 0 0.00 2953 2197 0 0 0.01 0.00 99.99 0.00 55 1 23 0 0.00 2854 2197 0 0 0.01 2 5 0 0.00 2819 2197 0 0 0.01 0.00 99.99 0.00 54 2 25 0 0.00 2731 2197 0 0 0.01 3 7 0 0.00 2712 2197 0 0 0.01 0.00 99.99 0.00 53 3 27 0 0.00 2681 2197 0 0 0.01 4 9 0 0.00 2646 2197 0 0 0.01 0.00 99.99 0.00 54 4 29 0 0.00 2639 2197 0 0 0.01 8 11 0 0.00 2619 2197 0 0 0.01 0.00 99.99 0.00 54 8 31 0 0.00 2641 2197 0 0 0.01 9 13 0 0.00 2696 2197 0 0 0.01 0.00 99.98 0.00 52 9 33 0 0.00 2665 2197 0 0 0.01 10 15 0 0.00 2760 2197 0 0 0.01 0.00 99.99 0.00 54 10 35 0 0.00 2776 2197 0 0 0.01 11 17 0 0.00 2761 2197 0 0 0.01 0.00 99.99 0.00 53 11 37 0 0.00 2779 2197 0 0 0.01 12 19 0 0.00 2800 2197 0 0 0.05 0.00 99.95 0.00 53 12 39 0 0.02 2831 2197 0 0 0.03 [root@overcloud-compute-0 ~]# Finally, one can easily create ones own tuned profile with C0, the question though is if this is desired. There's probably a reason why latency-performance started using C1, and all other profiles are based upon latency performance. https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/performance_tuning_guide/main-analyzeperf-tuned ~~~ latency-performance A server profile for typical latency performance tuning. This profile disables dynamic tuning mechanisms and transparent hugepages. It uses the performance governer for p-states through cpuspeed, and sets the I/O scheduler to deadline. Additionally, in Red Hat Enterprise Linux 6.5 and later, the profile requests a cpu_dma_latency value of 1. In Red Hat Enterprise Linux 6.4 and earlier, cpu_dma_latency requested a value of 0. ~~~ Anyway, here's the way to go to use cpu-partitioning locked to C0: ~~~ [root@overcloud-compute-0 ~]# cat /etc/tuned/cpu-partitioning-disable-cstates/tuned.conf [main] include=cpu-partitioning [cpu] force_latency=0 ~~~ ~~~ [root@overcloud-compute-0 ~]# tuned-adm profile cpu-partitioning [root@overcloud-compute-0 ~]# turbostat sleep 5 5.001639 sec Core CPU Avg_MHz Busy% Bzy_MHz TSC_MHz IRQ SMI CPU%c1 CPU%c3 CPU%c6 CPU%c7 CoreTmp PkgTmp PkgWatt RAMWatt PKG_% RAM_% - - 242 10.04 2400 2209 0 0 89.96 0.00 0.00 0.00 61 65 64.87 9.67 0.00 0.00 0 0 2430 100.00 2400 2228 0 0 0.00 0.00 0.00 0.00 59 65 35.04 3.92 0.00 0.00 0 20 2422 100.00 2400 2220 0 0 0.00 1 2 26 1.06 2400 2220 0 0 98.94 0.00 0.00 0.00 57 1 22 0 0.00 2401 2220 0 0 100.00 2 4 0 0.00 2400 2220 0 0 100.00 0.00 0.00 0.00 57 2 24 0 0.00 2400 2220 0 0 100.00 3 6 0 0.00 2401 2220 0 0 100.00 0.00 0.00 0.00 58 3 26 0 0.00 2402 2220 0 0 100.00 4 8 0 0.00 2402 2220 0 0 100.00 0.00 0.00 0.00 57 4 28 0 0.00 2400 2220 0 0 100.00 8 10 0 0.00 2401 2220 0 0 100.00 0.00 0.00 0.00 57 8 30 0 0.00 2403 2220 0 0 100.00 9 12 0 0.00 2402 2220 0 0 100.00 0.00 0.00 0.00 57 9 32 0 0.00 2403 2220 0 0 100.00 10 14 0 0.00 2401 2220 0 0 100.00 0.00 0.00 0.00 56 10 34 0 0.00 2401 2220 0 0 100.00 11 16 0 0.00 2401 2220 0 0 100.00 0.00 0.00 0.00 56 11 36 0 0.00 2402 2220 0 0 100.00 12 18 0 0.00 2402 2220 0 0 100.00 0.00 0.00 0.00 56 12 38 0 0.00 2402 2220 0 0 100.00 0 1 2403 100.00 2400 2203 0 0 0.00 0.00 0.00 0.00 61 64 29.83 5.75 0.00 0.00 0 21 2397 100.00 2400 2198 0 0 0.00 1 3 0 0.00 2402 2198 0 0 100.00 0.00 0.00 0.00 59 1 23 0 0.00 2401 2198 0 0 100.00 2 5 0 0.00 2401 2198 0 0 100.00 0.00 0.00 0.00 59 2 25 0 0.00 2401 2198 0 0 100.00 3 7 0 0.00 2401 2198 0 0 100.00 0.00 0.00 0.00 58 3 27 0 0.00 2385 2198 0 0 100.00 4 9 0 0.00 2402 2197 0 0 100.00 0.00 0.00 0.00 58 4 29 0 0.00 2403 2197 0 0 100.00 8 11 0 0.00 2402 2197 0 0 100.00 0.00 0.00 0.00 58 8 31 0 0.00 2401 2197 0 0 100.00 9 13 0 0.00 2400 2197 0 0 100.00 0.00 0.00 0.00 57 9 33 0 0.00 2401 2197 0 0 100.00 10 15 0 0.00 2402 2197 0 0 100.00 0.00 0.00 0.00 59 10 35 0 0.00 2401 2197 0 0 100.00 11 17 0 0.00 2401 2197 0 0 100.00 0.00 0.00 0.00 57 11 37 0 0.00 2401 2197 0 0 100.00 12 19 0 0.00 2400 2197 0 0 100.00 0.00 0.00 0.00 58 12 39 0 0.02 2400 2197 0 0 99.98 [root@overcloud-compute-0 ~]# tuned-adm profile cpu-partitioning-disable-cstates [root@overcloud-compute-0 ~]# turbostat sleep 5 5.001900 sec Core CPU Avg_MHz Busy% Bzy_MHz TSC_MHz IRQ SMI CPU%c1 CPU%c3 CPU%c6 CPU%c7 CoreTmp PkgTmp PkgWatt RAMWatt PKG_% RAM_% - - 2411 100.00 2400 2210 0 0 0.00 0.00 0.00 0.00 63 67 86.52 9.61 0.00 0.00 0 0 2426 100.00 2400 2224 0 0 0.00 0.00 0.00 0.00 61 67 45.71 3.90 0.00 0.00 0 20 2425 100.00 2400 2223 0 0 0.00 1 2 2425 100.00 2400 2223 0 0 0.00 0.00 0.00 0.00 61 1 22 2425 100.00 2400 2223 0 0 0.00 2 4 2425 100.00 2400 2223 0 0 0.00 0.00 0.00 0.00 62 2 24 2425 100.00 2400 2223 0 0 0.00 3 6 2425 100.00 2400 2223 0 0 0.00 0.00 0.00 0.00 61 3 26 2425 100.00 2400 2223 0 0 0.00 4 8 2425 100.00 2400 2223 0 0 0.00 0.00 0.00 0.00 61 4 28 2425 100.00 2400 2223 0 0 0.00 8 10 2425 100.00 2400 2223 0 0 0.00 0.00 0.00 0.00 60 8 30 2425 100.00 2400 2223 0 0 0.00 9 12 2425 100.00 2400 2223 0 0 0.00 0.00 0.00 0.00 61 9 32 2425 100.00 2400 2223 0 0 0.00 10 14 2425 100.00 2400 2223 0 0 0.00 0.00 0.00 0.00 61 10 34 2425 100.00 2400 2223 0 0 0.00 11 16 2425 100.00 2400 2223 0 0 0.00 0.00 0.00 0.00 61 11 36 2425 100.00 2400 2223 0 0 0.00 12 18 2425 100.00 2400 2223 0 0 0.00 0.00 0.00 0.00 59 12 38 2425 100.00 2400 2223 0 0 0.00 0 1 2403 100.00 2400 2202 0 0 0.00 0.00 0.00 0.00 63 65 40.81 5.71 0.00 0.00 0 21 2397 100.00 2400 2198 0 0 0.00 1 3 2397 100.00 2400 2198 0 0 0.00 0.00 0.00 0.00 63 1 23 2397 100.00 2400 2198 0 0 0.00 2 5 2397 100.00 2400 2198 0 0 0.00 0.00 0.00 0.00 63 2 25 2397 100.00 2400 2198 0 0 0.00 3 7 2397 100.00 2400 2198 0 0 0.00 0.00 0.00 0.00 61 3 27 2397 100.00 2400 2198 0 0 0.00 4 9 2397 100.00 2400 2198 0 0 0.00 0.00 0.00 0.00 62 4 29 2397 100.00 2400 2198 0 0 0.00 8 11 2397 100.00 2400 2197 0 0 0.00 0.00 0.00 0.00 61 8 31 2397 100.00 2400 2197 0 0 0.00 9 13 2397 100.00 2400 2197 0 0 0.00 0.00 0.00 0.00 61 9 33 2397 100.00 2400 2197 0 0 0.00 10 15 2397 100.00 2400 2197 0 0 0.00 0.00 0.00 0.00 62 10 35 2397 100.00 2400 2197 0 0 0.00 11 17 2397 100.00 2400 2197 0 0 0.00 0.00 0.00 0.00 62 11 37 2397 100.00 2400 2197 0 0 0.00 12 19 2397 100.00 2400 2197 0 0 0.00 0.00 0.00 0.00 62 12 39 2397 100.00 2400 2197 0 0 0.00 [root@overcloud-compute-0 ~]# grep -i affi /etc/systemd/system.conf #CPUAffinity=1 2 CPUAffinity=0 1 20 21 ~~~ (In reply to Andreas Karis from comment #8) > If I follow Franck's recommendation, then we should not reach C1 for the > cpu-partitioning profile. Or we should have a switch there that allows us to > set /usr/libexec/tuned/pmqos-static.py cpu_dma_latency=0 We could add a configuration file to modify the latency in the underlying latency-performance profile, e.g.: $ cat /etc/tuned/latency-performance-variables.conf cpu_dma_latency=1 But given that doing the same thing by overriding the profile is fairly simple, maybe it's not worth it: > Anyway, here's the way to go to use cpu-partitioning locked to C0: > [root@overcloud-compute-0 ~]# cat > /etc/tuned/cpu-partitioning-disable-cstates/tuned.conf > [main] > include=cpu-partitioning > > [cpu] > force_latency=0 As long as it's documented, it should suffice. But I'm fine to add the configuration file. AFAIK the acpi_idle driver doesn't allow locking to C0, i.e. the effect of the following two boot command lines is the same: processor.max_cstate=0 intel_idle.max_cstate=0 processor.max_cstate=1 intel_idle.max_cstate=0 i.e.: intel_idle.max_cstate=0 disables intel_idle driver and acpi_idle driver is used processor.max_cstate=0 if acpi_idle driver is used, it silently changes max_cstate=0 to max_cstate=1, so C1 is used always. Adding 'idle=poll' will add polling loop which will not allow CPU to sleep, thus it will appear as locked in C0. But this will significantly increase power consumption and the CPU may become very hot. In the kernel documentation there is written that this setting is "not recommended" (idle=poll in e.g. [1]). Tuned uses different mechanism for locking C-states - the PMQoS. It seems it allows locking CPU to C0, but if the acpi_idle driver doesn't allow it, there is probably good reason for not doing it. There is also a possibility that this behavior is a bug in the acpi_idle driver and that it shouldn't allow PMQoS to force C0. In Tuned we switched to C1 due to bug 1013085. IIRC there was some feedback from Intel regarding this, but I can't remember more at the moment. Jeremy can probably say more about this change. If you really need locking to C0, you can customize the Tuned profile. It's easy and documented - there is article about it [2]. Thus I tend to close this bugzilla as wontfix. [1] https://www.kernel.org/doc/html/v4.17/admin-guide/kernel-parameters.html [2] https://access.redhat.com/solutions/3384811 Development Management has reviewed and declined this request. You may appeal this decision by reopening this request. |