Bug 1473831 - Verizon would like to know the impact of setting halt_poll_ns=0 in NFV workloads
Verizon would like to know the impact of setting halt_poll_ns=0 in NFV workloads
Status: CLOSED NOTABUG
Product: Red Hat OpenStack
Classification: Red Hat
Component: qemu-kvm-rhev (Show other bugs)
10.0 (Newton)
x86_64 Linux
unspecified Severity low
: ---
: ---
Assigned To: Virtualization Maintenance
Shai Revivo
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2017-07-21 15:53 EDT by Siggy Sigwald
Modified: 2017-07-24 10:26 EDT (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-07-24 10:26:45 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Siggy Sigwald 2017-07-21 15:53:10 EDT
Verizon wants to know the precise impact of setting halt_poll_ns=0 and how it will impact cpu responsiveness in their environment. The relevant documentation they linked is https://www.ibm.com/support/knowledgecenter/beta/linuxonibm/liaag/wkvm/wkvm_c_tune_kvm.htm and their use case is described as:
PAcket Processing VNFs like vPGW, vSGW, vRAN (PDCP part)
The common theme among these VMs is they receive and send a lot of packets on behalf of the users
They all run DPDK PMDs

They want to know if it will be a performance boost with pinned vcpus and no oversubscription and if the vcpu will transition to idle state when pinned
Comment 1 Andreas Karis 2017-07-21 18:36:11 EDT
Hi,

This is about https://bugzilla.redhat.com/show_bug.cgi?id=1473831

Verizon wants to know the precise impact of setting halt_poll_ns=0 and how it will impact cpu responsiveness in their environment. The relevant documentation they linked is https://www.ibm.com/support/knowledgecenter/beta/linuxonibm/liaag/wkvm/wkvm_c_tune_kvm.htm and their use case is described as:
PAcket Processing VNFs like vPGW, vSGW, vRAN (PDCP part)
The common theme among these VMs is they receive and send a lot of packets on behalf of the users
They all run DPDK PMDs

They want to know if it will be a performance boost with pinned vcpus and no oversubscription and if the vcpu will transition to idle state when pinned

https://www.ibm.com/support/knowledgecenter/en/linuxonibm/liaag/wkvm/wkvm_c_tune_kvm.htm
https://github.com/torvalds/linux/blob/f4000cd99750065d5177555c0a805c97174d1b9f/Documentation/virtual/kvm/halt-polling.txt
~~~
Halt polling essentially presents a trade off between power usage and latency
and the module parameters should be used to tune the affinity for this. Idle
cpu time is essentially converted to host kernel time with the aim of decreasing
latency when entering the guest.
~~~

This seems like a case by case thing, but if I interpret this correctly, then higher values mean that the CPU will poll for a longer time, meaning that the vCPU is *more* responsive. In that context, does halt_poll_ns=0 mean that it should never poll, so go to sleep ASAP. Or does it mean that it should always poll?

Thanks,

Andreas
Comment 2 Andreas Karis 2017-07-24 10:26:45 EDT
Basically, when there are no running tasks in the Guest OS for a
certain CPU, the Guest kernel tries to halt that CPU. This causes a
trap to the Hypervisor, which (in a slightly oversimplified way) will
have two options:

 1. If halt_poll_ns == 0, it'll do one last check to see if, while
switching contexts and doing some bookkeeping, the Guest OS has
changed its mind and now wants to wake the CPU. If that's the case,
it'll enter the Guest immediately. but if the situation hasn't
changed, it'll release the CPU, allowing it to be used for other
tasks, or going to sleep.

 2. If halt_poll_ns > 0, it'll _actively_ wait for that amount of
nanoseconds (there's some throttling, but will ignore it, to keep
things simple), checking if the Guest wants to wake the CPU. Until the
wait expires, the CPU will _not_ be available for other tasks to be
used, nor will it enter any sleep state. This means that if the Guest
is doing short loops of running/sleeping, the effect of halt_poll_ns
!= 0 will manifest itself as a visible %sys CPU load on the Host side.

If they have no oversubscription, and CPUs are completely dedicated,
the only drawback of keeping the default value of halt_poll_ns
(400000) will be an increased CPU usage from the Host PoV, and thus,
an increased power usage. In exchange, the Guest will be more
responsive, in the sense of transitioning quicker from idle to active.
Of course, if there's a sustained load, there wouldn't be any effect,
neither positive nor negative.

On the other hand, if they're deploying their own NFV software, and
are aware of the timing of their run/wait cycles, they may want to
tune halt_poll_ns to be coherent with it.

Note You need to log in before you can comment on or make changes to this bug.