Bug 1170492

Summary: ignore emulatorpin when vcpu placement is auto
Product: Red Hat Enterprise Linux 7 Reporter: Jincheng Miao <jmiao>
Component: libvirtAssignee: Michal Privoznik <mprivozn>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.1CC: dyuan, honzhang, lhuang, mprivozn, mzhan, rbalakri
Target Milestone: rcKeywords: Upstream
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-1.2.13-1.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1170495 (view as bug list) Environment:
Last Closed: 2015-11-19 05:57:08 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1170484    
Bug Blocks: 1170495    

Description Jincheng Miao 2014-12-04 06:45:54 UTC
Description of problem:
vcpu placement is default to numatune placement or "static" if cpuset is specified.
So if user edit domain xml as:
  <vcpu placement='static' current='2'>6</vcpu>
  <numatune>
    <memory mode='strict' placement='auto'/>
  </numatune>

libvirt will turn vcpu placement to 'auto', like:
  <vcpu placement='auto' current='2'>6</vcpu>
  <numatune>
    <memory mode='strict' placement='auto'/>
  </numatune>

But this tuning will break the rule of emulatorpin is not allowed if attribute placement of element vcpu is "auto".

In other word, emulatorpin could be set when vcpu placement is 'static', and numatune memory placement is 'auto':
  <vcpu placement='auto' current='2'>6</vcpu>
  <cputune>
    <emulatorpin cpuset='1-3'/>
  </cputune>
  <numatune>
    <memory mode='strict' placement='auto'/>
  </numatune>



Version-Release number of selected component (if applicable):
libvirt-1.2.8-9.el7.x86_64
qemu-kvm-rhev-2.1.2-13.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1. edit domain xml
# virsh edit r7a
...
  <vcpu placement='static' current='2'>6</vcpu>
  <cputune>
    <emulatorpin cpuset='1-3'/>
  </cputune>
  <numatune>
    <memory mode='strict' placement='auto'/>
  </numatune>

...

# virsh dumpxml r7a
...
  <vcpu placement='auto' current='2'>6</vcpu>
  <cputune>
    <emulatorpin cpuset='1-3'/>
  </cputune>
  <numatune>
    <memory mode='strict' placement='auto'/>
  </numatune>
...

2. start guest
# virsh start r7a

3. check via emulatorpin
# virsh emulatorpin r7a
emulator: CPU Affinity
----------------------------------
       *: 1-3

4. check via cgroup
# cat /sys/fs/cgroup/cpuset/machine.slice/machine-qemu\\x2dr7a.scope/emulator/cpuset.cpus
0-7


Expect result:
Either report error for specifying emulatorpin when vcpu placement is auto,
or ignoring emulatorpin configuration.

Comment 1 Jincheng Miao 2014-12-04 07:06:05 UTC
This problem is also existing for iothreadpin of cputune.

And It's also existed in RHEL6, I will clone it to RHEL6.

Comment 2 Michal Privoznik 2015-01-22 13:09:28 UTC
Patch proposed upstream:

https://www.redhat.com/archives/libvir-list/2015-January/msg00845.html

Comment 3 Michal Privoznik 2015-01-30 13:02:44 UTC
I've just pushed the patch upstream:

commit bbd3eb50987676f486effe55feec1fe0d01cafd0
Author:     Michal Privoznik <mprivozn>
AuthorDate: Thu Jan 22 13:57:22 2015 +0100
Commit:     Michal Privoznik <mprivozn>
CommitDate: Fri Jan 30 13:51:22 2015 +0100

    conf: Don't mangle vcpu placement randomly
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1170492
    
    In one of our previous commits (dc8b7ce7) we've done a functional
    change even though it was intended as pure refactor. The problem is,
    that the following XML:
    
     <vcpu placement='static' current='2'>6</vcpu>
     <cputune>
       <emulatorpin cpuset='1-3'/>
     </cputune>
     <numatune>
       <memory mode='strict' placement='auto'/>
     </numatune>
    
    gets translated into this one:
    
     <vcpu placement='auto' current='2'>6</vcpu>
     <cputune>
       <emulatorpin cpuset='1-3'/>
     </cputune>
     <numatune>
       <memory mode='strict' placement='auto'/>
     </numatune>
    
    We should not change the vcpu placement mode. Moreover, we're doing
    something similar in case of emulatorpin and iothreadpin. If they were
    set, but vcpu placement was auto, we've mistakenly removed them from
    the domain XML even though we are able to set them independently on
    vcpus.
    
    Signed-off-by: Michal Privoznik <mprivozn>

v1.2.12-50-gbbd3eb5

Comment 6 Luyao Huang 2015-06-25 06:53:58 UTC
I can reproduce this issue with libvirt-1.2.8-16.el7.x86_64:

1. edit xml like this:

  <vcpu placement='static'>5</vcpu>
  <iothreads>2</iothreads>
  <cputune>
    <vcpupin vcpu='2' cpuset='1'/>
     <emulatorpin  cpuset='1'/>
  </cputune>
  <numatune>
    <memory mode='strict' placement='auto'/>
  </numatune>

2. recheck the xml after edit:

# virsh dumpxml test4

  <vcpu placement='auto'>5</vcpu>
  <iothreads>2</iothreads>
  <cputune>
    <vcpupin vcpu='2' cpuset='1'/>
    <emulatorpin cpuset='1'/>
  </cputune>
  <numatune>
    <memory mode='strict' placement='auto'/>
  </numatune>

And verify this issue with libvirt-1.2.16-1.el7.x86_64:

1. edit xml like this:

  <vcpu placement='static'>5</vcpu>
  <iothreads>2</iothreads>
  <cputune>
    <vcpupin vcpu='2' cpuset='1'/>
    <emulatorpin cpuset='2'/>
    <iothreadpin iothread='1' cpuset='3'/>
  </cputune>
  <numatune>
    <memory mode='strict' placement='auto'/>
  </numatune>

2. recheck the xml after edit:

# virsh dumpxml test4

  <vcpu placement='static'>5</vcpu>
  <iothreads>2</iothreads>
  <cputune>
    <vcpupin vcpu='2' cpuset='1'/>
    <emulatorpin cpuset='2'/>
    <iothreadpin iothread='1' cpuset='3'/>
  </cputune>
  <numatune>
    <memory mode='strict' placement='auto'/>
  </numatune>

3. start the guest and check the cgroup:

# cgget -g cpuset /machine.slice/machine-qemu\\x2dtest4.scope/iothread1
/machine.slice/machine-qemu\x2dtest4.scope/iothread1:
cpuset.isolcpus: 3
cpuset.memory_spread_slab: 0
cpuset.memory_spread_page: 0
cpuset.memory_pressure: 0
cpuset.memory_migrate: 1
cpuset.sched_relax_domain_level: -1
cpuset.sched_load_balance: 1
cpuset.mem_hardwall: 0
cpuset.mem_exclusive: 0
cpuset.cpu_exclusive: 0
cpuset.mems: 0
cpuset.cpus: 3

# cgget -g cpuset /machine.slice/machine-qemu\\x2dtest4.scope/emulator
/machine.slice/machine-qemu\x2dtest4.scope/emulator:
cpuset.isolcpus: 2
cpuset.memory_spread_slab: 0
cpuset.memory_spread_page: 0
cpuset.memory_pressure: 0
cpuset.memory_migrate: 1
cpuset.sched_relax_domain_level: -1
cpuset.sched_load_balance: 1
cpuset.mem_hardwall: 0
cpuset.mem_exclusive: 0
cpuset.cpu_exclusive: 0
cpuset.mems: 0
cpuset.cpus: 2

# cgget -g cpuset /machine.slice/machine-qemu\\x2dtest4.scope/vcpu2
/machine.slice/machine-qemu\x2dtest4.scope/vcpu2:
cpuset.isolcpus: 
cpuset.memory_spread_slab: 0
cpuset.memory_spread_page: 0
cpuset.memory_pressure: 0
cpuset.memory_migrate: 1
cpuset.sched_relax_domain_level: -1
cpuset.sched_load_balance: 1
cpuset.mem_hardwall: 0
cpuset.mem_exclusive: 0
cpuset.cpu_exclusive: 0
cpuset.mems: 0
cpuset.cpus: 1

Comment 8 errata-xmlrpc 2015-11-19 05:57:08 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2202.html