Bug 1126762

Summary: numatune and domiftune should be unsupported in session mode
Product: Red Hat Enterprise Linux 7 Reporter: Luyao Huang <lhuang>
Component: libvirtAssignee: Erik Skultety <eskultet>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.1CC: dyuan, eskultet, lcheng, lhuang, mzhan, phrdina, rbalakri, zpeng
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-1.2.13-1.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-11-19 05:46:39 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Luyao Huang 2014-08-05 08:35:01 UTC
Description of problem:
numatune and domiftune should be unsupported in session mode

Version-Release number of selected component (if applicable):
libvirt-1.2.7-1.el7.x86_64
kernel-3.10.0-140.el7.x86_64

How reproducible:
100%

Steps to Reproduce:

1.$ virsh uri
qemu:///session

2.$ virsh list --all
 Id    Name                           State
----------------------------------------------------
 -     sdsd                           shut off


3.$ virsh domiflist sdsd
Interface  Type       Source     Model       MAC
-------------------------------------------------------
-          user       -          virtio      52:54:00:5e:cc:c9
top0       user       -          virtio      52:54:00:5e:cc:c7
top10      user       -          virtio      52:54:00:5e:cc:c8


4.$ virsh domiftune sdsd top0 1124,23,345

5.$ virsh numatune sdsd 0 0

6.$ virsh numatune sdsd
numa_mode      : strict
numa_nodeset   : 0

$ virsh domiftune sdsd top0
inbound.average: 245
inbound.peak   : 3567
inbound.burst  : 255
outbound.average: 0
outbound.peak  : 0
outbound.burst : 0


7.$ virsh start sdsd
Domain sdsd started


6.$ virsh numatune sdsd
error: Unable to change numa parameters
error: Requested operation is not valid: cgroup cpuset controller is not mounted

7.$ virsh domiftune sdsd top0
inbound.average: 1124
inbound.peak   : 23
inbound.burst  : 345
outbound.average: 0
outbound.peak  : 0
outbound.burst : 0


8.$ virsh domiftune sdsd top0 124,231,322
error: Unable to set interface parameters
error: internal error: cannot set bandwidth limits on top0

Actual results:
Can't set limits for a running guest ,because cgroup is not mounted in session mode

Expected results:
numatune and domiftune should be unsupported in session mode
and report error like memtune:
error: Unable to get number of memory parameters
error: Operation not supported: Memory tuning is not available in session mode

Additional info:

Comment 3 Erik Skultety 2014-10-23 07:10:02 UTC
Fixed upstream:

commit 43b67f2e71ca50df6619b7a57dc76124ca956dc6
Author: Erik Skultety <eskultet>
Date:   Wed Oct 1 14:57:32 2014 +0200

    qemu: Disallow NUMA/network tuning for session mode
    
    Tuning NUMA or network interface parameters requires root
    privileges to manage cgroups. Thus an attempt to set some of these
    parameters in session mode on a running domain should be invalid
    followed by an error. An example might be memory tuning which raises
    an error in such case.
    
    The following behavior in session mode will be present after applying
    this patch:
    
      Tuning  |      SET      |   GET  |
    ----------|---------------|--------|
    NUMA      | shut off only | always |
    Memory    |     never     | never  |
    Interface |     never     | always |

v1.2.9-106-g43b67f2

Comment 4 Luyao Huang 2015-02-17 03:39:50 UTC
I found this patch will cause a new issue when try to change parameters:

[lhaung@lhuang ~]$ virsh numatune test6 0 1
error: Unable to change numa parameters
error: Operation not supported: NUMA tuning is not available in session mode

[lhaung@lhuang ~]$ virsh list --all
 Id    Name                           State
----------------------------------------------------
 2     test6                          running
 -     sdsd                           shut off

[lhaung@lhuang ~]$ virsh destroy test6
error: Failed to destroy domain test6
error: Timed out during operation: cannot acquire state change lock


I just sent a patch to upstream:

https://www.redhat.com/archives/libvir-list/2015-February/msg00569.html

Comment 5 Pavel Hrdina 2015-02-17 10:16:26 UTC
The patch for deadlock is pushed upstream now:

commit 337265bb52122ee536430e2493618ab4aa565987
Author: Luyao Huang <lhuang>
Date:   Tue Feb 17 11:37:52 2015 +0800

    qemu: fix vm deadlock when try to use numatune in session mode

Comment 7 lcheng 2015-04-21 08:01:34 UTC
Hi Erik,

I am virifying this bug. By comment 3, do you means that setting network interface parameters will fail in session mode on a shut off or running domain? In fact, I found setting network interface parameters was successful in session mode on a shut off domain. I am not sure which result is expected. Could you tell me where was wrong?

My test steps as follows:

[test@localhost ~]$ rpm -q libvirt
libvirt-1.2.14-1.el7.x86_64

[test@localhost ~]$ virsh uri
qemu:///session

[test@localhost ~]$ virsh list --all
 Id    Name                           State
----------------------------------------------------
 -     7                              shut off

[test@localhost ~]$ virsh domiflist 7
Interface  Type       Source     Model       MAC
-------------------------------------------------------
-          user       -          virtio      00:16:3e:3e:a9:11
top0       user       -          virtio      00:16:3e:3e:a9:12
top10      user       -          virtio      00:16:3e:3e:a9:13

[test@localhost ~]$ virsh domiftune 7 top0
inbound.average: 3456
inbound.peak   : 32
inbound.burst  : 678
outbound.average: 0
outbound.peak  : 0
outbound.burst : 0

[test@localhost ~]$ virsh domiftune 7 top0 1234,89,345

[test@localhost ~]$ virsh domiftune 7 top0
inbound.average: 1234
inbound.peak   : 89
inbound.burst  : 345
outbound.average: 0
outbound.peak  : 0
outbound.burst : 0

[test@localhost ~]$ virsh start 7
Domain 7 started

[test@localhost ~]$ virsh domiftune 7 top0
inbound.average: 1234
inbound.peak   : 89
inbound.burst  : 345
outbound.average: 0
outbound.peak  : 0
outbound.burst : 0

[test@localhost ~]$ virsh domiftune 7 top0 3456,67,432
error: Unable to set interface parameters
error: Operation not supported: Network bandwidth tuning is not available in session mode

Comment 8 Erik Skultety 2015-04-21 11:55:59 UTC
Hmm, I see you're using interface type user, have a look in daemon's log, there should be a warning "setting bandwidth on interface of type 'user' is not implemented yet" (as a fix for BZ 1165993). Because of the unsupported type, bandwidth setting is ignored and the domain starts. Any further requests to set the bandwidth will fail the way you encountered. If you change the interface type to e.g. 'network' (which is supported), the domain won't even start and will fail with the same error message. So technically, you're not doing anything wrong. 
There is one thing however I'm not completely sure of...When you try to set bandwidth to a shutoff domain, we update our config and all following requests for bandwidth configuration will succeed even though setting the bandwidth in session mode doesn't make much sense, as 'tc' requires root permissions and the domain won't start anyway. If it's really an issue for you, I'd suggest posting a question to the list and see if someone else has anything to say about it.

Comment 9 lcheng 2015-04-28 06:46:40 UTC
Additional test:

1. 
[test@localhost ~]$ rpm -q libvirt
libvirt-1.2.14-1.el7.x86_64
[test@localhost ~]$ virsh uri
qemu:///session

[test@localhost ~]$ virsh list --all
 Id    Name                           State
----------------------------------------------------
 -     7                              shut off

[test@localhost ~]$ virsh domiflist 7
Interface  Type       Source     Model       MAC
-------------------------------------------------------
top0       user       -          virtio      00:16:3e:3e:a9:12


2. Get/set numatune in session mode on a shut off domain.
[test@localhost ~]$ virsh numatune 7
numa_mode      : strict
numa_nodeset   : 0

[test@localhost ~]$ virsh numatune 7 2 1

[test@localhost ~]$ virsh numatune 7
numa_mode      : interleave
numa_nodeset   : 1

[test@localhost ~]$ virsh numatune 7 0 0


3. Get/set memtune in session mode on a shut off domain.
[test@localhost ~]$ virsh memtune 7
error: Unable to get number of memory parameters
error: Operation not supported: Memory tuning is not available in session mode

[test@localhost ~]$ virsh memtune 7 --hard-limit 102400
error: Unable to change memory parameters
error: Operation not supported: Memory tuning is not available in session mode


4. Get/set domiftune in session mode on a shut off domain.
[test@localhost ~]$ virsh domiftune 7 top0
inbound.average: 0
inbound.peak   : 0
inbound.burst  : 0
outbound.average: 0
outbound.peak  : 0
outbound.burst : 0

[test@localhost ~]$ virsh domiftune 7 top0 1234,45,321

[test@localhost ~]$ virsh domiftune 7 top0
inbound.average: 1234
inbound.peak   : 45
inbound.burst  : 321
outbound.average: 0
outbound.peak  : 0
outbound.burst : 0

[test@localhost ~]$ virsh start 7
Domain 7 started


5. Get/set numatune in session mode on a running domain.
[test@localhost ~]$ virsh numatune 7 0 0
error: Unable to change numa parameters
error: Operation not supported: NUMA tuning is not available in session mode

[test@localhost ~]$ virsh numatune 7
numa_mode      : strict
numa_nodeset   : 0


6. Get/set memtune in session mode on a running domain.
[test@localhost ~]$ virsh memtune 7
error: Unable to get number of memory parameters
error: Operation not supported: Memory tuning is not available in session mode

[test@localhost ~]$ virsh memtune 7 --hard-limit 102400
error: Unable to change memory parameters
error: Operation not supported: Memory tuning is not available in session mode


7. Get/set domiftune in session mode on a running domain.
[test@localhost ~]$ virsh dumpxml 7 | grep interface -A10
    <interface type='user'>
      <mac address='00:16:3e:3e:a9:12'/>
      <bandwidth>
        <inbound average='1234' peak='45' burst='321'/>
      </bandwidth>
      <target dev='top0'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
...

[test@localhost ~]$ virsh domiftune 7 top0
inbound.average: 1234
inbound.peak   : 45
inbound.burst  : 321
outbound.average: 0
outbound.peak  : 0
outbound.burst : 0

[test@localhost ~]$ virsh domiftune 7 top0 4567,31,453
error: Unable to set interface parameters
error: Operation not supported: Network bandwidth tuning is not available in session mode

[test@localhost ~]$ virsh destroy 7
Domain 7 destroyed

Comment 11 lcheng 2015-05-11 10:39:11 UTC
According to comment 7, comment 8 and comment 9, move to VERIFIED.

Comment 13 errata-xmlrpc 2015-11-19 05:46:39 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2202.html