RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 903123 - The value of steal time in "top" command always is "0.0% st" after guest migration
Summary: The value of steal time in "top" command always is "0.0% st" after guest migr...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: qemu-kvm
Version: 6.4
Hardware: x86_64
OS: Linux
high
medium
Target Milestone: rc
: ---
Assignee: Marcelo Tosatti
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks: 903889
TreeView+ depends on / blocked
 
Reported: 2013-01-23 09:19 UTC by huiqingding
Modified: 2013-11-21 06:30 UTC (History)
13 users (show)

Fixed In Version: qemu-kvm-0.12.1.2-2.404.el6
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 903889 (view as bug list)
Environment:
Last Closed: 2013-11-21 06:30:44 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)
The result of top command in the first source guest. (15.18 KB, image/png)
2013-01-23 09:22 UTC, huiqingding 		no flags Details
The result of top command in the second source guest. (15.11 KB, image/png)
2013-01-23 09:23 UTC, huiqingding 		no flags Details
After migration, the result of top command in the first guest. (15.37 KB, image/png)
2013-01-23 09:24 UTC, huiqingding 		no flags Details
After migration, the result of top command in the second guest. (15.25 KB, image/png)
2013-01-23 09:25 UTC, huiqingding 		no flags Details
View All


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2013:1553 0 normal SHIPPED_LIVE Important: qemu-kvm security, bug fix, and enhancement update 2013-11-20 21:40:29 UTC

Description huiqingding 2013-01-23 09:19:43 UTC 
Description of problem:
In source, boot two guests and bond them to a same cpu. Add 100% cpu load in both guests, the value of steal time are both about "50% st".
 
In destination, migrate the two guests from source and also bond them to a same cpu. After migration, the value of steal time in both destination guest always is "0.0% st".

Version-Release number of selected component (if applicable):
# uname -r
2.6.32-355.el6.x86_64
# rpm -qa | grep qemu-kvm
qemu-kvm-0.12.1.2-2.352.el6.x86_64


How reproducible:
100%

Steps to Reproduce:
1. In source, boot two guest and bond them to a same cpu:
# taskset -c 5 /usr/libexec/qemu-kvm -M pc -cpu SandyBridge -enable-kvm -m 2048 -smp 1 -name rhel6.4 -uuid 6afa5f93-2d4f-420f-81c6-e5fdddbd1c83 -drive file=rhel6-64.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,serial=40c061dd-5d60-4fc5-865f-55db700407f0,cache=none,werror=stop,rerror=stop,aio=threads -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0 -netdev tap,id=hostnet0,vhost=on -device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:1a:4a:42:0c:00,bus=pci.0,addr=0x3 -vnc :1 -monitor stdio
QEMU 0.12.1 monitor - type 'help' for more information
(qemu) 
#  taskset -c 5 /usr/libexec/qemu-kvm -M pc -cpu SandyBridge -enable-kvm -m 2048 -smp 1 -name rhel6.4 -uuid 6afa5f93-2d4f-420f-81c6-e5fdddbd1c83 -drive file=rhel6-64_2.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,serial=40c061dd-5d60-4fc5-865f-55db700407f0,cache=none,werror=stop,rerror=stop,aio=threads -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0 -netdev tap,id=hostnet0,vhost=on -device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:1a:4a:42:0d:00,bus=pci.0,addr=0x3 -vnc :2 -monitor stdio
QEMU 0.12.1 monitor - type 'help' for more information
(qemu)

2. Add 100% cpu load in both guest running the following command:
# for ((;;)) do x=1; done

3. In both two guests, use "top" command to check the steal time

4. In destination, migrate the two guests and also bond them to a same cpu:
#taskset -c 5 /usr/libexec/qemu-kvm -M pc -cpu SandyBridge -enable-kvm -m 2048 -smp 1 -name rhel6.4 -uuid 6afa5f93-2d4f-420f-81c6-e5fdddbd1c83 -drive file=rhel6-64_2.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,serial=40c061dd-5d60-4fc5-865f-55db700407f0,cache=none,werror=stop,rerror=stop,aio=threads -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0 -netdev tap,id=hostnet0,vhost=on -device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:1a:4a:42:0d:00,bus=pci.0,addr=0x3 -vnc :3 -monitor stdio -incoming tcp:0:5800
QEMU 0.12.1 monitor - type 'help' for more information
(qemu)

#  taskset -c 5 /usr/libexec/qemu-kvm -M pc -cpu SandyBridge -enable-kvm -m 2048 -smp 1 -name rhel6.4 -uuid 6afa5f93-2d4f-420f-81c6-e5fdddbd1c83 -drive file=rhel6-64_2.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,serial=40c061dd-5d60-4fc5-865f-55db700407f0,cache=none,werror=stop,rerror=stop,aio=threads -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0 -netdev tap,id=hostnet0,vhost=on -device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:1a:4a:42:0d:00,bus=pci.0,addr=0x3 -vnc :4 -monitor stdio -incoming tcp:0:5801
QEMU 0.12.1 monitor - type 'help' for more information
(qemu)

5. In both two guests, use "top" command to check the steal time
  
Actual results:
Step 3: the steal time of both guests is correct. In both guests, "top" command shows about "50% st". Please refer to "guest_1_src.png" and "guest_2_src.png".
Step 5: After migration, the steal time of both guests are "0.0% st". Please refer to "guest_1_dst.png" and "guest_2_dst.png".

Expected results:
The steal time in Step 3 and Step5 should be about "50% st" showed in top command.
Comment 2 huiqingding 2013-01-23 09:22:04 UTC 
Created attachment 685748 [details]
The result of top command in the first source guest.
Comment 3 huiqingding 2013-01-23 09:23:17 UTC 
Created attachment 685750 [details]
The result of top command in the second source guest.
Comment 4 huiqingding 2013-01-23 09:24:52 UTC 
Created attachment 685751 [details]
After migration, the result of top command in the first guest.
Comment 5 huiqingding 2013-01-23 09:25:34 UTC 
Created attachment 685752 [details]
After migration, the result of top command in the second guest.
Comment 6 huiqingding 2013-01-23 10:19:20 UTC 
"qemu-kvm -M rhel6.4.0" has also this problem.
Comment 7 huiqingding 2013-01-23 10:31:23 UTC 
Before migration, in source guest, we check /proc/stat twice:
# cat /proc/stat
cpu  34653 0 2602 1196 748 4 0 33433 0
cpu0 34653 0 2602 1196 748 4 0 33433 0
# cat /proc/stat
cpu  37273 0 2766 1196 748 4 0 37123 0
cpu0 37273 0 2766 1196 748 4 0 37123 0

We can see the steal time is updated from 33433 to 37123.

After migration, in destination guest, we check /proc/stat twice, found that steal time is not updated and always "42168".
# cat /proc/stat
cpu  56982 0 4128 1196 748 8 16 42168 0
cpu0 56982 0 4128 1196 748 8 16 42168 0
# cat /proc/stat
cpu  62499 0 4506 1196 748 8 16 42168 0
cpu0 62499 0 4506 1196 748 8 16 42168 0
Comment 9 Rik van Riel 2013-01-23 13:14:25 UTC 
This should not have anything to do with qemu-kvm, IIRC steal time is calculated entirely between the guest and the host kernel with no qemu intervention.
Comment 10 RHEL Program Management 2013-01-23 13:18:44 UTC 
This request was evaluated by Red Hat Product Management for
inclusion in the current release of Red Hat Enterprise Linux.
Because the affected component is not scheduled to be updated
in the current release, Red Hat is unable to address this
request at this time.

Red Hat invites you to ask your support representative to
propose this request, if appropriate, in the next release of
Red Hat Enterprise Linux.
Comment 12 Marcelo Tosatti 2013-01-24 01:15:11 UTC 
(In reply to comment #9)
> This should not have anything to do with qemu-kvm, IIRC steal time is
> calculated entirely between the guest and the host kernel with no qemu
> intervention.

Don't see MSR_KVM_STEAL_TIME being saved/restored in qemu-kvm, as its done 
for other kvmclock MSRs, in target-i386/kvm.c:

#endif
    msrs[n++].index = MSR_KVM_SYSTEM_TIME;
    msrs[n++].index = MSR_KVM_WALL_CLOCK;
    if (has_msr_async_pf_en) {
Comment 24 zhonglinzhang 2013-09-30 03:15:48 UTC 
Reproduce with qemu-kvm-0.12.1.2-2.392.el6.x86_64

Steps to Reproduce:
1. In source, boot two guest and bond them to a same cpu:
taskset -c 5 /usr/libexec/qemu-kvm -M pc -cpu SandyBridge -enable-kvm -m 2048 -smp 1 -name rhel6.51 -uuid 6afa5f93-2d4f-420f-81c6-e5fdddbd1c83 -drive file=/mnt/RHEL6.5-64-IDE.raw,if=none,id=drive-virtio-disk0,format=raw,serial=40c061dd-5d60-4fc5-865f-55db700407f0,cache=none,werror=stop,rerror=stop,aio=threads -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0 -netdev tap,id=hostnet0,vhost=on -device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:1a:4a:42:0c:00,bus=pci.0,addr=0x3 -vnc :1   -monitor stdio
QEMU 0.12.1 monitor - type 'help' for more information

taskset -c 5 /usr/libexec/qemu-kvm -M pc -cpu SandyBridge -enable-kvm -m 2048 -smp 1 -name rhel6.52 -uuid 6afa5f93-2d4f-420f-81c6-e5fdddbd1c83 -drive file=/mnt/RHEL222.raw,if=none,id=drive-virtio-disk0,format=raw,serial=40c061dd-5d60-4fc5-865f-55db700407f0,cache=none,werror=stop,rerror=stop,aio=threads -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0 -netdev tap,id=hostnet0,vhost=on -device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:1a:4a:42:0d:00,bus=pci.0,addr=0x3 -vnc :2   -monitor stdio
QEMU 0.12.1 monitor - type 'help' for more information

2. Add 100% cpu load in both guest running the following command:
# for ((;;)) do x=1; done

3. In both two guests, use "top" command to check the steal time

4. In destination, migrate the two guests and also bond them to a same cpu:
taskset -c 5 /usr/libexec/qemu-kvm -M pc -cpu SandyBridge -enable-kvm -m 2048 -smp 1 -name rhel6.51 -uuid 6afa5f93-2d4f-420f-81c6-e5fdddbd1c83 -drive file=/mnt/RHEL6.5-64-IDE.raw,if=none,id=drive-virtio-disk0,format=raw,serial=40c061dd-5d60-4fc5-865f-55db700407f0,cache=none,werror=stop,rerror=stop,aio=threads -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0 -netdev tap,id=hostnet0,vhost=on -device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:1a:4a:42:0c:00,bus=pci.0,addr=0x3 -vnc :1   -monitor stdio  -incoming tcp:0:5888
QEMU 0.12.1 monitor - type 'help' for more information

taskset -c 5 /usr/libexec/qemu-kvm -M pc -cpu SandyBridge -enable-kvm -m 2048 -smp 1 -name rhel6.52 -uuid 6afa5f93-2d4f-420f-81c6-e5fdddbd1c83 -drive file=/mnt/RHEL222.raw,if=none,id=drive-virtio-disk0,format=raw,serial=40c061dd-5d60-4fc5-865f-55db700407f0,cache=none,werror=stop,rerror=stop,aio=threads -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0 -netdev tap,id=hostnet0,vhost=on -device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:1a:4a:42:0d:00,bus=pci.0,addr=0x3 -vnc :2   -monitor stdio  -incoming tcp:0:6999

5. In both two guests, use "top" command to check the steal time
  
Actual results:
Step 3: the steal time of both guests is correct. In both guests, "top" command shows about "50% st". 
Step 5: After migration, the steal time of both guests are "0.0% st".

Expected results:
The steal time in Step 3 and Step5 should be about "50% st" showed in top command.



Verify this issue with qemu-kvm-0.12.1.2-2.407.el6.x86_64

Actual results:
The steal time in Step 3 and Step5 are about "50% st" showed in top command.


Based on above information, so this issue has been fixed.
Comment 26 Eduardo Habkost 2013-10-24 08:44:20 UTC 
Possibly related: bug 1022821
Comment 27 errata-xmlrpc 2013-11-21 06:30:44 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2013-1553.html
Note You need to log in before you can comment on or make changes to this bug.