Bug 1129259

Summary: Add traces to virtio-rng device
Product: Red Hat Enterprise Linux 7 Reporter: Amit Shah <amit.shah>
Component: qemu-kvm-rhevAssignee: Amit Shah <amit.shah>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.1CC: amit.shah, flang, hhuang, huding, juzhang, mazhang, michen, virt-maint, xfu
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: qemu-kvm-rhev-2.1.0-4.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-03-05 09:49:13 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Amit Shah 2014-08-12 12:22:45 UTC
Description of problem:

Backport traces for the virtio-rng device.  These are helpful to debug guest problems.

Comment 1 Miroslav Rezanina 2014-09-17 07:37:49 UTC
Fix included in qemu-kvm-rhev-2.1.0-4.el7

Comment 3 langfang 2014-09-19 08:38:26 UTC
 

Test on qemu-kvm-rhev-2.1.0-4.el7 version

I can find these traces about rng devices

#cat vim /usr/share/systemtap/tapset/qemu-kvm.stp
...

probe qemu.kvm.virtio_rng_guest_not_ready = process("/usr/libexec/qemu-kvm").mark("virtio_rng_guest_not_ready")
{
  rng = $arg1;
}
probe qemu.kvm.virtio_rng_pushed = process("/usr/libexec/qemu-kvm").mark("virtio_rng_pushed")
{
  rng = $arg1;
  len = $arg2;
}
probe qemu.kvm.virtio_rng_request = process("/usr/libexec/qemu-kvm").mark("virtio_rng_request")
{
  rng = $arg1;
  size = $arg2;
  quota = $arg3;
}

Comment 4 langfang 2014-09-22 01:40:45 UTC
Test this bug as follow version:
Host:
# uname -r 
3.10.0-144.el7.x86_64
# rpm -q qemu-kvm-rhev
qemu-kvm-rhev-2.1.0-4.el7.x86_64

Guest :rhel7

Prepare script for virtio_rng device.

# cat qemu.kvm.virtio_rng_guest_not_ready
probe qemu.kvm.virtio_rng_guest_not_ready 

{
	 printf("rng=%d\n",rng);

}


# cat qemu.kvm.virtio_rng_pushed.stp
probe qemu.kvm.virtio_rng_pushed

{
		 printf("rng=%d\n",rng);
		 	  printf("len=%d\n",len);

}
# cat qemu.kvm.virtio_rng_request.stp
probe qemu.kvm.virtio_rng_request

{
		 printf("rng=%d\n",rng);
		 	  printf("size=%d\n",size);
			  	   printf("quota=%d\n",quota);

}


Steps
1.Boot guest with "-device virtio-rng-pci,max-bytes=1024,period=1000"

2.While booting guest ,execute scripts 
# stap qemu.kvm.virtio_rng_pushed.stp


#stap qemu.kvm.virtio_rng_request.stp

#stap qemu.kvm.virtio_rng_guest_not_ready


Results:
# stap qemu.kvm.virtio_rng_request.stp
rng=140713073408696
size=0
quota=1024
rng=140713073408696
size=0
quota=1024
rng=140713073408696
....
#stap qemu.kvm.virtio_rng_pushed.stp
rng=140713073408696
len=64
rng=140713073408696
len=64
rng=140713073408696
len=64
rng=140713073408696
len=64
rng=140713073408696
...

# stap qemu.kvm.virtio_rng_guest_not_ready 
rng=140713073408696
rng=140713073408696
rng=140713073408696
rng=140713073408696
rng=140713073408696
...



Hi,Amit
   Could we can verify this bug as above test? 

best regards
fang lang

Comment 5 Amit Shah 2014-09-24 01:59:52 UTC
You're seeing some values, which is fine, I guess.

A better test is to check against a known thing.  Like, with the new initial randomness patch in bug 1119299, you should see traces at guest bootup about virtio_rng_pushed.  With previous guest kernels, you should not see them.

Also, with the new khwrngd in bug 1129195, reading /dev/random should be much faster inside the guest, and show a lot of traces.  With previous kernels, reading from /dev/random in guest should not show traces.

Can you please check it that way?  In doing that, you will also be verifiying those other two bugs :)

Comment 6 langfang 2014-09-24 07:25:55 UTC
(In reply to Amit Shah from comment #5)
> You're seeing some values, which is fine, I guess.
> 
> A better test is to check against a known thing.  Like, with the new initial
> randomness patch in bug 1119299, you should see traces at guest bootup about
> virtio_rng_pushed.  With previous guest kernels, you should not see them.
> 
Hi, Amit
   For comment4 test, i use the old kernel(guest:3.10.0-123.el7.x86_64) to test,that's mean i can get the traces on previous kernel.

Results:
stap qemu.kvm.virtio_rng_pushed.stp
rng=140094291921336
len=64
rng=140094291921336
len=64
rng=140094291921336
len=64
rng=140094291921336
len=64
rng=140094291921336
len=64
rng=140094291921336
len=64
...

> Also, with the new khwrngd in bug 1129195, reading /dev/random should be
> much faster inside the guest, and show a lot of traces.  With previous
> kernels, reading from /dev/random in guest should not show traces.
> 
> Can you please check it that way?  In doing that, you will also be
> verifiying those other two bugs :)

I test on previous kernel (guest:3.10.0-123.el7.x86_64),in guest ,do " dd if=/dev/random of=test bs=512 count=1000

# dd if=/dev/random of=test bs=512 count=10000 iflag=fullblock
10000+0 records in
10000+0 records out
5120000 bytes (5.1 MB) copied, 6.17742 s, 829 kB/s


Test on latest kernel(3.10.0-167.el7.x86_64)
#  dd if=/dev/random of=test bs=512 count=10000 iflag=fullblock
10000+0 records in
10000+0 records out
5120000 bytes (5.1 MB) copied, 5.40637 s, 947 kB/s

On latest kernel for guest , faster than previous kernel.

Anything i was wrong, please correct me .thanks

Comment 7 Amit Shah 2014-09-24 08:45:01 UTC
Please try with the default /dev/random device instead of urandom on host.

Comment 8 langfang 2014-09-24 09:10:40 UTC
(In reply to Amit Shah from comment #7)
> Please try with the default /dev/random device instead of urandom on host.

Test this bug as follow version:
Host:
# uname -r
3.10.0-167.el7.x86_64
# rpm -q qemu-kvm-rhev
qemu-kvm-rhev-2.1.0-4.el7.x86_64

Guest: 3.10.0-167.el7.x86_64

Prepare:

# cat qemu.kvm.virtio_rng_pushed.stp 
probe qemu.kvm.virtio_rng_pushed

{
		 printf("rng=%d\n",rng);
		 	  printf("len=%d\n",len);

}


Steps:
1.#stap qemu.kvm.virtio_rng_pushed.stp

2.Boot guest with 
-object rng-random,id=rng0,filename=/dev/random -device virtio-rng-pci,rng=rng0  

3.After guest boot up ,dd in guest

# dd if=/dev/random of=test bs=512 count=10000 iflag=fullblock

Results:
On host:
#stap qemu.kvm.virtio_rng_pushed.stp
rng=139746200938936
len=64
rng=139746200938936
len=64
rng=139746200938936
len=64
rng=139746200938936
len=64
....

In guest:
# dd if=/dev/random of=test bs=512 count=10000 iflag=fullblock
10000+0 records in
10000+0 records out
5120000 bytes (5.1 MB) copied, 15.639 s, 327 kB/s


Test on old kernel version 

# uname -r
3.10.0-167.el7.x86_64
# rpm -q qemu-kvm-rhev
qemu-kvm-rhev-2.1.0-4.el7.x86_64

Guest: 3.10.0-123.el7.x86_64

Steps As same as above 

Results:
On Host:can get traces log

In guest:
#  dd if=/dev/random of=test bs=512 count=10000 iflag=fullblock
10000+0 records in
10000+0 records out
5120000 bytes (5.1 MB) copied, 16.3997 s, 312 kB/s


MY CLI:
 /usr/libexec/qemu-kvm -cpu SandyBridge -enable-kvm -m 4G -smp 4,sockets=1,cores=4,threads=1 -name test -rtc base=localtime,clock=host,driftfix=slew  -k en-us  -boot menu=on -vnc :3 -vga cirrus -usb -device usb-tablet -monitor stdio -drive file=/home/RHEL-Server-7.0-64-virtio.qcow2,if=none,id=drive-system-disk,media=disk,format=qcow2,aio=native,werror=stop,rerror=stop -device virtio-blk-pci,drive=drive-system-disk,id=system-disk,addr=0x3 -qmp unix:/tmp/q1,server,nowait -net none -object rng-random,id=rng0,filename=/dev/random -device virtio-rng-pci,rng=rng0  -netdev tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup -device virtio-net-pci,netdev=hostnet0,id=vnet0,mac=52:54:00:1a:2b:02 -trace events=/flang/vrng-trace-events

Comment 9 langfang 2014-09-26 02:47:03 UTC
<amit> what if you only use ssh?
<amit> ie no mouse and not much keyboard after starting the dd?
<amit> and you can use /dev/urandom for this test

*Test this bug as above request

Host:
# uname -r
3.10.0-167.el7.x86_64
# rpm -q qemu-kvm-rhev
qemu-kvm-rhev-2.1.0-4.el7.x86_64

Guest:RHEL7 

3.10.0-167.el7.x86_64

Steps:
1.Boot guest with 
...-object rng-random,id=rng0,filename=/dev/urandom -device virtio-rng-pci,rng=rng0...

2.After guest boot up ,ssh guest,then dd

# dd if=/dev/random of=test bs=512 count=10000 iflag=fullblock
10000+0 records in
10000+0 records out
5120000 bytes (5.1 MB) copied, 3.37219 s, 1.5 MB/s


*Test on old kernel
# uname -r
3.10.0-167.el7.x86_64
# rpm -q qemu-kvm-rhev
qemu-kvm-rhev-2.1.0-4.el7.x86_64

Guest:
3.10.0-123.el7.x86_64

Steps as same as above

Results:
IN guest:

# dd if=/dev/random of=test bs=512 count=10000 iflag=fullblock
10000+0 records in
10000+0 records out
5120000 bytes (5.1 MB) copied, 4.48857 s, 1.1 MB/s

Comment 11 langfang 2014-09-28 05:59:41 UTC
Hi Amit
   QE set this bz as verified first. Detailed reasons as below:
   1.Current, with my understanding, 3 bzs(this bug,bug1119299,bug1129195)  involved in the rng device issue,With my understing.This bz is mainly focus on traces events.The others not only focus on traces.
   2.This bug is 'qemu-kvm-rhev' component bug,not depend on bugs. The other two are 'kernel' bugs.

   Amit,what's your option, do you want to let QE keep all these 3 bzs on qa status then verify it together finally or you want to verify this bug first,then test other two?


thanks 
fang lang

Comment 12 Amit Shah 2014-09-29 05:57:22 UTC
Yes, this is fine.

Comment 14 errata-xmlrpc 2015-03-05 09:49:13 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0624.html