Bug 802251 - kvm-perf backport: Lookup iobus devices by bsearch and resize kvm_io_range array dynamically
kvm-perf backport: Lookup iobus devices by bsearch and resize kvm_io_range ar...
Status: CLOSED WONTFIX
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: kernel (Show other bugs)
6.3
x86_64 Linux
medium Severity medium
: rc
: ---
Assigned To: Amos Kong
Virtualization Bugs
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-03-12 04:23 EDT by Amos Kong
Modified: 2015-05-24 20:06 EDT (History)
8 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-07-26 06:59:25 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
0001-lib-Add-generic-binary-search-function-to-the-kernel.patch 0002-KVM-Intelligent-device-lookup-on-I-O-bus.patch 0003-KVM-Resize-kvm_io_range-array-dynamically.patch (6.33 KB, application/x-gzip)
2012-03-12 04:23 EDT, Amos Kong
no flags Details
哈慈看KVM-tools patch (2.14 KB, patch)
2012-03-26 04:24 EDT, Amos Kong
no flags Details | Diff

  None (edit)
Description Amos Kong 2012-03-12 04:23:58 EDT
Created attachment 569298 [details]
0001-lib-Add-generic-binary-search-function-to-the-kernel.patch 0002-KVM-Intelligent-device-lookup-on-I-O-bus.patch 0003-KVM-Resize-kvm_io_range-array-dynamically.patch

Description of problem:

I'm try to resolve bz753692 by backport 4 patches from upstream,
  - [PATCH] lib: Add generic binary search function to the kernel. 
  - [PATCH] KVM: Intelligent device lookup on I/O bus
  - [PATCH] KVM: Resize kvm_io_range array dynamically
  - [PATCH] KVM: set upper bounds for iobus dev to limit userspace 
but mst suggest me to resolve that bz by another method.

Patch 1~2 make devices lookup by an efficient bsearch, Patch 3 will save some memory.
we need to test if patch 1,2,3 bring performance improvement. If it's useful for rhel, we can make an exception for this bz in 6.3.


brew build: https://brewweb.devel.redhat.com/taskinfo?taskID=4141396
Comment 1 Amos Kong 2012-03-12 22:46:19 EDT
Test method:

1. Create a guest.
2. Create 200 ioeventfds in that guest.
3. Write to a memory address in the guest which isn't taken by the
ioeventfds (i.e., worst case scenario for the lookup).
4. Measure vm exits per second. test 60 seconds


python script to write addr which is not taken by ioeventfds:
-----------------
import random, os
while True:
    #r = random.SystemRandom()
    #port = r.randint(0,256)
    port = 13
    outw_cmd = ("echo -e '\\%s' | dd of=/dev/port seek=%d bs=1 count=2" % (oct(0), port))
    print outw_cmd
    os.system(outw_cmd)
Comment 4 Amos Kong 2012-03-26 04:24:26 EDT
Created attachment 572696 [details]
哈慈看KVM-tools patch
Comment 5 Amos Kong 2012-03-26 04:29:02 EDT
Patch in comment #4 is used to hack kvm-tools to allocate multiple ioeventfds on startup. Since creating actual devices might impact testing.
Comment 6 Amos Kong 2012-07-17 04:35:52 EDT
Hi miya,

Please help to test if performance improvement exists with patches in commit #0, it's host kernel patch.

test steps:
1. clone kvm-tools code, apply patches in Comment #4, and compile
2. start a guest with changed kvm-tools
3. Use python script [1] to write a memory address in the guest which isn't taken by the ioeventfds (worst case scenario for the lookup).
4. Measure vm exits per second. test 600 seconds

repeat test for 3 times.


script [1]
--------------------
import random, os
while True:
    r = random.SystemRandom()
    port = r.randint(0,256)
    outw_cmd = ("echo -e '\\%s' | dd of=/dev/port seek=%d bs=1 count=2" % (oct(0), port))
    print outw_cmd
    os.system(outw_cmd)
--------------------
Comment 8 Amos Kong 2012-07-18 03:12:33 EDT
=== Test results from juzhang ===

conclusion summary : io_exits is improved about %3.89, exits is improved about %1.61

 2.6.32-286.x86_64
 ####################starting###################
 exits                   17519340     80647
 io_exits                 3752898     17628
 ***********1 times*******
 io_exits                 5520938     17720
 exits                   25685528     81403
 ***********2 times*******
 io_exits                 7300161     17302
 exits                   33878570     79823
 ***********3 times*******
 io_exits                 9069556     17263
 exits                   42049359     80426
 ***********4 times*******
 exits                   50234487     80278
 io_exits                10845042     17400
 ***********5 times*******
 io_exits                12613537     17103
 exits                   58401220     79647

 2.6.32-286.el6802251.x86_64
 ####################starting###################
 exits                   34321652     82416
 io_exits                 7554828     18258
 ***********1 times*******
 exits                   42621517     83866
 io_exits                 9392517     18754
 ***********2 times*******
 io_exits                11235984     17860
 exits                   50939770     81866
 ***********3 times*******
 exits                   59252510     82420
 io_exits                13077731     18297
 ***********4 times*******
 exits                   67561730     81607    
 io_exits                14918771     18076    
 ***********5 times*******
 io_exits                16760017     18278    
 exits                   75862582     82110    


So I would backport it to internal.
Comment 9 Ronen Hod 2012-07-23 10:34:41 EDT
(In reply to comment #8)
> === Test results from juzhang ===
> 
> conclusion summary : io_exits is improved about %3.89, exits is improved
> about %1.61
> 
>  2.6.32-286.x86_64

What are the numbers? The LHS numbers show a big increase, but I do not know what they are.

>  ####################starting###################
>  exits                   17519340     80647
>  io_exits                 3752898     17628
>  ***********1 times*******
>  io_exits                 5520938     17720
>  exits                   25685528     81403
>  ***********2 times*******
>  io_exits                 7300161     17302
>  exits                   33878570     79823
>  ***********3 times*******
>  io_exits                 9069556     17263
>  exits                   42049359     80426
>  ***********4 times*******
>  exits                   50234487     80278
>  io_exits                10845042     17400
>  ***********5 times*******
>  io_exits                12613537     17103
>  exits                   58401220     79647
> 
>  2.6.32-286.el6802251.x86_64
>  ####################starting###################
>  exits                   34321652     82416
>  io_exits                 7554828     18258
>  ***********1 times*******
>  exits                   42621517     83866
>  io_exits                 9392517     18754
>  ***********2 times*******
>  io_exits                11235984     17860
>  exits                   50939770     81866
>  ***********3 times*******
>  exits                   59252510     82420
>  io_exits                13077731     18297
>  ***********4 times*******
>  exits                   67561730     81607    
>  io_exits                14918771     18076    
>  ***********5 times*******
>  io_exits                16760017     18278    
>  exits                   75862582     82110    
> 
> 
> So I would backport it to internal.
Comment 10 Amos Kong 2012-07-23 22:48:30 EDT
(In reply to comment #9)
> (In reply to comment #8)
> > === Test results from juzhang ===
> > 
> > conclusion summary : io_exits is improved about %3.89, exits is improved
> > about %1.61
> > 
> >  2.6.32-286.x86_64
> 
> What are the numbers? The LHS numbers show a big increase, but I do not know
> what they are.


> >  ####################starting###################
> >  exits                   17519340     80647
> >  io_exits                 3752898     17628

initial 'exits','io_exits' count (please ignore last column)

> >  ***********1 times*******
> >  io_exits                 5520938     17720
> >  exits                   25685528     81403

count numbers after 100 seconds

> >  ***********2 times*******
> >  io_exits                 7300161     17302
> >  exits                   33878570     79823

count numbers after 200 seconds

> >  ***********3 times*******
> >  io_exits                 9069556     17263
> >  exits                   42049359     80426

count numbers after 300 seconds

> >  ***********4 times*******
> >  exits                   50234487     80278
> >  io_exits                10845042     17400

count numbers after 400 seconds

> >  ***********5 times*******
> >  io_exits                12613537     17103
> >  exits                   58401220     79647

count numbers after 500 seconds


compute the rise number during 500 seconds.

io_exits: X1 = 12613537 - 3752898 = 8860639
exits: Y1 = 58401220 - 17519340 = 40881880


> >  2.6.32-286.el6802251.x86_64
> >  ####################starting###################
> >  exits                   34321652     82416
> >  io_exits                 7554828     18258
> >  ***********1 times*******
> >  exits                   42621517     83866
> >  io_exits                 9392517     18754
> >  ***********2 times*******
> >  io_exits                11235984     17860
> >  exits                   50939770     81866
> >  ***********3 times*******
> >  exits                   59252510     82420
> >  io_exits                13077731     18297
> >  ***********4 times*******
> >  exits                   67561730     81607    
> >  io_exits                14918771     18076    
> >  ***********5 times*******
> >  io_exits                16760017     18278    
> >  exits                   75862582     82110    

compute the rise number during 500 seconds.
io_exits: X2 = 16760017 - 7554828 = 9205189
exits: Y2 = 75862582 - 34321652 = 41540930

(X2 - X1 ) / X1 = 0.038885
(Y2 - Y1 ) / Y1 = 0.016121

It means the search speed of io_bus dev was improved, then more io_exits & exits can occur.
Comment 11 Amos Kong 2012-07-26 01:34:12 EDT
hi avi,

do you think it's necessary to backport those patches to internal?
Comment 12 Avi Kivity 2012-07-26 06:46:59 EDT
IMO, no.  The improvement is only with a large number of ioeventfds, yes?  That's not a common configuration.
Comment 13 Amos Kong 2012-07-26 06:59:25 EDT
(In reply to comment #12)
> IMO, no.  The improvement is only with a large number of ioeventfds, yes? 

Yes, the 'improvement' only be obvious with a larger number of ioeventfds.

> That's not a common configuration.

So close this bug as WONTFIX.

Note You need to log in before you can comment on or make changes to this bug.