Bug 681166 - Improve performance of map encoding into amqp in python qpid client
Summary: Improve performance of map encoding into amqp in python qpid client
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise MRG
Classification: Red Hat
Component: python-qpid
Version: 1.3
Hardware: Unspecified
OS: Linux
high
medium
Target Milestone: 2.0
: ---
Assignee: Rafael H. Schloming
QA Contact: Petra Svobodová
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-03-01 11:10 UTC by Siddhesh Poyarekar
Modified: 2018-11-14 16:28 UTC (History)
7 users (show)

Fixed In Version: python-qpid-0.9.1079482
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2011-06-23 15:46:17 UTC
Target Upstream Version:


Attachments (Terms of Use)
patch to speed up map encoding (1.06 KB, application/octet-stream)
2011-03-01 11:10 UTC, Siddhesh Poyarekar
no flags Details
python program that acts as requestor as well as responder (4.72 KB, application/octet-stream)
2011-03-01 11:14 UTC, Siddhesh Poyarekar
no flags Details
statistics before the patch (1.83 KB, application/octet-stream)
2011-03-01 11:16 UTC, Siddhesh Poyarekar
no flags Details
statistics after the patch (1.83 KB, application/octet-stream)
2011-03-01 11:16 UTC, Siddhesh Poyarekar
no flags Details
script to invoke the test program in requestor mode multiple times (269 bytes, application/octet-stream)
2011-03-01 11:17 UTC, Siddhesh Poyarekar
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2011:0890 0 normal SHIPPED_LIVE Red Hat Enterprise MRG Messaging 2.0 Release 2011-06-23 15:42:41 UTC

Description Siddhesh Poyarekar 2011-03-01 11:10:59 UTC
Created attachment 481587 [details]
patch to speed up map encoding

Description of problem:
The current implementation of map encoding into amqp in python is very slow due to the fact that the dictionary is traversed in a simple for loop. Attached patch replaces the for loop with the map function to get significant performance improvement.

Attached test program as well as statistics with and without the patch.

Version-Release number of selected component (if applicable):
python-qpid-0.7.946106-14.el5

How reproducible:
Always

Steps to Reproduce:
1. Run the responder:

./qpid-map-test responder

2. Run the script which makes multiple requests:

./qpid-requestor

3. Wait for requestor to finish
  
Actual results:

The responder will log message size and time taken in milliseconds to send the message.

Comment 1 Siddhesh Poyarekar 2011-03-01 11:14:23 UTC
Created attachment 481589 [details]
python program that acts as requestor as well as responder

Comment 2 Siddhesh Poyarekar 2011-03-01 11:16:05 UTC
Created attachment 481591 [details]
statistics before the patch

Comment 3 Siddhesh Poyarekar 2011-03-01 11:16:36 UTC
Created attachment 481592 [details]
statistics after the patch

Comment 4 Siddhesh Poyarekar 2011-03-01 11:17:22 UTC
Created attachment 481593 [details]
script to invoke the test program in requestor mode multiple times

Comment 6 Siddhesh Poyarekar 2011-03-01 16:01:44 UTC
There's a similar problem with map decode as well. I can't see any simple way to fix that though.

Comment 8 Petra Svobodová 2011-05-19 13:45:00 UTC
The map encoding is faster in the new version, which proves the performance improvement. 

Comparision of results - time of 5000 (before modification) / time of 5000 (after modification):
- in attached files about 6.6
- in my testing machines about 4

Verified on RHEL 5.6 / 6.1rc i[36]86 / x86_64 and packages: 
[root@dhcp-37-202 bug681166]# rpm -qa | grep qpid
qpid-cpp-client-0.10-7.el5
qpid-cpp-client-devel-0.10-7.el5
qpid-cpp-server-store-0.10-7.el5
qpid-java-client-0.10-6.el5
python-qpid-qmf-0.10-8.el5
qpid-qmf-0.10-8.el5
qpid-cpp-client-ssl-0.10-7.el5
qpid-cpp-server-cluster-0.10-7.el5
qpid-java-common-0.10-6.el5
qpid-java-example-0.10-6.el5
qpid-qmf-devel-0.10-8.el5
qpid-tools-0.10-5.el5
qpid-cpp-server-0.10-7.el5
qpid-cpp-server-ssl-0.10-7.el5
qpid-cpp-server-xml-0.10-7.el5
python-qpid-0.10-1.el5
qpid-cpp-server-devel-0.10-7.el5
qpid-cpp-client-devel-docs-0.10-7.el5

--> VERIFIED

Comment 9 errata-xmlrpc 2011-06-23 15:46:17 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHEA-2011-0890.html


Note You need to log in before you can comment on or make changes to this bug.