Bug 591139 - RDMA client shutdown is broken (client hangs)
RDMA client shutdown is broken (client hangs)
Status: CLOSED ERRATA
Product: Red Hat Enterprise MRG
Classification: Red Hat
Component: qpid-cpp (Show other bugs)
beta
All Linux
urgent Severity high
: 1.3
: ---
Assigned To: Andrew Stitcher
Jan Sarenik
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2010-05-11 10:21 EDT by Andrew Stitcher
Modified: 2011-08-12 12:22 EDT (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Andrew Stitcher 2010-05-11 10:21:47 EDT
Description of problem:

Since the changes to the client shutdown in trunk r934503 rdma the client shutdown() callback will no longer be called when clients close an rdma connection.

This results in the client hanging whilst waiting for rdma connections to be completely closed.

How reproducible:

100%

Steps to Reproduce:
1. run "qpidd --auth no" (you will need to have a working rdma plugin)

To make sure that rdma is running, check that there is a line like:
    2010-05-10 17:55:04 notice Rdma: Listening on RDMA port 5672
in the output.

2. run perftest -Prdma -b <IP address of IB interface> 

3. Wait forever for exit after the test completes and print results.
Comment 1 Andrew Stitcher 2010-06-14 11:25:35 EDT
This is fixed as of r954499 upstream
Comment 4 Jan Sarenik 2010-06-30 09:50:21 EDT
Verified on RHEL5 x86_64
  qpid-cpp-server-rdma-0.7.946106-4.el5
  qpid-cpp-client-rdma-0.7.946106-4.el5
Comment 5 Jan Sarenik 2010-06-30 10:23:51 EDT
But I am unable to reproduce described bug.

It is strange that I get the same (good, no hang) results on version
which should be buggy:

bash-3.2# rpm -qa | grep qpid
qpidd-0.5.752581-34.el5
qpid-cpp-client-rdma-0.7.946106-2.el5
qpid-cpp-server-rdma-0.7.946106-2.el5
qpid-cpp-client-0.7.946106-2.el5
qpid-cpp-server-0.7.946106-2.el5
qpid-cpp-client-devel-0.7.946106-2.el5
bash-3.2# qpid-perftest -Prdma -b 192.168.55.26
Processing 1 messages from qpid-perftest_sub_ready . done.
Sending start 1 times to qpid-perftest_pub_start
Processing 1 messages from qpid-perftest_pub_done . done.
Processing 1 messages from qpid-perftest_sub_done . done.
... [SNIP] ...
Total transfers/sec:      122993
Total Mbytes/sec: 120.111
bash-3.2#
Comment 6 Andrew Stitcher 2010-07-04 16:55:44 EDT
I think that -2 had most of the fix in and you should try to reproduce with -1
Comment 7 Jan Sarenik 2010-07-07 04:05:36 EDT
So I have tried
  qpid-cpp-mrg-0.7.946106-1.el5
  qpid-cpp-mrg-0.7.935473-1.el5
  qpid-cpp-mrg-0.7.929717-1.el5

none of above reproduce the bug how you describe it.
I am sure I use RDMA as I get troughput ~136 MB/s.
May the bug be connected with something else (libibverbs,
particular IB driver, etc.)? I run the broker without
store or acl, focusing on RDMA.

On which machine did you experience the reported bug?
What hardware did you use?

I use mrg26.lab.bos.redhat.com, which has (by lscpi)
Mellanox Technologies MT25204 [InfiniHost III Lx HCA]
i.e. the package containing its driver is libmthca.
Comment 8 Jan Sarenik 2010-07-07 04:11:29 EDT
More info: The machine (mrg26) I tried to reproduce this bug on
was running RHEL5 x86_64 with following MRG packages installed:
  qpid-cpp-client-devel.x86_64
  qpid-cpp-client-rdma.x86_64
  qpid-cpp-client.x86_64
  qpid-cpp-server-rdma.x86_64
  qpid-cpp-server.x86_64
Comment 9 Jan Sarenik 2010-07-22 09:53:17 EDT
Though the bug is not reproducible, I am sure the described
bug is no longer valid in current versions. --> VERIFIED

Note You need to log in before you can comment on or make changes to this bug.