Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 489315

Summary: perftest shutdown seems to be not clean 'Error in shutdown: Connection closed'
Product: Red Hat Enterprise MRG Reporter: Frantisek Reznicek <freznice>
Component: qpid-cppAssignee: Gordon Sim <gsim>
Status: CLOSED ERRATA QA Contact: Jan Sarenik <jsarenik>
Severity: low Docs Contact:
Priority: low    
Version: 1.1CC: esammons, gsim, jsarenik, mgoulish
Target Milestone: 1.3   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
A shutdown performed by "src/tests/perftest.cpp" closed a connection without checking whether it was still open. This produced the following error: "Error in shutdown: Connection closed". With this update, 'perftest' checks whether a connection is still open. If so, 'perftest' closes the connection and the session.
 Story Points: ---
Clone Of: Environment:
Last Closed: 2010-10-14 15:57:15 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Test script used by bug reporter
none
Script none

Description Frantisek Reznicek 2009-03-09 13:22:01 UTC 
Description of problem:
The perftest c++ test client ends with following messages when closing because of heartbeat lost:
...
2009-mar-09 09:14:57 warning Connection closed
PublishThread exception: Connection closed
2009-mar-09 09:14:57 warning Connection closed
SubscribeThread exception: Connection closed
Error in shutdown: Connection closed
Error in shutdown: Connection closed

The problem was observed on both RHEL4.7 and RHEL5.3 i386 / x86_64

Version-Release number of selected component (if applicable):
qpidd-0.4.750054-1.el5, rhm-0.4.3138-2.el5, qpidc-perftest-0.4.750054-1.el5


How reproducible:
100%

Steps to Reproduce:
See transcript below...
  
Actual results:
Quite strange messages in perftest logfile

Expected results:
Less strange messages there

Additional info:
[root@ibm-firefly bz461932]# # part A
[root@ibm-firefly bz461932]#
[root@ibm-firefly bz461932]# rm -rf *.log data/ *.ecode
[root@ibm-firefly bz461932]#
[root@ibm-firefly bz461932]#
[root@ibm-firefly bz461932]# ( qpidd --auth no --log-enable warning+ --data-dir data >qpidd.log 2>&1 ; \
>   echo "qpidd ecode:$?" > qpidd.ecode ) &
[1] 27692
[root@ibm-firefly bz461932]#
[root@ibm-firefly bz461932]# sleep 5
[root@ibm-firefly bz461932]#
[root@ibm-firefly bz461932]# ( perftest --heartbeat 2 --log-enable warning+ >perftest.log 2>&1  ; \
>   echo "perftest ecode:$?" > perftest.ecode ) &
[2] 27703
[root@ibm-firefly bz461932]#
[root@ibm-firefly bz461932]# sleep 1
[root@ibm-firefly bz461932]#
[root@ibm-firefly bz461932]# qpidd_pid=$(netstat -nlp | grep qpidd | awk '{print $(NF)}' | awk -F/ '{print $1}')
[root@ibm-firefly bz461932]# perftest_pid=$(ps aux | grep perftest | grep -v grep | awk '{print $2}')
[root@ibm-firefly bz461932]#
[root@ibm-firefly bz461932]# if [ -n "${qpidd_pid}" -a -n "${perftest_pid}" ]; then
>   echo "1st:qpidd&perftest running"
> else
>   echo "ERROR:1st:qpidd&perftest running"
> fi
1st:qpidd&perftest running
[root@ibm-firefly bz461932]#
[root@ibm-firefly bz461932]#
[root@ibm-firefly bz461932]# sleep 4
[root@ibm-firefly bz461932]#
[root@ibm-firefly bz461932]# qpidd_pid=$(netstat -nlp | grep qpidd | awk '{print $(NF)}' | awk -F/ '{print $1}')
[root@ibm-firefly bz461932]# perftest_pid=$(ps aux | grep perftest | grep -v grep | awk '{print $2}')
[root@ibm-firefly bz461932]#
[root@ibm-firefly bz461932]# if [ -n "${qpidd_pid}" -a -n "${perftest_pid}" ]; then
>   echo "2nd:qpidd&perftest running"
> else
>   echo "ERROR:2nd:qpidd&perftest running"
> fi
2nd:qpidd&perftest running
[root@ibm-firefly bz461932]#
[root@ibm-firefly bz461932]#
[root@ibm-firefly bz461932]# # stop qpidd
[root@ibm-firefly bz461932]# echo "stop qpidd"
stop qpidd
[root@ibm-firefly bz461932]# kill -STOP ${qpidd_pid}
[root@ibm-firefly bz461932]#
[root@ibm-firefly bz461932]# qpidd_pid=$(netstat -nlp | grep qpidd | awk '{print $(NF)}' | awk -F/ '{print $1}')
[root@ibm-firefly bz461932]# perftest_pid=$(ps aux | grep perftest | grep -v grep | awk '{print $2}')
[root@ibm-firefly bz461932]#
[root@ibm-firefly bz461932]# if [ -n "${qpidd_pid}" -a -n "${perftest_pid}" ]; then
>   echo "3rd:qpidd&perftest running"
> else
>   echo "ERROR:3th:qpidd&perftest running"
> fi
3rd:qpidd&perftest running
[root@ibm-firefly bz461932]#
[root@ibm-firefly bz461932]# sleep 2
[root@ibm-firefly bz461932]#
[root@ibm-firefly bz461932]# qpidd_pid=$(netstat -nlp | grep qpidd | awk '{print $(NF)}' | awk -F/ '{print $1}')
[root@ibm-firefly bz461932]# perftest_pid=$(ps aux | grep perftest | grep -v grep | awk '{print $2}')
[root@ibm-firefly bz461932]#
[root@ibm-firefly bz461932]# if [ -n "${qpidd_pid}" -a -n "${perftest_pid}" ]; then
>   echo "4th:qpidd&perftest running"
> else
>   echo "ERROR:4th:qpidd&perftest running"
> fi
4th:qpidd&perftest running
[root@ibm-firefly bz461932]#
[root@ibm-firefly bz461932]# sleep 2
[2]+  Done                    ( perftest --heartbeat 2 --log-enable warning+ > perftest.log 2>&1; echo "perftest ecode:$?" > perftest.ecode )
[root@ibm-firefly bz461932]#
[root@ibm-firefly bz461932]# qpidd_pid=$(netstat -nlp | grep qpidd | awk '{print $(NF)}' | awk -F/ '{print $1}')
[root@ibm-firefly bz461932]# perftest_pid=$(ps aux | grep perftest | grep -v grep | awk '{print $2}')
[root@ibm-firefly bz461932]#
[root@ibm-firefly bz461932]# if [ -n "${qpidd_pid}" -a -z "${perftest_pid}" ]; then
>   echo "5th:qpidd running, perftest dead"
> else
>   echo "ERROR:5th:qpidd running, perftest dead"
> fi
5th:qpidd running, perftest dead
[root@ibm-firefly bz461932]#
[root@ibm-firefly bz461932]#
[root@ibm-firefly bz461932]# kill -CONT ${qpidd_pid}
[root@ibm-firefly bz461932]# kill ${qpidd_pid}
[root@ibm-firefly bz461932]#
[root@ibm-firefly bz461932]# sleep 2
[1]+  Done                    ( qpidd --auth no --log-enable warning+ --data-dir data > qpidd.log 2>&1; echo "qpidd ecode:$?" > qpidd.ecode )
[root@ibm-firefly bz461932]#
[root@ibm-firefly bz461932]# qpidd_pid=$(netstat -nlp | grep qpidd | awk '{print $(NF)}' | awk -F/ '{print $1}')
[root@ibm-firefly bz461932]# perftest_pid=$(ps aux | grep perftest | grep -v grep | awk '{print $2}')
[root@ibm-firefly bz461932]#
[root@ibm-firefly bz461932]# if [ -z "${qpidd_pid}" -a -z "${perftest_pid}" ]; then
>   echo "6th:qpidd&perftest dead"
> else
>   echo "ERROR:6th:qpidd&perftest dead"
> fi
6th:qpidd&perftest dead
[root@ibm-firefly bz461932]#
[root@ibm-firefly bz461932]#
[root@ibm-firefly bz461932]# cat *.ecode
perftest ecode:0
qpidd ecode:0
[root@ibm-firefly bz461932]#
[root@ibm-firefly bz461932]# cat perftest.log
Processing 1 messages from sub_ready . done.
Sending start 1 times to pub_start
Processing 1 messages from pub_done .2009-mar-09 09:14:56 warning Connection closed
Controller exception: Connection closed
Error in shutdown: Connection closed
2009-mar-09 09:14:57 warning Connection closed
PublishThread exception: Connection closed
2009-mar-09 09:14:57 warning Connection closed
SubscribeThread exception: Connection closed
Error in shutdown: Connection closed
Error in shutdown: Connection closed
Comment 1 Gordon Sim 2009-03-12 17:06:08 UTC 
Fixed by r752937.
Comment 4 Jan Sarenik 2009-09-07 09:54:28 UTC 
It would help if the current packages were built into the devel repo.
Comment 6 Jan Sarenik 2010-02-12 14:51:28 UTC 
Created attachment 390500 [details]
Test script used by bug reporter
Comment 7 Jan Sarenik 2010-02-12 15:19:29 UTC 
After running the test script on current -candidate qpidd on RHEL5
(there are still no current qpidc builds for RHEL4)
this is the content of perftest.log:

----- 
Processing 1 messages from perftest_sub_ready . done.
Sending start 1 times to perftest_pub_start
Processing 1 messages from perftest_pub_done .2010-02-12 15:52:28 warning Connection closed
PublishThread exception: Connection closed
2010-02-12 15:52:28 warning Connection closed
Controller exception: Connection closed
2010-02-12 15:52:29 warning Connection closed
SubscribeThread exception: Connection closed
----- 

So there are no more lines containing shutdown errors.

qpidc 0.7.908272-1.el5  both i686 and x86_64

I will test RHEL4 as soon as packages arrive in dist-4E-mrg-candidate repo.
Comment 9 Jan Sarenik 2010-03-04 16:40:46 UTC 
Created attachment 397857 [details]
Script

RHEL4 (i386, x86_64):
  qpid-cpp-server-0.7.916826-2.el4
  qpid-cpp-client-devel-0.7.916826-2.el4
Comment 10 mick 2010-10-07 19:54:02 UTC 
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
Cause:
    Client dtor in src/tests/perftest.cpp closed connection without first making sure that it was still open.

  Consequence:
    'Error in shutdown: Connection closed' reported by perftest shutdown.

  Fix:
    Check that connection is still open.  If so, close it and the session.

  Result:
    Fewer weird error messages when perftest is shut down.
Comment 11 Martin Prpič 2010-10-10 11:50:50 UTC 
    Technical note updated. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    Diffed Contents:
@@ -1,11 +1 @@
-Cause:
+A shutdown performed by "src/tests/perftest.cpp" closed a connection without checking whether it was still open. This produced the following error: "Error in shutdown: Connection closed". With this update, 'perftest' checks whether a connection is still open. If so, 'perftest' closes the connection and the session.-    Client dtor in src/tests/perftest.cpp closed connection without first making sure that it was still open.
-
-  Consequence:
-    'Error in shutdown: Connection closed' reported by perftest shutdown.
-
-  Fix:
-    Check that connection is still open.  If so, close it and the session.
-
-  Result:
-    Fewer weird error messages when perftest is shut down.
Comment 13 errata-xmlrpc 2010-10-14 15:57:15 UTC 
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2010-0773.html