Red Hat Bugzilla – Bug 489315
perftest shutdown seems to be not clean 'Error in shutdown: Connection closed'
Last modified: 2015-11-15 19:07:00 EST
Description of problem: The perftest c++ test client ends with following messages when closing because of heartbeat lost: ... 2009-mar-09 09:14:57 warning Connection closed PublishThread exception: Connection closed 2009-mar-09 09:14:57 warning Connection closed SubscribeThread exception: Connection closed Error in shutdown: Connection closed Error in shutdown: Connection closed The problem was observed on both RHEL4.7 and RHEL5.3 i386 / x86_64 Version-Release number of selected component (if applicable): qpidd-0.4.750054-1.el5, rhm-0.4.3138-2.el5, qpidc-perftest-0.4.750054-1.el5 How reproducible: 100% Steps to Reproduce: See transcript below... Actual results: Quite strange messages in perftest logfile Expected results: Less strange messages there Additional info: [root@ibm-firefly bz461932]# # part A [root@ibm-firefly bz461932]# [root@ibm-firefly bz461932]# rm -rf *.log data/ *.ecode [root@ibm-firefly bz461932]# [root@ibm-firefly bz461932]# [root@ibm-firefly bz461932]# ( qpidd --auth no --log-enable warning+ --data-dir data >qpidd.log 2>&1 ; \ > echo "qpidd ecode:$?" > qpidd.ecode ) & [1] 27692 [root@ibm-firefly bz461932]# [root@ibm-firefly bz461932]# sleep 5 [root@ibm-firefly bz461932]# [root@ibm-firefly bz461932]# ( perftest --heartbeat 2 --log-enable warning+ >perftest.log 2>&1 ; \ > echo "perftest ecode:$?" > perftest.ecode ) & [2] 27703 [root@ibm-firefly bz461932]# [root@ibm-firefly bz461932]# sleep 1 [root@ibm-firefly bz461932]# [root@ibm-firefly bz461932]# qpidd_pid=$(netstat -nlp | grep qpidd | awk '{print $(NF)}' | awk -F/ '{print $1}') [root@ibm-firefly bz461932]# perftest_pid=$(ps aux | grep perftest | grep -v grep | awk '{print $2}') [root@ibm-firefly bz461932]# [root@ibm-firefly bz461932]# if [ -n "${qpidd_pid}" -a -n "${perftest_pid}" ]; then > echo "1st:qpidd&perftest running" > else > echo "ERROR:1st:qpidd&perftest running" > fi 1st:qpidd&perftest running [root@ibm-firefly bz461932]# [root@ibm-firefly bz461932]# [root@ibm-firefly bz461932]# sleep 4 [root@ibm-firefly bz461932]# [root@ibm-firefly bz461932]# qpidd_pid=$(netstat -nlp | grep qpidd | awk '{print $(NF)}' | awk -F/ '{print $1}') [root@ibm-firefly bz461932]# perftest_pid=$(ps aux | grep perftest | grep -v grep | awk '{print $2}') [root@ibm-firefly bz461932]# [root@ibm-firefly bz461932]# if [ -n "${qpidd_pid}" -a -n "${perftest_pid}" ]; then > echo "2nd:qpidd&perftest running" > else > echo "ERROR:2nd:qpidd&perftest running" > fi 2nd:qpidd&perftest running [root@ibm-firefly bz461932]# [root@ibm-firefly bz461932]# [root@ibm-firefly bz461932]# # stop qpidd [root@ibm-firefly bz461932]# echo "stop qpidd" stop qpidd [root@ibm-firefly bz461932]# kill -STOP ${qpidd_pid} [root@ibm-firefly bz461932]# [root@ibm-firefly bz461932]# qpidd_pid=$(netstat -nlp | grep qpidd | awk '{print $(NF)}' | awk -F/ '{print $1}') [root@ibm-firefly bz461932]# perftest_pid=$(ps aux | grep perftest | grep -v grep | awk '{print $2}') [root@ibm-firefly bz461932]# [root@ibm-firefly bz461932]# if [ -n "${qpidd_pid}" -a -n "${perftest_pid}" ]; then > echo "3rd:qpidd&perftest running" > else > echo "ERROR:3th:qpidd&perftest running" > fi 3rd:qpidd&perftest running [root@ibm-firefly bz461932]# [root@ibm-firefly bz461932]# sleep 2 [root@ibm-firefly bz461932]# [root@ibm-firefly bz461932]# qpidd_pid=$(netstat -nlp | grep qpidd | awk '{print $(NF)}' | awk -F/ '{print $1}') [root@ibm-firefly bz461932]# perftest_pid=$(ps aux | grep perftest | grep -v grep | awk '{print $2}') [root@ibm-firefly bz461932]# [root@ibm-firefly bz461932]# if [ -n "${qpidd_pid}" -a -n "${perftest_pid}" ]; then > echo "4th:qpidd&perftest running" > else > echo "ERROR:4th:qpidd&perftest running" > fi 4th:qpidd&perftest running [root@ibm-firefly bz461932]# [root@ibm-firefly bz461932]# sleep 2 [2]+ Done ( perftest --heartbeat 2 --log-enable warning+ > perftest.log 2>&1; echo "perftest ecode:$?" > perftest.ecode ) [root@ibm-firefly bz461932]# [root@ibm-firefly bz461932]# qpidd_pid=$(netstat -nlp | grep qpidd | awk '{print $(NF)}' | awk -F/ '{print $1}') [root@ibm-firefly bz461932]# perftest_pid=$(ps aux | grep perftest | grep -v grep | awk '{print $2}') [root@ibm-firefly bz461932]# [root@ibm-firefly bz461932]# if [ -n "${qpidd_pid}" -a -z "${perftest_pid}" ]; then > echo "5th:qpidd running, perftest dead" > else > echo "ERROR:5th:qpidd running, perftest dead" > fi 5th:qpidd running, perftest dead [root@ibm-firefly bz461932]# [root@ibm-firefly bz461932]# [root@ibm-firefly bz461932]# kill -CONT ${qpidd_pid} [root@ibm-firefly bz461932]# kill ${qpidd_pid} [root@ibm-firefly bz461932]# [root@ibm-firefly bz461932]# sleep 2 [1]+ Done ( qpidd --auth no --log-enable warning+ --data-dir data > qpidd.log 2>&1; echo "qpidd ecode:$?" > qpidd.ecode ) [root@ibm-firefly bz461932]# [root@ibm-firefly bz461932]# qpidd_pid=$(netstat -nlp | grep qpidd | awk '{print $(NF)}' | awk -F/ '{print $1}') [root@ibm-firefly bz461932]# perftest_pid=$(ps aux | grep perftest | grep -v grep | awk '{print $2}') [root@ibm-firefly bz461932]# [root@ibm-firefly bz461932]# if [ -z "${qpidd_pid}" -a -z "${perftest_pid}" ]; then > echo "6th:qpidd&perftest dead" > else > echo "ERROR:6th:qpidd&perftest dead" > fi 6th:qpidd&perftest dead [root@ibm-firefly bz461932]# [root@ibm-firefly bz461932]# [root@ibm-firefly bz461932]# cat *.ecode perftest ecode:0 qpidd ecode:0 [root@ibm-firefly bz461932]# [root@ibm-firefly bz461932]# cat perftest.log Processing 1 messages from sub_ready . done. Sending start 1 times to pub_start Processing 1 messages from pub_done .2009-mar-09 09:14:56 warning Connection closed Controller exception: Connection closed Error in shutdown: Connection closed 2009-mar-09 09:14:57 warning Connection closed PublishThread exception: Connection closed 2009-mar-09 09:14:57 warning Connection closed SubscribeThread exception: Connection closed Error in shutdown: Connection closed Error in shutdown: Connection closed
Fixed by r752937.
It would help if the current packages were built into the devel repo.
Created attachment 390500 [details] Test script used by bug reporter
After running the test script on current -candidate qpidd on RHEL5 (there are still no current qpidc builds for RHEL4) this is the content of perftest.log: ----- Processing 1 messages from perftest_sub_ready . done. Sending start 1 times to perftest_pub_start Processing 1 messages from perftest_pub_done .2010-02-12 15:52:28 warning Connection closed PublishThread exception: Connection closed 2010-02-12 15:52:28 warning Connection closed Controller exception: Connection closed 2010-02-12 15:52:29 warning Connection closed SubscribeThread exception: Connection closed ----- So there are no more lines containing shutdown errors. qpidc 0.7.908272-1.el5 both i686 and x86_64 I will test RHEL4 as soon as packages arrive in dist-4E-mrg-candidate repo.
Created attachment 397857 [details] Script RHEL4 (i386, x86_64): qpid-cpp-server-0.7.916826-2.el4 qpid-cpp-client-devel-0.7.916826-2.el4
Technical note added. If any revisions are required, please edit the "Technical Notes" field accordingly. All revisions will be proofread by the Engineering Content Services team. New Contents: Cause: Client dtor in src/tests/perftest.cpp closed connection without first making sure that it was still open. Consequence: 'Error in shutdown: Connection closed' reported by perftest shutdown. Fix: Check that connection is still open. If so, close it and the session. Result: Fewer weird error messages when perftest is shut down.
Technical note updated. If any revisions are required, please edit the "Technical Notes" field accordingly. All revisions will be proofread by the Engineering Content Services team. Diffed Contents: @@ -1,11 +1 @@ -Cause: +A shutdown performed by "src/tests/perftest.cpp" closed a connection without checking whether it was still open. This produced the following error: "Error in shutdown: Connection closed". With this update, 'perftest' checks whether a connection is still open. If so, 'perftest' closes the connection and the session.- Client dtor in src/tests/perftest.cpp closed connection without first making sure that it was still open. - - Consequence: - 'Error in shutdown: Connection closed' reported by perftest shutdown. - - Fix: - Check that connection is still open. If so, close it and the session. - - Result: - Fewer weird error messages when perftest is shut down.
An advisory has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on therefore solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHSA-2010-0773.html