Description of problem: When (exclusively) subscribing to a (nonexclusive) queue with another exclusive consumer, a small memory leak occurs. Version-Release number of selected component (if applicable): qpid-cpp-server-0.18-39.el6.x86_64 How reproducible: 100% Steps to Reproduce: 1. Create exclusive subscriber: qpid-receive -a "q; { link: { x-subscribe : { exclusive:true } }, create:always}" -f & 2. Attempt to subscribe to the same queue in a loop: while true; do qpid-receive -a "q; { link: { x-subscribe : { exclusive:true } }, create:always}" >/dev/null 2>&1; sleep 0.1; done 3. in another terminal, monitor memory: while true; do ps aux | grep qpidd | grep daemon | awk -F "--config" '{ print $1 }'; sleep 60; done Actual results: 3. shows small but constant memory usage growth Expected results: 3. to show no memory growth Additional info:
The leak is present also in qpid-cpp-server-0.34-15.el6.x86_64 but it is not present in current qpid upstream (sic!).
Long test showed the leak can be present also in upstream - ps shows little bit confusing data.. ps output (just RSS column) taken every minute from a test against upstream broker: 4168 8120 8164 8416 8444 8568 8576 8672 8708 8816 8852 8892 8916 8988 9040 9192 9272 9308 9484 9576 9636 9680 9812 9868 9940 9984 9980 10116 10164 10220 10304 10356 10428 10516 10528 10576 10648 10668 10732 10784 10844 10936 10992 11040 11188 11264 11324 11384 11456 11508 11540 11592 11640 11676 11788 11820 11884 11920 12000 12100 12152 12176 12272 12304 12364 12388 12460 12508 12556 12636 12692 12724 12772 12816 12896 12960 13044 13080 13164 13276 13392 13392 13400 13444 13452 13460 13416 13540 13548 13572 13568 13588 13588 13624 13660 13696 13696 13704 13724 13728 13724 13744 13772 13780 13800 13848 13844 13844 13848 13864 13912 13924 13932 14168 14176 14228 14232 14252 14256 14304 14332 14340 14376 14392 14444 14456 14480 14508 14528 14544 14568 14584 14612 14640 14648 14672 14712 14720 14740 14784 14800 14800 14840 14876 14912 14964 14972 14988 15140 15152 15196 15216 15240 15292 15356 15436 15448 15544 15576 15624 15664 15688 15740 15768 15832 15896 15904 15944 15964 15988 16004 16028 16068 16120 16056 16056 16060 16076 16088 16084 16088 16136 16136 16172 16172 16172 16172 16168 16172 16176 16184 16180 16180 16180 16180 16176 16176 16176 16176 16176 16172 16172 16200 16200 16200 16196 16196 16196 16196 16192 16192 16192 16192 16192 16192 16192 16188 16188 16212 16208 16232 16240 16244 16232 16288 16300 16300 16296 16296 16296 16296 16308 16308 16308 16324 16368 16372 16432 16436 16436 16448 16476 16476 16500 17860 17896 17896 17896 17900 17896 17900 17900 17916 17920 17932 17948 17260 17016 17012 17012 17048 17012 17012 17012 17008 17008 17008 17008 17008 17008 17004 17004 17004 17000 17000 17000 17000 17000 17000 17000 17004 17004 17004 17004 17004 17004 17000 16224 16224 16220 16220 16220 16216 16216 16216 16216 16212 16212 16212 16208 16208 16208 16208 16204 16204 16204 16204 16204 16204 16204 16200 16200 16200 16200 16196 16196 16244 16244 16240 16240 16240 16236 16372 16380 16384 16392 16400 16404 16408 16416 16348 16348 16348 16348 16344 16344 16344 16344 16340 16340 16340 16340 16340 16336 16336 16336 16336 16336 16336 16332 16332 16332 16332 16332 16332 16332 16332 16332 16328 16328 16328 16328 16328 16324 16324 16320 16368 16368 16368 16368 16364 16364 16364 16364 16360 16340 16340 16340 16340 16336 16336 16336 16336 16336 16332 16332
(In reply to Pavel Moravec from comment #2) > Long test showed the leak can be present also in upstream - ps shows little > bit confusing data.. > > ps output (just RSS column) taken every minute from a test against upstream > broker: > > 4168 > 8120 > 8164 > 8416 > 8444 > 8568 > 8576 > 8672 > 8708 > 8816 > 8852 > 8892 > 8916 > 8988 > 9040 > 9192 > 9272 > 9308 > 9484 > 9576 > 9636 > 9680 > 9812 > 9868 > 9940 > 9984 > 9980 > 10116 > 10164 > 10220 > 10304 > 10356 > 10428 > 10516 > 10528 > 10576 > 10648 > 10668 > 10732 > 10784 > 10844 > 10936 > 10992 > 11040 > 11188 > 11264 > 11324 > 11384 > 11456 > 11508 > 11540 > 11592 > 11640 > 11676 > 11788 > 11820 > 11884 > 11920 > 12000 > 12100 > 12152 > 12176 > 12272 > 12304 > 12364 > 12388 > 12460 > 12508 > 12556 > 12636 > 12692 > 12724 > 12772 > 12816 > 12896 > 12960 > 13044 > 13080 > 13164 > 13276 > 13392 > 13392 > 13400 > 13444 > 13452 > 13460 > 13416 > 13540 > 13548 > 13572 > 13568 > 13588 > 13588 > 13624 > 13660 > 13696 > 13696 > 13704 > 13724 > 13728 > 13724 > 13744 > 13772 > 13780 > 13800 > 13848 > 13844 > 13844 > 13848 > 13864 > 13912 > 13924 > 13932 > 14168 > 14176 > 14228 > 14232 > 14252 > 14256 > 14304 > 14332 > 14340 > 14376 > 14392 > 14444 > 14456 > 14480 > 14508 > 14528 > 14544 > 14568 > 14584 > 14612 > 14640 > 14648 > 14672 > 14712 > 14720 > 14740 > 14784 > 14800 > 14800 > 14840 > 14876 > 14912 > 14964 > 14972 > 14988 > 15140 > 15152 > 15196 > 15216 > 15240 > 15292 > 15356 > 15436 > 15448 > 15544 > 15576 > 15624 > 15664 > 15688 > 15740 > 15768 > 15832 > 15896 > 15904 > 15944 > 15964 > 15988 > 16004 > 16028 > 16068 > 16120 > 16056 > 16056 > 16060 > 16076 > 16088 > 16084 > 16088 > 16136 > 16136 > 16172 > 16172 > 16172 > 16172 > 16168 > 16172 > 16176 > 16184 > 16180 > 16180 > 16180 > 16180 > 16176 > 16176 > 16176 > 16176 > 16176 > 16172 > 16172 > 16200 > 16200 > 16200 > 16196 > 16196 > 16196 > 16196 > 16192 > 16192 > 16192 > 16192 > 16192 > 16192 > 16192 > 16188 > 16188 > 16212 > 16208 > 16232 > 16240 > 16244 > 16232 > 16288 > 16300 > 16300 > 16296 > 16296 > 16296 > 16296 > 16308 > 16308 > 16308 > 16324 > 16368 > 16372 > 16432 > 16436 > 16436 > 16448 > 16476 > 16476 > 16500 > 17860 > 17896 > 17896 > 17896 > 17900 > 17896 > 17900 > 17900 > 17916 > 17920 > 17932 > 17948 > 17260 > 17016 > 17012 > 17012 > 17048 > 17012 > 17012 > 17012 > 17008 > 17008 > 17008 > 17008 > 17008 > 17008 > 17004 > 17004 > 17004 > 17000 > 17000 > 17000 > 17000 > 17000 > 17000 > 17000 > 17004 > 17004 > 17004 > 17004 > 17004 > 17004 > 17000 > 16224 > 16224 > 16220 > 16220 > 16220 > 16216 > 16216 > 16216 > 16216 > 16212 > 16212 > 16212 > 16208 > 16208 > 16208 > 16208 > 16204 > 16204 > 16204 > 16204 > 16204 > 16204 > 16204 > 16200 > 16200 > 16200 > 16200 > 16196 > 16196 > 16244 > 16244 > 16240 > 16240 > 16240 > 16236 > 16372 > 16380 > 16384 > 16392 > 16400 > 16404 > 16408 > 16416 > 16348 > 16348 > 16348 > 16348 > 16344 > 16344 > 16344 > 16344 > 16340 > 16340 > 16340 > 16340 > 16340 > 16336 > 16336 > 16336 > 16336 > 16336 > 16336 > 16332 > 16332 > 16332 > 16332 > 16332 > 16332 > 16332 > 16332 > 16332 > 16328 > 16328 > 16328 > 16328 > 16328 > 16324 > 16324 > 16320 > 16368 > 16368 > 16368 > 16368 > 16364 > 16364 > 16364 > 16364 > 16360 > 16340 > 16340 > 16340 > 16340 > 16336 > 16336 > 16336 > 16336 > 16336 > 16332 > 16332 I'm not sure this is actually a leak. The last 200 values recorded for memory (i.e. 3 hours and 20 minutes worth) show a very small difference and the memory goes down as well as up. I see similar numbers just opening and closing a receiver - i.e. with no failed attempt at exclusive queue subscription. I think the initial growth may just be memory pressure from creating and deleting the various objects for connection, session & susbscription very rapidly. Once the available heap has expanded sufficiently, extra memory is not required. This may of course be different from the originally reported issue on -18 or -34.
https://issues.apache.org/jira/browse/QPID-7373 may be the same issue. Note that that bug depends solely on creating many sequential connections and is unrelated to exclusive access to a queue. Since my proof of concept fix for QPID-7373 seems to address the memory leak in the provided reproducer in my environment (RHEL6, 6core/12thread cpu), I am hopeful the problems may be the same.
Recommendation for QE: it is worth testing it with different number of worker threads. I.e. once with default number, then with 1 and also with say 10. (preliminary results of mine testing are the leak is fixed)
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2016-2049.html