Bug 1273257 - Bugzilla job queue does not release pgpool connections
Summary: Bugzilla job queue does not release pgpool connections
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Bugzilla
Classification: Community
Component: Internal Tools
Version: 4.4
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: 4.4
Assignee: PnT DevOps Devs
QA Contact: tools-bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-10-20 04:54 UTC by Mark Keir
Modified: 2018-12-09 06:29 UTC (History)
2 users (show)

Fixed In Version: 4.4.10045.3
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-12-16 02:13:39 UTC
Embargoed:


Attachments (Terms of Use)

Description Mark Keir 2015-10-20 04:54:01 UTC
Description of problem:

The "jobqueue.pl" processes associated with the Bugzilla queue do not release the pgpool frontend connections cleanly on termination.

Version-Release number of selected component (if applicable):


How reproducible:

Have a running "bugzilla-queue" service attaching to a pgpool load balanced postgresql cluster

Steps to Reproduce:
1. jobqueue.pl is running
2. kill -9 {pid of jobqueue.pl}
3. alternate step: service bugzilla-queue stop;service bugzilla-queue start

Actual results:

when process is killed, a connection will be left on the pgpool VIP in a "DISCARD" state which can be show from "ps -efl".  This connection can be reaped by killing the associated process left running on the pgpool frontend server.

alternate step: when cleanly stopped, there are no stray processes left on the pgpool server.  On service restart, a new batch of connections for the bugzilla-queue service are claimed, 12 queues x 2 servers is 24 new connections reaped.

Expected results:

in the alternate step, on clean severance, it would be expected that the connections are released and not result in a step function consumption of the connection pool.

Additional info:

Comment 1 Rony Gong 🔥 2015-11-05 07:40:13 UTC
Test this on build 4.4.10043-3, Fail:

Firstly, make sure all all job queues are running
1.kill -9 {pid of jobqueue.pl}
2.Check in the pgpool server, find DISCARD

[root@bugzilla-qe-02 /]# ps -efl |grep -i  pool|grep -i bugzilla-qe-06 
 pgpool: bugs bugs bugzilla-qe-06.host.stage.eng.rdu2.redhat.com(39109) DISCARD

3.Then execute: service bugzilla-queue stop;service bugzilla-queue start
in the web server side:

4:Check in the pgool side, find 25 connections from webserver 06, 
The Discard still keep there.
[root@bugzilla-qe-02 /]# ps -efl |grep -i  pool|grep -i bugzilla-qe-06 |wc -l
25

[root@bugzilla-qe-02 /]# ps -efl |grep -i  pool|grep -i bugzilla-qe-06
5 S root     19603 11724  0  80   0 - 62326 poll_s 07:13 ?        00:00:00 pgpool: bugs bugs bugzilla-qe-06.host.stage.eng.rdu2.redhat.com(39109) DISCARD

5.Wait 2 mins, 12 connections lost, butt he Discard still keep there.
[root@bugzilla-qe-02 /]# ps -efl |grep -i  pool|grep -i bugzilla-qe-06 |wc -l
13

Comment 2 Jeff Fearn 🐞 2015-11-05 21:51:30 UTC
(In reply to Rony Gong from comment #1)
> 4:Check in the pgool side, find 25 connections from webserver 06, 
> The Discard still keep there.

This is not what this patch does, it is impossible for any change in BZ to change the state PGPool is in.

This patch stops the discard state from being created by running `service bugzilla-queue stop`.

Comment 3 Rony Gong 🔥 2015-11-09 01:43:11 UTC
(In reply to Jeff Fearn from comment #2)
> (In reply to Rony Gong from comment #1)
> > 4:Check in the pgool side, find 25 connections from webserver 06, 
> > The Discard still keep there.
> 
> This is not what this patch does, it is impossible for any change in BZ to
> change the state PGPool is in.
> 
> This patch stops the discard state from being created by running `service
> bugzilla-queue stop`.

As Dev described, this bug fixed now.

Comment 4 Matt Tyson 🤬 2015-12-16 02:13:39 UTC
This change is now live. If there are any issues, do not reopen this bug. Instead, you should create a new bug and reference this bug.


Note You need to log in before you can comment on or make changes to this bug.