Bug 1278612 - specialised queue workers ignore configuration and handle all tasks instead
Summary: specialised queue workers ignore configuration and handle all tasks instead
Keywords:
Status: CLOSED WORKSFORME
Alias: None
Product: Bugzilla
Classification: Community
Component: Bugzilla General
Version: 4.4
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: PnT DevOps Devs
QA Contact: tools-bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-11-06 01:08 UTC by Sage Grigull
Modified: 2015-11-11 04:04 UTC (History)
3 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2015-11-11 04:04:40 UTC
Embargoed:


Attachments (Terms of Use)

Description Sage Grigull 2015-11-06 01:08:06 UTC
Description of problem:
queue-workers appear to be ignoring configuration into dedicated task roles.

Version-Release number of selected component (if applicable):
4.4.10042-7

How reproducible:
assumed 100%

Steps to Reproduce:
1. disable all bugzilla-queue workers
2. enable only a worker that has been configured to handle a dedicated tasks.  e.g. jira
3. observe tasks on all queues decrease

Actual results:
all job types are handled by specialized, dedicated worker

Expected results:
only job types matching configuration are handled by specialized, dedicated workers

Additional info:

Comment 1 Jason McDonald 2015-11-06 01:22:01 UTC
There was a recent change on production to dispatch emails directly rather than put them through the job queue.  That change was made to clear a big backlog of email jobs, but is still in place.

If that change is going to become permanent, I would suggest reducing the number of job queue processors (to say 4 or 6 per web server) and removing the queue specialization mechanism, which was originally implemented to stop backlogged rules engine jobs from blocking email jobs (which can't happen if the emails aren't going through the job queue).

I estimate that about 90% of the job queue throughput used to be email jobs. Reducing the number of queues by 1/2 or 2/3 would therefore be quite safe, would reduce the load on the database when the Rules Engine kill switch is set, and would reduce the number of pgpool connections permanently consumed by the job queues.

Comment 2 Sage Grigull 2015-11-06 01:26:49 UTC
Unfortunately I have forgotten the specifics; I do recall that other queue types besides mail were being handled when they shouldn't have.

Comment 3 Jeff Fearn 🐞 2015-11-06 02:09:42 UTC
IIRC The issue was that non-mail queues where actually processing mail jobs.

Comment 4 Sage Grigull 2015-11-06 03:25:30 UTC
Given I cannot effectively replicate this issuer at this time, I am happy for this issue to be closed for now.

I can clone it if it pops up again.


Note You need to log in before you can comment on or make changes to this bug.