Bug 1249465 - [PgPool]Can't create bug when the jobs number got to 52474
Summary: [PgPool]Can't create bug when the jobs number got to 52474
Keywords:
Status: CLOSED WORKSFORME
Alias: None
Product: Bugzilla
Classification: Community
Component: Database
Version: 4.4
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: 4.4
Assignee: PnT DevOps Devs
QA Contact: Rony Gong 🔥
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-08-03 05:13 UTC by Rony Gong 🔥
Modified: 2018-12-09 06:29 UTC (History)
6 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2015-08-31 00:16:04 UTC
Embargoed:


Attachments (Terms of Use)

Description Rony Gong 🔥 2015-08-03 05:13:46 UTC
Description of problem:
[PgPool]Can't create bug when the jobs number got to 52474

Version-Release number of selected component (if applicable):
4.4.9038-3

How reproducible:
Always

Steps to Reproduce:
1.Stop all bugzilla-queue services
2.Then do a very long test against rdu pgpool environment, it will generate 52474 bugzilla jobs
3.Then try to create a new bug.

Actual results:
Page show error when create bug:

Inserting a TheSchwartz::Job=HASH(0x7f0e47eab978) job into the Job Queue failed with the following error: Failed to execute INSERT INTO ts_job (funcid, arg, uniqkey, insert_time, run_after, grabbed_until, priority, coalesce) VALUES (?, ?, ?, ?, ?, ?, ?, ?) with funcid, arg, uniqkey, insert_time, run_after, grabbed_until, priority, coalesce: DBD::Pg::st execute failed: ERROR: right sibling's left-link doesn't match: block 550 links to 1 instead of expected 555 in index "ts_job_funcid_idx" at /usr/share/perl5/vendor_perl/Data/ObjectDriver/Driver/DBI.pm line 396. 


Traceback:
 at /var/www/html/bugzilla/Bugzilla/JobQueue.pm line 98
	Bugzilla::JobQueue::insert(...) called at /var/www/html/bugzilla/Bugzilla/Mailer.pm line 49
	Bugzilla::Mailer::MessageToMTA(...) called at /var/www/html/bugzilla/Bugzilla/BugMail.pm line 461
	Bugzilla::BugMail::sendMail(...) called at /var/www/html/bugzilla/Bugzilla/BugMail.pm line 288
	Bugzilla::BugMail::Send(...) called at /var/www/html/bugzilla/post_bug.cgi line 259
	ModPerl::ROOT::Bugzilla::ModPerl::ResponseHandler::var_www_html_bugzilla_post_bug_2ecgi::handler(...) called at /usr/lib64/perl5/vendor_perl/ModPerl/RegistryCooker.pm line 204
	eval {...} called at /usr/lib64/perl5/vendor_perl/ModPerl/RegistryCooker.pm line 204
	ModPerl::RegistryCooker::run(...) called at /usr/lib64/perl5/vendor_perl/ModPerl/RegistryCooker.pm line 170
	ModPerl::RegistryCooker::default_handler(...) called at /usr/lib64/perl5/vendor_perl/ModPerl/Registry.pm line 31
	ModPerl::Registry::handler(...) called at /var/www/html/bugzilla/mod_perl.pl line 134
	Bugzilla::ModPerl::ResponseHandler::handler(...) called at -e line 0
	eval {...} called at -e line 0


Expected results:
52474 jobs in the queue

Additional info:

Comment 3 Jeff Fearn 🐞 2015-08-17 00:23:21 UTC
After fixing Bug 1250305, Bug 1251315, Bug 1251317, Bug 1252181, and Bug 1245411, we can not duplicate this. None of those bugs affect the DB directly and none should trigger this issue. It's possible some combination of them trigger the error but we can't prove or test that.

We really need a broken DB, or a reliable way of breaking the DB indexes, to test or triage this issue.

Comment 4 Matt Tyson 🤬 2015-08-30 23:54:21 UTC
The index corruption issue is either a bug in postgres, or a hardware fault.

It's not a problem caused by bugzilla or the job queues.

Comment 5 Jason McDonald 2015-08-31 00:16:04 UTC
This issue hasn't been seen for a few weeks now, and is very unlikely to be a problem with the Bugzilla codebase anyway, so let's close it until someone is able to reproduce it.


Note You need to log in before you can comment on or make changes to this bug.