Bug 1296181 - Bugzilla rule engine - kill switch activated due to deadlock in DB
Summary: Bugzilla rule engine - kill switch activated due to deadlock in DB
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Bugzilla
Classification: Community
Component: Internal Tools
Version: 4.4
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: 4.4
Assignee: Matt Tyson 🤬
QA Contact: tools-bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-01-06 14:27 UTC by Libor Miksik
Modified: 2018-12-09 06:29 UTC (History)
5 users (show)

Fixed In Version: 4.4.11048.2
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-02-15 00:57:23 UTC
Embargoed:


Attachments (Terms of Use)

Description Libor Miksik 2016-01-06 14:27:23 UTC
kill switch was automatically activated with following error message:
Please for investigation what was wrong. I've disabled kill switch to unblock all the rules.

Thanks!


DBD::Pg::db do failed: ERROR: deadlock detected DETAIL: Process 11342 waits for ShareLock on transaction 44851708; blocked by process 11386. Process 11386 waits for ExclusiveLock on tuple (56238,1) of relation 527931 of database 16384; blocked by process 11342. HINT: See server log for query details. [for Statement "UPDATE bugs SET delta_ts = ? WHERE bug_id = ?"] at Bugzilla/Bug.pm line 1318 Bugzilla::Bug::update('Bugzilla::Bug=HASH(0x6ff4c78)') called at /loader/0x5c0e818/Bugzilla/Extension/RuleEngine/Job.pm line 1006 Bugzilla::Extension::RuleEngine::Job::__ANON__() called at /usr/share/perl5/vendor_perl/Try/Tiny.pm line 76 eval {...} called at /usr/share/perl5/vendor_perl/Try/Tiny.pm line 67 Try::Tiny::try('CODE(0x6dc9518)', 'Try::Tiny::Catch=REF(0x4b4ab08)') called at /loader/0x5c0e818/Bugzilla/Extension/RuleEngine/Job.pm line 1024 Bugzilla::Extension::RuleEngine::Job::_rule_do_action('Bugzilla::Extension::RuleEngine::Job', 'Bugzilla::Bug=HASH(0x6ff4c78)', 'Bugzilla::Extension::RuleEngine::Rule=HASH(0x6c62a10)', 'ARRAY(0x5cec860)') called at /loader/0x5c0e818/Bugzilla/Extension/RuleEngine/Job.pm line 297 Bugzilla::Extension::RuleEngine::Job::do_job_on_bug('Bugzilla::Extension::RuleEngine::Job', 'Bugzilla::Bug=HASH(0x6ff4c78)', 'ARRAY(0x5cec128)') called at /loader/0x5c0e818/Bugzilla/Extension/RuleEngine/Job.pm line 224 Bugzilla::Extension::RuleEngine::Job::do_job('Bugzilla::Extension::RuleEngine::Job', 'HASH(0x6a79d98)') called at /loader/0x5c0e818/Bugzilla/Extension/RuleEngine/Job.pm line 141 eval {...} called at /loader/0x5c0e818/Bugzilla/Extension/RuleEngine/Job.pm line 136 Bugzilla::Extension::RuleEngine::Job::work('Bugzilla::Extension::RuleEngine::Job', 'TheSchwartz::Job=HASH(0x457fba8)') called at /usr/local/share/perl5/TheSchwartz/Worker.pm line 29 eval {...} called at /usr/local/share/perl5/TheSchwartz/Worker.pm line 29 TheSchwartz::Worker::work_safely('Bugzilla::Extension::RuleEngine::Job', 'TheSchwartz::Job=HASH(0x457fba8)') called at /usr/local/share/perl5/TheSchwartz.pm line 678 TheSchwartz::work_once('Bugzilla::JobQueue=HASH(0x64dce10)') called at Bugzilla/JobQueue.pm line 176 Bugzilla::JobQueue::work_once('Bugzilla::JobQueue=HASH(0x64dce10)') called at /usr/local/share/perl5/TheSchwartz.pm line 634 TheSchwartz::work_until_done('Bugzilla::JobQueue=HASH(0x64dce10)') called at Bugzilla/JobQueue/Runner.pm line 229 eval {...} called at Bugzilla/JobQueue/Runner.pm line 226 Bugzilla::JobQueue::Runner::_do_work('Bugzilla::JobQueue::Runner=HASH(0x25d3f40)', 'work_until_done') called at Bugzilla/JobQueue/Runner.pm line 207 Bugzilla::JobQueue::Runner::gd_run('Bugzilla::JobQueue::Runner=HASH(0x25d3f40)') called at /usr/share/perl5/vendor_perl/Daemon/Generic.pm line 190 Daemon::Generic::new('Bugzilla::JobQueue::Runner') called at /var/www/html/bugzilla/jobqueue.pl line 23

Comment 1 Matt Tyson 🤬 2016-01-08 05:36:59 UTC
Bugzilla should be detecting and retrying later since this error is transient.

It seems the code isn't properly detecting the message.  I've tweaked it a bit which should stop this from re-occurring.

Comment 3 Hui Wang 2016-01-25 10:11:12 UTC
I have tried to reproduce the issue in comment#0, but failed.
Change the status to verified at that moment. If you can reproduce this issue, pls reopen this issue and pls help to give the reproduceable scenario.

Comment 5 Matt Tyson 🤬 2016-02-15 00:57:23 UTC
This change is now live. If there are any issues, do not reopen this bug.
Instead, you should create a new bug and reference this bug.


Note You need to log in before you can comment on or make changes to this bug.