Bug 1344153 - Serialization error on bug creation
Summary: Serialization error on bug creation
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Bugzilla
Classification: Community
Component: Database
Version: 4.4
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
Assignee: PnT DevOps Devs
QA Contact: tools-bugs
URL:
Whiteboard:
Depends On: 1610130
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-06-08 23:16 UTC by Matt Tyson 🤬
Modified: 2019-01-23 23:14 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-01-23 23:14:16 UTC
Embargoed:


Attachments (Terms of Use)
script (23.64 KB, text/x-python)
2016-06-22 08:30 UTC, Rony Gong 🔥
no flags Details

Description Matt Tyson 🤬 2016-06-08 23:16:19 UTC
Reported to happen on bug creation.

*xmlrpclib.Fault: <Fault 32000: "Red Hat Bugzilla's database reported a query serialization error. Most likely this occurred because another user or process attempted to change the same data that you were attempting to change. Please press Back and retry the transaction.

INSERT INTO dependencies (dependson, blocked) VALUES (?,?) at /var/www/html/bugzilla/Bugzilla/Bug.pm line 1148 Bugzilla::Bug::update('Bugzilla::Bug=HASH(0x7fbe96b37a28)') called at /var/www/html/bugzilla/Bugzilla/WebService/Bug.pm line 768 Bugzilla::WebService::Bug::update('Bugzilla::WebService::Bug', 'HASH(0x7fbe96b9f060)') called at /usr/share/perl5/vendor_perl/SOAP/Lite.pm line 2875 eval {...} called at /usr/share/perl5/vendor_perl/SOAP/Lite.pm line 2860 eval {...} called at /usr/share/perl5/vendor_perl/SOAP/Lite.pm line 2826 SOAP::Server::handle('Bugzilla::WebService::Server::XMLRPC=HASH(0x7fbe95d84db8)', '<?xml version=\\'1.0\\'?>\\x{a}<methodCall>\\x{a}<methodName>Bug.update</...') called at /usr/share/perl5/vendor_perl/SOAP/Transport/HTTP.pm line 438 SOAP::Transport::HTTP::Server::handle('Bugzilla::WebService::Server::XMLRPC=HASH(0x7fbe95d84db8)') called at /usr/share/perl5/vendor_perl/SOAP/Transport/HTTP.pm line 831 SOAP::Transport::HTTP::Apache::handler('Bugzilla::WebService::Server::XMLRPC=HASH(0x7fbe95d84db8)') called at /var/www/html/bugzilla/xmlrpc.cgi line 39

Comment 1 Jeff Fearn 🐞 2016-06-13 05:39:25 UTC
Qiang, can you get the script of Gil and see if you can duplicate the serialization error on the performance cluster?

BTW this [1] is about the best explanation I've read about why this happens.

1: http://stackoverflow.com/questions/12837708/predicate-locking-in-postgresql-9-2-1-with-serializable-isolation#12842204

Comment 2 Rony Gong 🔥 2016-06-22 08:22:54 UTC
(In reply to Jeff Fearn from comment #1)
> Qiang, can you get the script of Gil and see if you can duplicate the
> serialization error on the performance cluster?
> 
> BTW this [1] is about the best explanation I've read about why this happens.
> 
> 1:
> http://stackoverflow.com/questions/12837708/predicate-locking-in-postgresql-
> 9-2-1-with-serializable-isolation#12842204

I could reproduce this on the performance cluster, seems this error happened when updating flags

steps:
1.Download the attachment
2.Create a file in your home dir  ~/.bugzilla_auth  with following setting:
[bugzilla]
user = <username>
password = <password>
server = https://bugzilla-qe-05.host.stage.eng.rdu2.redhat.com/xmlrpc.cgi

3.Then run:
$ python clone_us_to_ds.py --debug  --from-list=1334726,1306325 --orig-target-milestone=ovirt-4.0.1 --include-post --skip-bad-bugs 


This error couldn't be reproduced 100%, however very easy to reproduce.

Comment 3 Rony Gong 🔥 2016-06-22 08:30:26 UTC
Created attachment 1170631 [details]
script

Comment 4 Jeff Fearn 🐞 2016-08-01 05:35:38 UTC
I couldn't duplicate this on my test VM, but it runs pretty slow. The devel cluster is still not running so I can't test on that.

On qe-05 I get "Fault: <Fault 130: 'You tried to grant pm_ack. Only a user with the required permissions may make this change.'>"

Looking at the code it looks like it might be changing flags that are also being set by rules engine jobs, so that might be causing this, if so then we can't resolve that.

Comment 5 Matt Tyson 🤬 2016-09-27 06:06:10 UTC
I'm not able to replicate this as my copy of the bugzilla database isn't in the right state, and the script does not run.

Is this bug still an issue?  If so, how can I set my test instance up so that the script will run?

I notice in the script (and as I mentioned in RT) that the script will create the bug, then immediately try to set flags.

This is likely causing a race condition with the rules engine.  You should modify the script to apply the flags at bug creation time.  This will avoid the race condition with the rules engine.

The sequence of events in your script would be like this

1) Script creates a new bug
2) Rules engine starts working
3) Script updates some flags
4) Rules engine starts again


Steps 2 and 3 would be happening at the same time, which would cause this issue.

If you applied the flags at bug creation it would be:

1) Script creates a new bug
2) Rules engine starts working

And there should be no conflict.

Comment 6 Jeff Fearn 🐞 2016-10-26 23:48:58 UTC
Moving to post bet5a work as we haven't been able to isolate this.

Comment 7 gshinar 2017-03-27 11:21:54 UTC
I'm currently using a loop of 3 retries to update Bugzilla with new stuff. 
I haven't seen this loop goes beyond the second try so I'm good with this solution for now

Comment 8 Jeff Fearn 🐞 2019-01-23 23:14:16 UTC
In preparation for entering maintenance mode we are closing bugs that are unlikely to be scheduled in maintenance sprints.


Note You need to log in before you can comment on or make changes to this bug.