Red Hat Bugzilla – Bug 108811
Oracle Deadlock caused by ContentItem.beforeSave()
Last modified: 2007-04-18 12:59:02 EDT
Description of problem:
Let's assume we have an Article object and want to change the CMS
folder it is located in. When ContentItem.setParent() is called and
the article is saved, the following happens:
1. beforeSave() calls setAncestors(getParent())
2. Although the parent is not being modified, its own beforeSave
method is called. (I'm not sure if this a bug or intended behaviour.)
3. Since the parent is a Folder, we again are in
ContentItem.beforeSave, and recursively call setAncestors for all
folders up to the root folder of the content section.
4. Persistence executes the SQL queries in this order: First,
cms_items.parent_id is set for the Article, and then
cms_items.ancestors is set starting with the root folder and ending
with the new parent folder.
This means that every thread which moves a content item or creates a
new one has to lock the root folder's row in cms_items, which hurts
scalability and can result in an Oracle deadlock (ORA-00060), which
happened when a customer did performance tests under heavy load.
I have attached a scenario description where this leads to a deadlock.
Besides the deadlock, the failure to properly lock the rows in
cms_items can result in the ancestors column getting corrupted - I
have also included a scenario for this.
Version-Release number of selected component (if applicable):
Should be fairly easy to reproduce under heavy load, although I have
only a bug report from a customer. But a look at the executed SQL
queries shows that there indeed is a problem.
Steps to Reproduce:
See attached file.
setAncestors must not be called for the item's new parent. Either
beforeSave has to be changed so it only calls setAncestors when the
parent has changed, or persistence has to be changed so it doesn't
call beforeSave for the parent.
See changelist 37597 for a fix to the deadlock problem. Corruption of
the ancestors column could be prevented by locking all affected rows
before reading and modifying the ancestors column (select for update).
Created attachment 95664 [details]
Deadlock and Data Corruption Scenarios
Created attachment 95665 [details]
Log output for single setParent call
Thanks. Carsten, I believe you are correct, and we're currently
testing a patch for this under load. -> ashah, who wrote the patch.
Carsten submits some of the best bug reports I've seen.
Well-researched, well-explained, and well-patched.
patch applied to 6.0.x (37880) and dev (37881)
This path has broken the hierarchy denormalization for live items
again - see bug 109718