Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1381352 - Locks queries that include exclusive lock search can be slow
Summary: Locks queries that include exclusive lock search can be slow
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Tasks Plugin
Version: 6.2.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: Unspecified
Assignee: Ivan Necas
QA Contact: Lukas Pramuk
URL:
Whiteboard:
Depends On:
Blocks: 1394374
TreeView+ depends on / blocked
 
Reported: 2016-10-03 19:37 UTC by Bryan Kearney
Modified: 2020-03-11 15:16 UTC (History)
8 users (show)

Fixed In Version: tfm-rubygem-foreman-tasks-0.7.14.11-1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1394374 (view as bug list)
Environment:
Last Closed: 2016-12-12 17:14:08 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Foreman Issue Tracker 16779 0 None None None 2016-10-03 19:37:57 UTC
Red Hat Product Errata RHBA-2016:2940 0 normal SHIPPED_LIVE Satellite 6.2.5 Async Bug Release 2016-12-12 22:08:21 UTC

Description Bryan Kearney 2016-10-03 19:37:55 UTC 
When a locking query includes limit on 'exclusive' flag, the execution plan
can be inefficient due to missing index.

<pre>
foreman=# explain analyze SELECT COUNT(*) FROM "foreman_tasks_locks"
   INNER JOIN "foreman_tasks_tasks" ON "foreman_tasks_tasks"."id" = "foreman_tasks_locks"."task_id" WHERE (foreman_tasks_tasks.state != 'stopped')
   AND ("foreman_tasks_locks"."task_id" NOT IN ('8abb9ca3-815f-49ea-b97b-6aa505f2a29a')) AND "foreman_tasks_locks"."name" = 'task_owner' AND
   "foreman_tasks_locks"."resource_id" = 1 AND "foreman_tasks_locks"."resource_type" = 'User' AND "foreman_tasks_locks"."exclusive" = 't';
</pre>

This sql is generated when locks are getting checked when starting a new task, that requires exclusive lock.

I've also noticed we use (COUNT(*)), while we are only interested into whether any such a record exist or not, so
using `exists?` instead of `any?` in https://github.com/theforeman/foreman-tasks/blob/4868e9df20c86fb54ab182fca4e7c1dd8fd8b6d0/app/models/foreman_tasks/lock.rb#L55
would be a bit more efficient. The most important thing though is the index here.
Comment 1 Bryan Kearney 2016-10-03 19:37:59 UTC 
Created from redmine issue http://projects.theforeman.org/issues/16779
Comment 2 Bryan Kearney 2016-10-03 19:38:02 UTC 
Upstream bug assigned to inecas
Comment 3 Bryan Kearney 2016-11-01 14:01:09 UTC 
Moving this bug to POST for triage into Satellite 6 since the upstream issue http://projects.theforeman.org/issues/16779 has been resolved.
Comment 7 Lukas Pramuk 2016-12-12 14:07:06 UTC 
VERIFIED.

@satellite-6.2.5-1.0.el6sat.noarch
tfm-rubygem-foreman-tasks-0.7.14.11-1.el6sat.noarch


Used following manual reproducer:

1. Install Satellite 6.2.4

1. Populate it with data:
To have enough tasks and locks I published 950 CVs, each having 35 small repositories. With this setup we have 950 CV publish tasks each having 144 exclusive locks!

2. Run SQL from comment #0:
foreman=> explain analyze SELECT COUNT(*) FROM "foreman_tasks_locks"...
...
 Total runtime: 4.029 ms
(9 rows)

3. Check CV publish task planning time 
Whole task started at: 2016-12-12 09:48:02 +0100
1st task action started at: 2016-12-12 08:49:48 UTC

Planning time = 106 seconds !!!

4. Upgrade to Satellite 6.2.5

5. Run SQL from comment #0:
foreman=> explain analyze SELECT COUNT(*) FROM "foreman_tasks_locks"...
...
 Total runtime: 0.208 ms
(9 rows)

foreman=> explain analyze SELECT COUNT('') FROM "foreman_tasks_locks"...
...
 Total runtime: 0.088 ms
(9 rows)

>>> based on altered db structures (new indices) the execution time dropped from ~ 4 sec to 0.088 sec which ~ 50 times faster

6. Check CV publish task planning time after upgrade
(this step is pending, upgrade not able to finish - import_rpms taking ages)

>>> I'll provide data once upgrade finises
Comment 8 errata-xmlrpc 2016-12-12 17:14:08 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:2940
Note You need to log in before you can comment on or make changes to this bug.