Bug 1293911 - Concurrent Actions::Katello::System::GenerateApplicability tasks fail in paused with DB transacation abort
Concurrent Actions::Katello::System::GenerateApplicability tasks fail in paus...
Status: CLOSED ERRATA
Product: Red Hat Satellite 6
Classification: Red Hat
Component: Errata Management (Show other bugs)
6.1.5
Unspecified Unspecified
high Severity high (vote)
: GA
: --
Assigned To: Justin Sherrill
: Triaged
Depends On:
Blocks: 1122832 1317008
  Show dependency treegraph
 
Reported: 2015-12-23 08:33 EST by Peter Vreman
Modified: 2017-09-19 09:49 EDT (History)
17 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-07-27 07:14:57 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Peter Vreman 2015-12-23 08:33:03 EST
Description of problem:
The task GenerateApplicability for a system cannot handle concurrent requests. E.g. running multiple separate yum commands on a client after eachother multiple requests to upload the package profile will be done. 

This leads then to paused tasks becuase of failing database transactions:

PGError: ERROR:  current transaction is aborted, commands ignored until end of transaction block
: DELETE FROM "katello_system_errata" WHERE "katello_system_errata"."system_id" = 155

Id: e254b260-24c2-42ba-a119-75152a315745
Label: Actions::Katello::System::GenerateApplicability
Name: Generate applicability
Owner: foreman_admin
Started at: 2015-12-21 20:09:05 UTC
Ended at: 2015-12-21 20:12:51 UTC
State: stopped
Result: success
Params: {"services_checked"=>["pulp", "pulp_auth"], "system_ids"=>[155], "locale"=>"en"}



Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. On a client execute yum command quickly after each other so it uplaods the package profile.
Example below:
for i in `seq 1 5`; do yum install -y bash-doc ; yum remove -y bash-doc; done
2.
3.

Actual results:
paused tasks

Expected results:
No paused tasks


Additional info:
Comment 6 Pavel Moravec 2016-05-17 07:01:50 EDT
Also hit in 6.1.8. pgsql denies the transaction due to pgsql error in logs:

ERROR:  duplicate key value violates unique constraint "katello_system_errata_eid_sid"
DETAIL:  Key (erratum_id, system_id)=(10222, 710) already exists.
STATEMENT:  INSERT INTO katello_system_errata (erratum_id, system_id) VALUES (10222, 710), (10223, 710), (10224, 710), (10287, 710), (10288, 710), (10290, 710), (10291, 710), (10293, 710), (10294, 710), (10296, 710), (10300, 710), (10302, 710), (10303, 710), (10304, 710), (10305, 710), (10306, 710), (10307, 710), (10308, 710), (10202, 710), (10347, 710), (10206, 710), (10153, 710), (10154, 710), (10162, 710), (10164, 710), (10165, 710), (10166, 710), (10167, 710), (10168, 710), (10170, 710), (10176, 710), (10177, 710), (10180, 710), (10181, 710), (10182, 710), (10185, 710), (10186, 710), (10327, 710), (10415, 710), (10256, 710), (10258, 710), (10344, 710)

Does it raise a bell? Why katello tries to insert something that is (partially) in the db already?
Comment 11 Bryan Kearney 2016-05-31 09:09:58 EDT
MOving to 6.2, verified. Dev believes 6.2 should address this and QE was not able to re-produce on the 6.2 beta code.
Comment 12 Lukas Zapletal 2016-06-07 07:00:41 EDT
I reproduced this once again today with 6.2 beta2 drop:

[root@ibm-ls22-03 ~]# rpm -q satellite katello foreman
satellite-6.2.0-14.2.el7sat.noarch
katello-3.0.0-6.el7sat.noarch
foreman-1.11.0.34-1.el7sat.noarch

I have a system where I encountered the error once during client testing.

Actions::Katello::Host::GenerateApplicability

PG::Error: ERROR:  current transaction is aborted, commands ignored until end of transaction block: DELETE FROM "katello_content_facet_errata" WHERE "katello_content_facet_errata"."content_facet_id" = 22

Resuming the task solved the issue.
Comment 14 Lukas Zapletal 2016-06-07 07:11:57 EDT
For the record, SQL statements which caused this are:

2016-06-07 09:23:51 CEST ERROR:  duplicate key value violates unique constraint "katello_content_facet_errata_eid_caid"
2016-06-07 09:23:51 CEST DETAIL:  Key (erratum_id, content_facet_id)=(18, 15) already exists.
2016-06-07 09:23:51 CEST STATEMENT:  INSERT INTO katello_content_facet_errata (erratum_id, content_facet_id) VALUES (18, 15), (107, 15), (158, 15), (210, 15), (253, 15), (261, 15), (321, 15), (439, 15), (404, 15), (424, 15), (483, 15), (561, 15), (589, 15), (613, 15), (699, 15), (774, 15), (833, 15), (799, 15), (849, 15), (867, 15), (914, 15), (916, 15), (964, 15), (989, 15), (1154, 15), (5643, 15), (1690, 15), (5671, 15), (5343, 15), (6242, 15), (5573, 15), (5451, 15), (5811, 15), (5834, 15), (4916, 15), (4945, 15), (5011, 15), (5874, 15), (5051, 15), (5213, 15), (5248, 15), (5256, 15), (5264, 15), (5272, 15), (6160, 15), (5479, 15), (5526, 15), (6030, 15), (6066, 15), (6092, 15), (6178, 15), (6180, 15), (6261, 15), (6350, 15), (6360, 15), (6429, 15), (6548, 15), (6569, 15), (6639, 15), (6663, 15), (6760, 15), (6790, 15), (6808, 15), (7727, 15), (6963, 15), (7177, 15), (7192, 15), (7203, 15), (7296, 15), (7320, 15), (7331, 15), (7368, 15), (7377, 15), (7410, 15), (7421, 15), (7432, 15), (7463, 15), (7479, 15), (7545, 15), (7618, 15), (7635, 15)
2016-06-07 09:23:51 CEST ERROR:  current transaction is aborted, commands ignored until end of transaction block
2016-06-07 09:23:51 CEST STATEMENT:  DELETE FROM "katello_content_facet_errata" WHERE "katello_content_facet_errata"."content_facet_id" = 15

2016-06-07 09:40:31 CEST ERROR:  duplicate key value violates unique constraint "katello_content_facet_errata_eid_caid"
2016-06-07 09:40:31 CEST DETAIL:  Key (erratum_id, content_facet_id)=(1566, 22) already exists.
2016-06-07 09:40:31 CEST STATEMENT:  INSERT INTO katello_content_facet_errata (erratum_id, content_facet_id) VALUES (1566, 22), (2298, 22), (424, 22), (2828, 22), (2903, 22), (637, 22), (3931, 22), (4011, 22), (4340, 22)
2016-06-07 09:40:31 CEST ERROR:  current transaction is aborted, commands ignored until end of transaction block
2016-06-07 09:40:31 CEST STATEMENT:  DELETE FROM "katello_content_facet_errata" WHERE "katello_content_facet_errata"."content_facet_id" = 2

Makes me to believe itt's the constraint "katello_content_facet_errata_eid_caid" which makes this to fail.
Comment 18 Justin Sherrill 2016-06-13 09:03:49 EDT
Moving this back to verified as https://bugzilla.redhat.com/show_bug.cgi?id=1344708 was opened to address comment #11 and QE no longer sees this issue (and its slightly different from comment #11).
Comment 19 Bryan Kearney 2016-07-27 07:14:57 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1501

Note You need to log in before you can comment on or make changes to this bug.