Bug 1830403

Summary: Capsule sync fails when promoting a content view to more than one lifecyle env at the same time
Product: Red Hat Satellite Reporter: Rafael Cavalcanti <rcavalca>
Component: Capsule - ContentAssignee: Justin Sherrill <jsherril>
Status: CLOSED ERRATA QA Contact: Vladimír Sedmík <vsedmik>
Severity: high Docs Contact:
Priority: high    
Version: 6.7.0CC: adam.x.goodall, amarirom, bkearney, dgross, egolov, gpadholi, jalviso, jkrajice, jpasqual, jsherril, kagarwal, karnsing, kkinge, ktordeur, kupadhya, momran, myoder, ngalvin, pdudley, pmoravec, remy.couture, rrajput, sean, wclark
Target Milestone: 6.8.0Keywords: Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: tfm-rubygem-katello-3.16.0-0 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1851152 (view as bug list) Environment:
Last Closed: 2020-10-27 13:02:11 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
HOTFIX RPM for Satellite 6.7.0
none
HOTFIX RPM for Satellite 6.7.1 none

Description Rafael Cavalcanti 2020-05-01 20:14:59 UTC 
Description of problem:

When promoting a CV to a LCE, it will create a Capsule sync task, if this task is not completed and we promote the CV to another LCE, the second Capsule sync will fail.

Version-Release number of selected component (if applicable):

Satellite 6.7

How reproducible:

100%


Steps to Reproduce:
1. Create a content view
2. Create 2 LCE
3. Publish a new version of this content view
4. Promote this CV to the first LCE
5. Without waiting for the first capsule sync to finish, promote the CV to the second LCE. It will create a second capsule sync task which will be locked by the first one still running and it will fail.


Actual results:

On Satellite webUI -> Task page we will see the task with error as below:
---
	Synchronize capsule 'janga'	stopped	error	May 01, 2020, 3:32:21 PM  1 second
	
	Sync Content View on Capsule(s)	stopped	warning	May 01, 2020, 3:32:21 PM 1 second
---

Expected results:

The second Capsule sync should not be locked by the first Capsule sync.

Additional info:
Comment 3 wclark 2020-05-01 22:48:18 UTC 
Created attachment 1683911 [details]
HOTFIX RPM for Satellite 6.7.0
Comment 4 wclark 2020-05-01 22:50:22 UTC 
HOTFIX is available for Satellite 6.7.0. Installation instructions:

1. Take a complete backup or snapshot of Satellite server before installing the hotfix

2. Download the hotfix RPM from this BZ and copy it to Satellite server

3. # yum install /root/tfm-rubygem-katello-3.14.0.20-2.HOTFIXRHBZ1830403.el7sat.noarch.rpm --disableplugin=foreman-protector

4. # systemctl restart httpd dynflowd
Comment 5 Bryan Kearney 2020-05-04 16:55:12 UTC 
Upstream bug assigned to jsherril
Comment 6 Bryan Kearney 2020-05-04 16:55:14 UTC 
Moving this bug to POST for triage into Satellite 6 since the upstream issue https://projects.theforeman.org/issues/29678 has been resolved.
Comment 7 Justin Sherrill 2020-06-01 13:48:21 UTC 
*** Bug 1837000 has been marked as a duplicate of this bug. ***
Comment 8 Brad Buckingham 2020-06-03 11:05:00 UTC 
*** Bug 1834562 has been marked as a duplicate of this bug. ***
Comment 11 Pavel Moravec 2020-06-05 12:29:18 UTC 
Justin,
does a scenario "have a Capsule in Library Environment, enable few repos, sync them concurrently, see that some automatically triggered Caps sync tasks fails due to an acquired lock" - is this scenario observed on 6.7.0 also fixed by the HF / same fix?

I expect so, just would like to confirm it. (and I know, Caps shouldnt be attached to Library env..)

Thanks in advance for an answer.
Comment 15 wclark 2020-06-09 19:57:55 UTC 
Created attachment 1696378 [details]
HOTFIX RPM for Satellite 6.7.1
Comment 16 wclark 2020-06-09 19:59:30 UTC 
HOTFIX is available for Satellite 6.7.1. Installation instructions:

1. Take a complete backup or snapshot of Satellite server before installing the hotfix

2. Download the hotfix RPM from this BZ and copy it to Satellite server

3. # yum install /root/tfm-rubygem-katello-3.14.0.21-2.HOTFIXRHBZ1830403.el7sat.noarch.rpm --disableplugin=foreman-protector

4. # systemctl restart httpd dynflowd
Comment 17 wclark 2020-06-09 20:18:46 UTC 
Hi Pavel,

I'm not Justin, but I believe you are correct that the scenario you described should also be resolved by the same HF.

Since the fix was to define resource locks in app/lib/actions/katello/capsule_content/sync_capsule.rb as having the "linking" lock type, it shouldn't matter whether the Capsule sync got triggered due to a repo being synced in Library or due to a CV version being promoted to another environment.

I believe rguerra, the original reporter of the BZ, has the reproducer where we tested the HF, so to confirm you could work together to assign Library to a Capsule and test sync a few repos. With the HF installed, the Capsule syncs shouldn't get blocked.
Comment 22 errata-xmlrpc 2020-10-27 13:02:11 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Satellite 6.8 release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:4366