Bug 854697 - After manifest upload fails with bad repo url, manifest can no longer be uploaded at all, even after url is fixed
After manifest upload fails with bad repo url, manifest can no longer be uplo...
Product: Red Hat Satellite 6
Classification: Red Hat
Component: Content Management (Show other bugs)
Unspecified Unspecified
unspecified Severity high (vote)
: Unspecified
: --
Assigned To: Brad Buckingham
Og Maciel
: Triaged
Depends On:
  Show dependency treegraph
Reported: 2012-09-05 11:17 EDT by Jeff Weiss
Modified: 2014-11-09 17:52 EST (History)
7 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
When users set the content delivery network (CDN) to an invalid value and imported a manifest, the import of the manifest in to candlepin succeeded but the creation of the repos using candlepin/CDN failed. With this fix, if a previous import fails, the orchestration reverses the import of the manifest so the user can fix the CDN URL and attempt to re-import.
Story Points: ---
Clone Of:
Last Closed: 2012-12-04 14:52:28 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
test manifest (25.97 KB, application/zip)
2012-09-05 11:17 EDT, Jeff Weiss
no flags Details

External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2012:1543 normal SHIPPED_LIVE Important: CloudForms System Engine 1.1 update 2012-12-04 19:39:57 EST

  None (edit)
Description Jeff Weiss 2012-09-05 11:17:29 EDT
Created attachment 610084 [details]
test manifest

Description of problem:

Version-Release number of selected component (if applicable):
Katello Version: 1.1.9-1.git.130.edb680c.el6_3

How reproducible:

Steps to Reproduce:
1. Create new org, switch to it
2. Go to upload attached manifest to the org, enter repo url https://asdf.com
3. Select the manifest file and upload it
4. Fails with Connection timed out
5. Now try again use the correct repo url for the manifest http://inecas.fedorapeople.org/fakerepos/cds/
6. Upload the same manifest file again

Actual results:
Subscription manifest upload for provider 'Red Hat' failed.
Reason: Import is older than existing data
Resources::Candlepin::Owner: 409 Conflict {"displayMessage":"Import is older than existing data"} (POST /candlepin/owners/asdfasdf/imports) (RestClient::Conflict)
{"displayMessage":"Import is older than existing data"}

Expected results:
Upload completes successfully 2nd time

Additional info:
Appears to be a problem where the content is not checked BEFORE importing into candlepin. Or at the very least done in a transactional way that can be rolled back.
Comment 2 Ivan Necas 2012-09-07 02:06:45 EDT
There is a cli command `provider refresh_products` that does is able to re-create missing repositories based on the CDN.

We need to import the manifest to Candlepin to get the urls and certificates. Therefore we can't do the check before that. Unfortunately, AFAIK Candlepin doesn't support roll-backing the manifest import.
Comment 3 Brad Buckingham 2012-09-07 11:14:29 EDT
After investigation and discussion with Devan (candlepin) and Ivan (katello), it seems that we'll need some modifications to candlepin to address this bugzilla.  Today, we do not have a way from katello to determine whether or not a manifest has already been imported.  Given that candlepin processes the manifest, it is probably the best place to determine whether or not it has been previously imported, returning a response such as 'this manifest has already been imported'.  Using that response, katello can use the refresh_products suggested by Ivan above to handle the creation of the repositories in katello.

The issue raised has been added to the candlepin backlog.
Comment 6 Brad Buckingham 2012-09-12 13:04:29 EDT
katello pull request:

katello commit:

The above commit contains changes such that if a user attempts to import
a manifest (when a previous import has not been successfully completed) 
and the import fails, the orchestration will rollback
(i.e. unimport) the manifest.

This change is to address a scenario like the following:
- user sets the CDN to an invalid value
- user imports a manifest
- the import of the manifest in to candlepin succeeds; however,
  the creation of the repos using candlepin/cdn fails (since
  the cdn url is bad)

With this change, the user can fix the cdn url and attempt a
Comment 8 Og Maciel 2012-09-16 12:30:58 EDT
Tried to follow the same exact steps mentioned in the issue. The first 2 times I tried to import the manifest file with the correct CDN value, I received a message stating "Owner has already imported from another distributor". Today I tried to import it again, but using the http://asdf.com CDN and now I got:

  Subscription manifest upload for provider 'Red Hat' failed.
Reason: Import is older than existing data
Resources::Candlepin::Owner: 409 Conflict {"displayMessage":"Import is older than existing data"} (POST /candlepin/owners/AlphabetSoup/imports) (RestClient::Conflict)
{"displayMessage":"Import is older than existing data"}
Comment 9 Og Maciel 2012-09-17 09:54:18 EDT
Ok, so this issue can be verified if I follow the steps to the "T", but if you try to import the manifest using an incorrect CDN url more than once before using the correct url, then you hit the error mentioned in comment #c8
Comment 10 Brad Buckingham 2012-09-17 11:14:54 EDT
The changes introduced 'delete' the manifest from candlepin, if the total successful imports is 1 (i.e. this is the first import attempt).  Since katello does not currently store the import history locally, the 'total' is determiend by looking at the candlepin history.  The issue we are encountering is essentially the following:

- user sets CDN to invalid value

- user attempts to import a manifest
  - the import of the manifest to candlepin succeeds (history total=1)
  - the creation of repos fails
  - katello deletes the manifest from candlepin (i.e. total == 1)

- user attempts to import a manifest (again)
  - the import of the manifest to to candlepin succeeds (history total=2)
  - the creation of repos fails
  - katello does not delete manifest from candlepin (i.e. total > 1)

- user attempts to import a manifest (again)
  - the import of the manifest to candlepin fails... reason being, it could not
    be deleted on the previous failure, so candlepin reports that this import
    is 'older than existing data'

Unfortunately, katello does not currently have a way with current data to determine on that second failed attempt to delete the manifest.  It is quite possible that first success was a true success, systems were registered, subscribed and are consuming content.  As a result, we cannot simply delete the manifest again, without affecting existing subscriptions.

In order to address this behavior, the following are a couple of things that could potentially help:

1. Update candlepin to destroy the import history for an owner, when a manifest is deleted.  Since the manifest is no longer there, not sure that the history is very useful.  If this was done, in the above scenario the second failed attempt would again delete the manifest from candlepin.

2. Update candlepin to report to katello that the manifest has already been imported (i.e. rather than report 'older than existing data', report 'already imported').  Using this information, katello could be updated to make a decision to continue with the second phase of the import process, which is to create the repositories.  (Note: this is currently on candlepin backlog)

There are likely other solutions; however, posing a couple of possibilities.
Comment 11 Mike McCune 2012-09-17 11:26:49 EDT
What Brad said in comment #10 is correct.  There isn't a simple way to fix this in the time we have for the current release we are shipping this in.  In our upcoming release we will need to make changes in Katello and Candlepin to better address the issue to support better rollbacks of our imports.

I filed a 2.0 bug here:


and I'd vote we move this to VERIFIED
Comment 12 Og Maciel 2012-09-17 11:29:34 EDT
Verified using:                                                                                                      
* candlepin-0.7.8-1.el6cf.noarch                                                                                     
* candlepin-selinux-0.7.8-1.el6cf.noarch                                                                             
* candlepin-tomcat6-0.7.8-1.el6cf.noarch                                                                             
* katello-1.1.12-7.el6cf.noarch                                                                                      
* katello-all-1.1.12-7.el6cf.noarch                                                                                  
* katello-candlepin-cert-key-pair-1.0-1.noarch                                                                       
* katello-certs-tools-1.1.8-1.el6cf.noarch                                                                           
* katello-cli-1.1.8-4.el6cf.noarch                                                                                   
* katello-cli-common-1.1.8-4.el6cf.noarch                                                                            
* katello-common-1.1.12-7.el6cf.noarch                                                                               
* katello-configure-1.1.9-3.el6cf.noarch                                                                             
* katello-glue-candlepin-1.1.12-7.el6cf.noarch                                                                       
* katello-glue-pulp-1.1.12-7.el6cf.noarch                                                                            
* katello-qpid-broker-key-pair-1.0-1.noarch                                                                          
* katello-qpid-client-key-pair-1.0-1.noarch                                                                          
* katello-selinux-1.1.1-1.el6cf.noarch                                                                               
* pulp-1.1.12-1.el6cf.noarch                                                                                         
* pulp-common-1.1.12-1.el6cf.noarch                                                                                  
* pulp-selinux-server-1.1.12-1.el6cf.noarch
Comment 14 errata-xmlrpc 2012-12-04 14:52:28 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

Comment 15 Mike McCune 2013-08-16 14:19:42 EDT
getting rid of 6.0.0 version since that doesn't exist

Note You need to log in before you can comment on or make changes to this bug.