Bug 1295880

Summary: katello-disconnected fails to sync and export ISO/Image repositories
Product: Red Hat Satellite Reporter: Jesus M. Rodriguez <jesusr>
Component: Inter Satellite SyncAssignee: Jesus M. Rodriguez <jesusr>
Status: CLOSED ERRATA QA Contact: Katello QA List <katello-qa-list>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.1.5CC: bkearney, cwelton, jmatthew, mmccune
Target Milestone: UnspecifiedKeywords: Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
The disconnected server was only able to handle rpm repositories, not ISO repositories. The code was updated to support iso repositories as well. (BZ#1295880)
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-02-10 15:37:48 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1212602, 1292898, 1294535    

Description Jesus M. Rodriguez 2016-01-05 16:39:43 UTC
[root@localhost ~]# katello-disconnected disable --all
[root@localhost ~]# katello-disconnected enable -r cf-me-5_3-for-rhel-6-files--x86_64
[root@localhost ~]# katello-disconnected configure
[root@localhost ~]# katello-disconnected sync

No errors reported after calling sync.

[root@localhost ~]# mkdir cfme
[root@localhost ~]# katello-disconnected export --repos cf-me-5_3-for-rhel-6-files--x86_64 --target cfme/

Export finishes QUICKLY which is abnormal, but no errors. Output directory only contains a listing file.

[root@localhost ~]# ls cfme/content/dist/cf-me/server/5.3/x86_64/files/
listing

What I expected was directories containing the virt images that are contained in the repos:

[root@localhost cfme]# ls -lr cf-me-5_4-for-rhel-6-files--x86_64/
total 10932628
-rw-r--r--. 1 apache apache       1766 Dec 21 11:57 PULP_MANIFEST
-rw-r--r--. 1 apache apache  618496000 Dec 19 17:28 cfme-vsphere-5.4-56.x86_64.vsphere.ova
-rw-r--r--. 1 apache apache  582082560 Dec 19 17:25 cfme-vsphere-5.4-53.x86_64.vsphere.ova
-rw-r--r--. 1 apache apache  628838400 Dec 19 17:27 cfme-vsphere-5.4-47.x86_64.vsphere.ova
-rw-r--r--. 1 apache apache  563824640 Dec 19 17:25 cfme-vsphere-5.4-46.x86_64.vsphere.ova
-rw-r--r--. 1 apache apache  564541440 Dec 19 17:27 cfme-vsphere-5.4-43.x86_64.vsphere.ova
-rw-r--r--. 1 apache apache  622985216 Dec 19 17:20 cfme-rhos-5.4-33.x86_64.qcow2
-rw-r--r--. 1 apache apache  602210304 Dec 19 17:19 cfme-rhos-5.4-30.x86_64.qcow2
-rw-r--r--. 1 apache apache  675676160 Dec 19 17:19 cfme-rhos-5.4-28.x86_64.qcow2
-rw-r--r--. 1 apache apache  594477056 Dec 19 17:19 cfme-rhos-5.4-27.x86_64.qcow2
-rw-r--r--. 1 apache apache  605880320 Dec 19 17:29 cfme-rhos-5.4-26.x86_64.qcow2
-rw-r--r--. 1 apache apache  560033986 Dec 19 17:24 cfme-rhevm-5.4-32.x86_64.rhevm.ova
-rw-r--r--. 1 apache apache  542812494 Dec 19 17:22 cfme-rhevm-5.4-29.x86_64.rhevm.ova
-rw-r--r--. 1 apache apache  597329036 Dec 19 17:29 cfme-rhevm-5.4-26.x86_64.rhevm.ova
-rw-r--r--. 1 apache apache  548143916 Dec 19 17:29 cfme-rhevm-5.4-25.x86_64.rhevm.ova
-rw-r--r--. 1 apache apache  531896600 Dec 19 17:21 cfme-rhevm-5.4-24.x86_64.rhevm.ova
-rw-r--r--. 1 apache apache 2355761152 Dec 19 17:25 cfme-hyperv-5.4-3.x86_64.vhd
[root@localhost cfme]# ls -lr cf-me-5_5-for-rhel-7-files--x86_64/
total 3759616
-rw-r--r--. 1 apache apache        455 Dec 21 11:57 PULP_MANIFEST
-rw-r--r--. 1 apache apache  640839680 Dec 19 17:20 cfme-vsphere-5.5.0.13-2.x86_64.vsphere.ova
-rw-r--r--. 1 apache apache  687996928 Dec 19 17:20 cfme-rhos-5.5.0.13-2.x86_64.qcow2
-rw-r--r--. 1 apache apache  624628407 Dec 19 17:19 cfme-rhevm-5.5.0.13-2.x86_64.rhevm.ova
-rw-r--r--. 1 apache apache 1896372736 Dec 19 17:27 cfme-hyperv-5.5.0.13-2.x86_64.vhd

Comment 1 Jesus M. Rodriguez 2016-01-05 16:44:25 UTC
My research found a few problems:

1) when talking to pulp, the ISO distributor and importer needed to be used in order for pulp to sync ISO/files/images repos.

2) the export wasn't tarring up the isos directory which caused the exported content not to contain any of the ISO/files/images that were synced by pulp.

3) the expand_export.sh script would only expand the content-export-?? files and would skip any other exported content files in the directory. This is because tar stops processing once it finds the end of file from the cat command. So while content-export-* might match all the files in the directory, only the -?? get processed first the last one has an EOF which causes tar to stop.

Comment 2 Jesus M. Rodriguez 2016-01-05 16:44:54 UTC
A pull request was created to address the above issues:

https://github.com/Katello/katello-utils/pull/4

Comment 3 Jesus M. Rodriguez 2016-01-05 16:53:57 UTC
TEST PLAN
-----------
To test follow the instructions for setting up a Disconnected Sync host:

https://access.redhat.com/documentation/en-US/Red_Hat_Satellite/6.0/html/User_Guide/sect-Disconnected_Satellite.html

Then ⁠section 4.3.2. Synchronizing Content, sync only a "files" repository (makes it easier to test)

# katello-disconnected disable --all
# katello-disconnected enable -r cf-me-5_4-for-rhel-6-files--x86_64
# katello-disconnected configure
# katello-disconnected sync

*) verify /var/lib/pulp/content/iso has some content in it

Once sync is complete then EXPORT the content to a temporary directory, following the instructions found here: https://access.redhat.com/documentation/en-US/Red_Hat_Satellite/6.0/html/User_Guide/Exporting_Content.html

*) verify the expand_export.sh script contains the new cat | tar commands.

run expand_export.sh

*) verify that you have .ova and .qcow2 files

Comment 8 Corey Welton 2016-02-08 20:57:26 UTC
RHCI team has verified functionality of this BZ and performed best-effort to assure there have been no regressions due to the changes added.


Marking verified.

Comment 10 errata-xmlrpc 2016-02-10 15:37:48 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:0161