Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1313801 - unable to sync content to capsule
Summary: unable to sync content to capsule
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Installation
Version: 6.2.0
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: Unspecified
Assignee: Justin Sherrill
QA Contact: Kedar Bidarkar
URL: http://projects.theforeman.org/issues...
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-03-02 11:27 UTC by Kedar Bidarkar
Modified: 2019-11-14 07:31 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-07-27 09:03:44 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)
sync_to_Capsule_fails (621.66 KB, application/x-xz)
2016-03-02 11:27 UTC, Kedar Bidarkar 		no flags Details
capsule sync status (58.23 KB, image/png)
2016-03-11 16:03 UTC, Kedar Bidarkar 		no flags Details
View All


Links
System ID Private Priority Status Summary Last Updated
Foreman Issue Tracker 14081 0 Normal Closed Turn off repo authenication in katello 2021-01-06 18:53:58 UTC
Red Hat Product Errata RHBA-2016:1500 0 normal SHIPPED_LIVE Red Hat Satellite 6.2 Base Libraries 2016-07-27 12:24:38 UTC

Description Kedar Bidarkar 2016-03-02 11:27:09 UTC 
Created attachment 1132256 [details]
sync_to_Capsule_fails

Description of problem:

Syncing content to capsule fails with the below error.

~]# hammer -u admin -p changeme capsule content synchronize --id 2
Ignoring ruby-libvirt-0.5.2 because its extensions are not built.  Try: gem pristine ruby-libvirt --version 0.5.2
[...................................................................    ] [95%]
RPM1004: Error retrieving metadata: Not found
RPM1004: Error retrieving metadata: Not found


More details available in the foreman_debug.

Version-Release number of selected component (if applicable):
Sat6.2.0-Beta-SNAP1

How reproducible:


Steps to Reproduce:
1. Import manifest and sync Redhat content
2. Sync "redhat content" which is part of life-cycle env to capsule
3.

Actual results:
unable to sync content to capsule 

Expected results:
Should be able to sync content to capsule.

Additional info:

Looks like the custom content get's synced but the RedHat content does not.

rhel7_sattools_repo]# pwd
/var/lib/pulp/published/yum/http/repos/Default_Organization/Dev/rhel72_cv/custom/rhel7_sattools_katello/rhel7_sattools_repo
[abc@xyz rhel7_sattools_repo]# ls | wc -l 
94
[abc@xyz rhel7_sattools_repo]#
Comment 2 John Mitsch 2016-03-03 14:25:29 UTC 
Connecting redmine issue http://projects.theforeman.org/issues/14040 from this bug
Comment 3 John Mitsch 2016-03-04 15:26:53 UTC 
This is actually blocked by a m2crypto issue https://gitlab.com/m2crypto/m2crypto/issues/103 that makes the ueber certs used to verify the repositories not valid if they are valid after 2050.
Comment 4 Mike McCune 2016-03-07 18:50:30 UTC 
We are going to temporarily work around this via this PR:

https://github.com/Katello/puppet-katello/pull/114
Comment 5 Kedar Bidarkar 2016-03-11 16:03:26 UTC 
Created attachment 1135273 [details]
capsule sync status

Capsule sync completes 100 %  as seen from the UI, but content actually couldn't sync. 

I don't see any DIR "rpm" under " /var/lib/pulp/content/units/".

 units]# ls ; pwd
distribution  yum_repo_metadata_file
/var/lib/pulp/content/units
Comment 6 Kedar Bidarkar 2016-03-11 16:19:14 UTC 
Some more info:

The RedHat content files appear to exist but for some reason, they do not. 
I think the softlinks exist but the actual content is missing. 

os]# ll zsh* ; pwd
lrwxrwxrwx. 1 apache apache 125 Mar 11 10:17 zsh-5.0.2-14.el7.x86_64.rpm -> /var/lib/pulp/content/units/rpm/cf/a0c3b4f43ef5e2866e6b8ea2c4133dd05ae26544f6be9b7756b8ed3134a500/zsh-5.0.2-14.el7.x86_64.rpm
lrwxrwxrwx. 1 apache apache 128 Mar 11 10:17 zsh-5.0.2-7.el7_1.1.x86_64.rpm -> /var/lib/pulp/content/units/rpm/ce/917e77d2633632da302055bb47d1e49dd9892a6f2ef28d5017b63d650c4f91/zsh-5.0.2-7.el7_1.1.x86_64.rpm
lrwxrwxrwx. 1 apache apache 128 Mar 11 10:17 zsh-5.0.2-7.el7_1.2.x86_64.rpm -> /var/lib/pulp/content/units/rpm/13/465cd7f1c3463a312a6573d0111a038d2271dedf8466107910e1691a5a6d09/zsh-5.0.2-7.el7_1.2.x86_64.rpm
lrwxrwxrwx. 1 apache apache 124 Mar 11 10:17 zsh-5.0.2-7.el7.x86_64.rpm -> /var/lib/pulp/content/units/rpm/a3/84dcc455923eb535b4c7a5cec7aec946304f8813e56f1655eceb93a9276894/zsh-5.0.2-7.el7.x86_64.rpm
/var/lib/pulp/published/yum/https/repos/Default_Organization/Dev/rhel7_capsule_cv/content/dist/rhel/server/7/7Server/x86_64/os
Comment 7 Bryan Kearney 2016-03-11 17:03:46 UTC 
Upstream bug assigned to jsherril
Comment 8 Bryan Kearney 2016-03-11 17:03:49 UTC 
Upstream bug component is Installer
Comment 9 John Mitsch 2016-03-11 17:10:23 UTC 
(In reply to Kedar Bidarkar from comment #6)
> Some more info:
> 
> The RedHat content files appear to exist but for some reason, they do not. 
> I think the softlinks exist but the actual content is missing. 
> 
> os]# ll zsh* ; pwd
> lrwxrwxrwx. 1 apache apache 125 Mar 11 10:17 zsh-5.0.2-14.el7.x86_64.rpm ->
> /var/lib/pulp/content/units/rpm/cf/
> a0c3b4f43ef5e2866e6b8ea2c4133dd05ae26544f6be9b7756b8ed3134a500/zsh-5.0.2-14.
> el7.x86_64.rpm
> lrwxrwxrwx. 1 apache apache 128 Mar 11 10:17 zsh-5.0.2-7.el7_1.1.x86_64.rpm
> ->
> /var/lib/pulp/content/units/rpm/ce/
> 917e77d2633632da302055bb47d1e49dd9892a6f2ef28d5017b63d650c4f91/zsh-5.0.2-7.
> el7_1.1.x86_64.rpm
> lrwxrwxrwx. 1 apache apache 128 Mar 11 10:17 zsh-5.0.2-7.el7_1.2.x86_64.rpm
> ->
> /var/lib/pulp/content/units/rpm/13/
> 465cd7f1c3463a312a6573d0111a038d2271dedf8466107910e1691a5a6d09/zsh-5.0.2-7.
> el7_1.2.x86_64.rpm
> lrwxrwxrwx. 1 apache apache 124 Mar 11 10:17 zsh-5.0.2-7.el7.x86_64.rpm ->
> /var/lib/pulp/content/units/rpm/a3/
> 84dcc455923eb535b4c7a5cec7aec946304f8813e56f1655eceb93a9276894/zsh-5.0.2-7.
> el7.x86_64.rpm
> /var/lib/pulp/published/yum/https/repos/Default_Organization/Dev/
> rhel7_capsule_cv/content/dist/rhel/server/7/7Server/x86_64/os

What you are seeing is typical of the new default download policy "on_demand" introduced by the lazy sync feature
Comment 10 Kedar Bidarkar 2016-03-16 18:10:43 UTC 
Moving this to ON_QA as per comment9
Comment 11 Kedar Bidarkar 2016-03-21 10:39:25 UTC 
VERIFIED With SAT62-SNAP4-compose1

We can now sync content to capsule.
Comment 13 Kedar Bidarkar 2016-04-12 05:15:35 UTC 
Syncing Capsule fails, turns red at 98% 

 with the below error as per /v/l/m from capsule.


Apr 12 01:02:20 katello1 pulp: pulp_rpm.plugins.importers.yum.parse.treeinfo:ERROR: some distro file downloads failed
Apr 12 01:02:21 katello1 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (4010-66624)
Apr 12 01:02:21 katello1 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (4010-66624) Traceback (most recent call last):
Apr 12 01:02:21 katello1 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (4010-66624)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 228, in run
Apr 12 01:02:21 katello1 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (4010-66624)     dist_sync.run()
Apr 12 01:02:21 katello1 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (4010-66624)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/parse/treeinfo.py", line 125, in run
Apr 12 01:02:21 katello1 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (4010-66624)     self._run(tmp_dir)
Apr 12 01:02:21 katello1 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (4010-66624)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/parse/treeinfo.py", line 189, in _run
Apr 12 01:02:21 katello1 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (4010-66624)     for destination, location in downloaded:
Apr 12 01:02:21 katello1 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (4010-66624)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/parse/treeinfo.py", line 274, in download_files
Apr 12 01:02:21 katello1 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (4010-66624)     raise DownloadFailed()
Apr 12 01:02:21 katello1 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (4010-66624) DownloadFailed
Apr 12 01:02:21 katello1 pulp: py.warnings:WARNING: (4010-66624) /usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py:804: DeprecationWarning: save is deprecated. Use insert_one or replace_one instead
Apr 12 01:02:21 katello1 pulp: py.warnings:WARNING: (4010-66624)   sync_result_collection.save(sync_result)
Apr 12 01:02:21 katello1 pulp: py.warnings:WARNING: (4010-66624)
Apr 12 01:02:21 katello1 pulp: pulp.server.async.tasks:INFO: Task failed : [303fa070-5a9b-4835-8eed-f52185647efc]
Apr 12 01:02:21 katello1 pulp: celery.worker.job:ERROR: (3693-66624) Task pulp.server.managers.repo.sync.sync[303fa070-5a9b-4835-8eed-f52185647efc] raised unexpected: PulpExecutionException('Importer indicated a failed response',)
Apr 12 01:02:21 katello1 pulp: celery.worker.job:ERROR: (3693-66624) Traceback (most recent call last):
Apr 12 01:02:21 katello1 pulp: celery.worker.job:ERROR: (3693-66624)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
Apr 12 01:02:21 katello1 pulp: celery.worker.job:ERROR: (3693-66624)     R = retval = fun(*args, **kwargs)
Apr 12 01:02:21 katello1 pulp: celery.worker.job:ERROR: (3693-66624)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 473, in __call__
Apr 12 01:02:21 katello1 pulp: celery.worker.job:ERROR: (3693-66624)     return super(Task, self).__call__(*args, **kwargs)
Apr 12 01:02:21 katello1 pulp: celery.worker.job:ERROR: (3693-66624)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 103, in __call__
Apr 12 01:02:21 katello1 pulp: celery.worker.job:ERROR: (3693-66624)     return super(PulpTask, self).__call__(*args, **kwargs)
Apr 12 01:02:21 katello1 pulp: celery.worker.job:ERROR: (3693-66624)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in __protected_call__
Apr 12 01:02:21 katello1 pulp: celery.worker.job:ERROR: (3693-66624)     return self.run(*args, **kwargs)
Apr 12 01:02:21 katello1 pulp: celery.worker.job:ERROR: (3693-66624)   File "/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py", line 810, in sync
Apr 12 01:02:21 katello1 pulp: celery.worker.job:ERROR: (3693-66624)     raise pulp_exceptions.PulpExecutionException(_('Importer indicated a failed response'))
Apr 12 01:02:21 katello1 pulp: celery.worker.job:ERROR: (3693-66624) PulpExecutionException: Importer indicated a failed response
Apr 12 01:02:21 katello1 pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[f1d12da3-980f-4be4-bbcc-53774234b209] succeeded in 0.00668240699451s: None
Comment 14 Kedar Bidarkar 2016-04-12 05:57:55 UTC 
Apr 12 01:02:20 katello1 pulp: pulp_rpm.plugins.importers.yum.parse.treeinfo:ERROR: some distro file downloads failed
----------------------------------------------------------------------

Tried provisioning both RHEL7 and RHEL6 repos, 

RHEL7:  provisioning was successful ( boot files could be fetched from isolated capsule by the HOST).

RHEL6 : It was unable to fetch the boot files { vmlinuz and initrd )

May be the above "ERROR: some distro file downloads failed" is being referred to RHEL6 distro files.
Comment 15 Justin Sherrill 2016-04-12 12:49:58 UTC 
Kedar, 

Can we open a new bz for this since this is an entirely different issue (in the same area)?  

Also please attach a sosreport from the satellite and capsule to that bz.

Thanks,

-Justin
Comment 16 Kedar Bidarkar 2016-04-12 13:28:18 UTC 
ok, will be closing this and raising a new bug with foreman-debug.
Comment 18 errata-xmlrpc 2016-07-27 09:03:44 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1500
Note You need to log in before you can comment on or make changes to this bug.