Red Hat Bugzilla – Bug 1313801
unable to sync content to capsule
Last modified: 2016-07-27 05:03:44 EDT
Created attachment 1132256 [details] sync_to_Capsule_fails Description of problem: Syncing content to capsule fails with the below error. ~]# hammer -u admin -p changeme capsule content synchronize --id 2 Ignoring ruby-libvirt-0.5.2 because its extensions are not built. Try: gem pristine ruby-libvirt --version 0.5.2 [................................................................... ] [95%] RPM1004: Error retrieving metadata: Not found RPM1004: Error retrieving metadata: Not found More details available in the foreman_debug. Version-Release number of selected component (if applicable): Sat6.2.0-Beta-SNAP1 How reproducible: Steps to Reproduce: 1. Import manifest and sync Redhat content 2. Sync "redhat content" which is part of life-cycle env to capsule 3. Actual results: unable to sync content to capsule Expected results: Should be able to sync content to capsule. Additional info: Looks like the custom content get's synced but the RedHat content does not. rhel7_sattools_repo]# pwd /var/lib/pulp/published/yum/http/repos/Default_Organization/Dev/rhel72_cv/custom/rhel7_sattools_katello/rhel7_sattools_repo [abc@xyz rhel7_sattools_repo]# ls | wc -l 94 [abc@xyz rhel7_sattools_repo]#
Connecting redmine issue http://projects.theforeman.org/issues/14040 from this bug
This is actually blocked by a m2crypto issue https://gitlab.com/m2crypto/m2crypto/issues/103 that makes the ueber certs used to verify the repositories not valid if they are valid after 2050.
We are going to temporarily work around this via this PR: https://github.com/Katello/puppet-katello/pull/114
Created attachment 1135273 [details] capsule sync status Capsule sync completes 100 % as seen from the UI, but content actually couldn't sync. I don't see any DIR "rpm" under " /var/lib/pulp/content/units/". units]# ls ; pwd distribution yum_repo_metadata_file /var/lib/pulp/content/units
Some more info: The RedHat content files appear to exist but for some reason, they do not. I think the softlinks exist but the actual content is missing. os]# ll zsh* ; pwd lrwxrwxrwx. 1 apache apache 125 Mar 11 10:17 zsh-5.0.2-14.el7.x86_64.rpm -> /var/lib/pulp/content/units/rpm/cf/a0c3b4f43ef5e2866e6b8ea2c4133dd05ae26544f6be9b7756b8ed3134a500/zsh-5.0.2-14.el7.x86_64.rpm lrwxrwxrwx. 1 apache apache 128 Mar 11 10:17 zsh-5.0.2-7.el7_1.1.x86_64.rpm -> /var/lib/pulp/content/units/rpm/ce/917e77d2633632da302055bb47d1e49dd9892a6f2ef28d5017b63d650c4f91/zsh-5.0.2-7.el7_1.1.x86_64.rpm lrwxrwxrwx. 1 apache apache 128 Mar 11 10:17 zsh-5.0.2-7.el7_1.2.x86_64.rpm -> /var/lib/pulp/content/units/rpm/13/465cd7f1c3463a312a6573d0111a038d2271dedf8466107910e1691a5a6d09/zsh-5.0.2-7.el7_1.2.x86_64.rpm lrwxrwxrwx. 1 apache apache 124 Mar 11 10:17 zsh-5.0.2-7.el7.x86_64.rpm -> /var/lib/pulp/content/units/rpm/a3/84dcc455923eb535b4c7a5cec7aec946304f8813e56f1655eceb93a9276894/zsh-5.0.2-7.el7.x86_64.rpm /var/lib/pulp/published/yum/https/repos/Default_Organization/Dev/rhel7_capsule_cv/content/dist/rhel/server/7/7Server/x86_64/os
Upstream bug assigned to jsherril@redhat.com
Upstream bug component is Installer
(In reply to Kedar Bidarkar from comment #6) > Some more info: > > The RedHat content files appear to exist but for some reason, they do not. > I think the softlinks exist but the actual content is missing. > > os]# ll zsh* ; pwd > lrwxrwxrwx. 1 apache apache 125 Mar 11 10:17 zsh-5.0.2-14.el7.x86_64.rpm -> > /var/lib/pulp/content/units/rpm/cf/ > a0c3b4f43ef5e2866e6b8ea2c4133dd05ae26544f6be9b7756b8ed3134a500/zsh-5.0.2-14. > el7.x86_64.rpm > lrwxrwxrwx. 1 apache apache 128 Mar 11 10:17 zsh-5.0.2-7.el7_1.1.x86_64.rpm > -> > /var/lib/pulp/content/units/rpm/ce/ > 917e77d2633632da302055bb47d1e49dd9892a6f2ef28d5017b63d650c4f91/zsh-5.0.2-7. > el7_1.1.x86_64.rpm > lrwxrwxrwx. 1 apache apache 128 Mar 11 10:17 zsh-5.0.2-7.el7_1.2.x86_64.rpm > -> > /var/lib/pulp/content/units/rpm/13/ > 465cd7f1c3463a312a6573d0111a038d2271dedf8466107910e1691a5a6d09/zsh-5.0.2-7. > el7_1.2.x86_64.rpm > lrwxrwxrwx. 1 apache apache 124 Mar 11 10:17 zsh-5.0.2-7.el7.x86_64.rpm -> > /var/lib/pulp/content/units/rpm/a3/ > 84dcc455923eb535b4c7a5cec7aec946304f8813e56f1655eceb93a9276894/zsh-5.0.2-7. > el7.x86_64.rpm > /var/lib/pulp/published/yum/https/repos/Default_Organization/Dev/ > rhel7_capsule_cv/content/dist/rhel/server/7/7Server/x86_64/os What you are seeing is typical of the new default download policy "on_demand" introduced by the lazy sync feature
Moving this to ON_QA as per comment9
VERIFIED With SAT62-SNAP4-compose1 We can now sync content to capsule.
Syncing Capsule fails, turns red at 98% with the below error as per /v/l/m from capsule. Apr 12 01:02:20 katello1 pulp: pulp_rpm.plugins.importers.yum.parse.treeinfo:ERROR: some distro file downloads failed Apr 12 01:02:21 katello1 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (4010-66624) Apr 12 01:02:21 katello1 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (4010-66624) Traceback (most recent call last): Apr 12 01:02:21 katello1 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (4010-66624) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 228, in run Apr 12 01:02:21 katello1 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (4010-66624) dist_sync.run() Apr 12 01:02:21 katello1 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (4010-66624) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/parse/treeinfo.py", line 125, in run Apr 12 01:02:21 katello1 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (4010-66624) self._run(tmp_dir) Apr 12 01:02:21 katello1 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (4010-66624) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/parse/treeinfo.py", line 189, in _run Apr 12 01:02:21 katello1 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (4010-66624) for destination, location in downloaded: Apr 12 01:02:21 katello1 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (4010-66624) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/parse/treeinfo.py", line 274, in download_files Apr 12 01:02:21 katello1 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (4010-66624) raise DownloadFailed() Apr 12 01:02:21 katello1 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (4010-66624) DownloadFailed Apr 12 01:02:21 katello1 pulp: py.warnings:WARNING: (4010-66624) /usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py:804: DeprecationWarning: save is deprecated. Use insert_one or replace_one instead Apr 12 01:02:21 katello1 pulp: py.warnings:WARNING: (4010-66624) sync_result_collection.save(sync_result) Apr 12 01:02:21 katello1 pulp: py.warnings:WARNING: (4010-66624) Apr 12 01:02:21 katello1 pulp: pulp.server.async.tasks:INFO: Task failed : [303fa070-5a9b-4835-8eed-f52185647efc] Apr 12 01:02:21 katello1 pulp: celery.worker.job:ERROR: (3693-66624) Task pulp.server.managers.repo.sync.sync[303fa070-5a9b-4835-8eed-f52185647efc] raised unexpected: PulpExecutionException('Importer indicated a failed response',) Apr 12 01:02:21 katello1 pulp: celery.worker.job:ERROR: (3693-66624) Traceback (most recent call last): Apr 12 01:02:21 katello1 pulp: celery.worker.job:ERROR: (3693-66624) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task Apr 12 01:02:21 katello1 pulp: celery.worker.job:ERROR: (3693-66624) R = retval = fun(*args, **kwargs) Apr 12 01:02:21 katello1 pulp: celery.worker.job:ERROR: (3693-66624) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 473, in __call__ Apr 12 01:02:21 katello1 pulp: celery.worker.job:ERROR: (3693-66624) return super(Task, self).__call__(*args, **kwargs) Apr 12 01:02:21 katello1 pulp: celery.worker.job:ERROR: (3693-66624) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 103, in __call__ Apr 12 01:02:21 katello1 pulp: celery.worker.job:ERROR: (3693-66624) return super(PulpTask, self).__call__(*args, **kwargs) Apr 12 01:02:21 katello1 pulp: celery.worker.job:ERROR: (3693-66624) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in __protected_call__ Apr 12 01:02:21 katello1 pulp: celery.worker.job:ERROR: (3693-66624) return self.run(*args, **kwargs) Apr 12 01:02:21 katello1 pulp: celery.worker.job:ERROR: (3693-66624) File "/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py", line 810, in sync Apr 12 01:02:21 katello1 pulp: celery.worker.job:ERROR: (3693-66624) raise pulp_exceptions.PulpExecutionException(_('Importer indicated a failed response')) Apr 12 01:02:21 katello1 pulp: celery.worker.job:ERROR: (3693-66624) PulpExecutionException: Importer indicated a failed response Apr 12 01:02:21 katello1 pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[f1d12da3-980f-4be4-bbcc-53774234b209] succeeded in 0.00668240699451s: None
Apr 12 01:02:20 katello1 pulp: pulp_rpm.plugins.importers.yum.parse.treeinfo:ERROR: some distro file downloads failed ---------------------------------------------------------------------- Tried provisioning both RHEL7 and RHEL6 repos, RHEL7: provisioning was successful ( boot files could be fetched from isolated capsule by the HOST). RHEL6 : It was unable to fetch the boot files { vmlinuz and initrd ) May be the above "ERROR: some distro file downloads failed" is being referred to RHEL6 distro files.
Kedar, Can we open a new bz for this since this is an entirely different issue (in the same area)? Also please attach a sosreport from the satellite and capsule to that bz. Thanks, -Justin
ok, will be closing this and raising a new bug with foreman-debug.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2016:1500