Bug 1370302 - ISO repository to Capsule synchronization fails with error "Katello::Errors::PulpError: PLP0000: Importer indicated a failed response"
Summary: ISO repository to Capsule synchronization fails with error "Katello::Errors::...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite 6
Classification: Red Hat
Component: Capsule - Content
Version: 6.2.0
Hardware: All
OS: Linux
medium
medium vote
Target Milestone: Unspecified
Assignee: Stephen Benjamin
QA Contact: jcallaha
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-08-25 21:24 UTC by dgupte
Modified: 2019-08-12 16:33 UTC (History)
18 users (show)

Fixed In Version: katello-3.3.0,katello-installer-base-3.0.0.96-1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1480358 (view as bug list)
Environment:
Last Closed: 2017-09-25 18:59:44 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2017:2803 normal SHIPPED_LIVE Satellite 6.2.12 bug fix update 2017-10-12 19:22:49 UTC
Foreman Issue Tracker 17590 None None None 2016-12-06 15:53:50 UTC

Description dgupte 2016-08-25 21:24:49 UTC
Description of problem:

Unable to synchronize repository 
Red Hat Enterprise Linux 7 Server ISOs x86_64 7.2 to Capsule server.

hammer> capsule content synchronize --id 2
[.............................................................................................] [100%]
Error: PLP0000: Importer indicated a failed response
PLP0034: The distributor dedhat_inc-Library-PuppetCapsuleSync indicated a failed response when publishing repository  Rehat_inc-Library-PuppetCapsuleSync.


Version-Release number of selected component (if applicable):
6.2.1

Steps to Reproduce:
1.  Enable and sync repository " 				
Red Hat Enterprise Linux 7 Server ISOs x86_64 7.2" on satellite server.
2. Try to sync repository content to capsule server.


Actual results:

Capsule synchronization task fails with error:

Katello::Errors::PulpError: PLP0000: Importer indicated a failed response 

Expected results:

Should able to sync ISO repository.


Additional info:

Complete traceback:-

---
pulp_tasks:
- exception: 
  task_type: pulp.server.managers.repo.sync.sync
  _href: "/pulp/api/v2/tasks/9e876f2e-27c5-452b-aecc-87353767ea18/"
  task_id: 9e876f2e-27c5-452b-aecc-87353767ea18
  tags:
  - pulp:repository:dedhat_inc-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_7_Server_ISOs_x86_64_7_2
  - pulp:action:sync
  finish_time: '2016-08-25T21:14:36Z'
  _ns: task_status
  start_time: '2016-08-25T21:14:35Z'
  traceback: |
    Traceback (most recent call last):
      File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
        R = retval = fun(*args, **kwargs)
      File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 473, in __call__
        return super(Task, self).__call__(*args, **kwargs)
      File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 103, in __call__
        return super(PulpTask, self).__call__(*args, **kwargs)
      File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in __protected_call__
        return self.run(*args, **kwargs)
      File "/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py", line 810, in sync
        raise pulp_exceptions.PulpExecutionException(_('Importer indicated a failed response'))
    PulpExecutionException: Importer indicated a failed response
  spawned_tasks: []
  progress_report:
    iso_importer:
      error_message:
        response_code: 403
        response_msg: Forbidden
      traceback: 
      finished_bytes: 0
      num_isos: 
      state: manifest_failed
      total_bytes: 
      state_times:
        not_started: '2016-08-25T21:14:35'
        manifest_in_progress: '2016-08-25T21:14:35'
        manifest_failed: '2016-08-25T21:14:35'
      num_isos_finished: 0
      iso_error_messages: []
  queue: reserved_resource_worker-1@example.com.dq
  state: error
  worker_name: reserved_resource_worker-1@example.com
  result: 
  error:
    code: PLP0000
    data: {}
    description: Importer indicated a failed response
    sub_errors: []
  _id:
    "$oid": 57bf5fb5840cd35eb4031f12
  id: 57bf5fb5840cd35eb4031f12
poll_attempts:
  total: 5
  failed: 1

Comment 3 Michael Hrivnak 2016-09-08 20:28:19 UTC
Thanks for including the task status! That's very helpful.

It looks like the error reported is "403 Forbidden".

That suggests the Capsule was not allowed to retrieve content from the Satellite. The most likely explanation is that the Capsule was using an entitlement certificate that the Satellite did not find acceptable. How it would have come to have an invalid certificate is hard to know.

If you can look at the importer config for that repo on the capsule, you can access the certificate and inspect it. It would be interesting to:

- make sure it is not expired
- make sure it is signed by the CA being used on the Satellite for verification
- try manually with curl or similar to access the same content on the Satellite with that certificate

I'm NEEDSINFOing Brad to see if there's anything else he suggests from the katello side on how to investigate whether there is a certificate mismatch. Maybe he can suggest a theory on how this could happen.

Of course please NEEDSINFO me if it looks like pulp might be doing something incorrect.

Comment 8 Mike McCune 2016-09-19 22:00:24 UTC
WORKAROUND:

Adding this line to the [main] section in /etc/pulp/repo_auth.conf on the Satellite and restarting httpd fixes it:

repo_url_prefixes: /pulp/repos,/pulp/ostree/web,/pulp/isos


after restarting just re-sync from Satellite to Capsule via the 'hammer capsule content synchronize' or UI options.

Comment 12 Stephen Benjamin 2016-12-06 15:53:46 UTC
Created redmine issue http://projects.theforeman.org/issues/17590 from this bug

Comment 13 Bryan Kearney 2016-12-21 21:21:09 UTC
Upstream bug assigned to jsherril@redhat.com

Comment 17 pm-sat@redhat.com 2017-01-20 19:19:37 UTC
Upstream bug assigned to stbenjam@redhat.com

Comment 18 pm-sat@redhat.com 2017-01-20 19:19:43 UTC
Moving this bug to POST for triage into Satellite 6 since the upstream issue http://projects.theforeman.org/issues/17590 has been resolved.

Comment 25 jcallaha 2017-08-31 19:24:57 UTC
Verified in Satellite 6.2.12 Snap 1.

I was able to successfully sync iso content to two separate capsules from two separate satellites. 

capsule 1
---------
[root@ibm-x3550m3-07 iso]# pwd
/var/lib/pulp/content/units/iso

[root@ibm-x3550m3-07 iso]# ll
total 0
drwxr-xr-x. 3 apache apache 76 Aug 31 20:43 14
drwxr-xr-x. 3 apache apache 76 Aug 31 20:43 27
drwxr-xr-x. 3 apache apache 76 Aug 31 20:55 30
drwxr-xr-x. 3 apache apache 76 Aug 31 20:54 32
drwxr-xr-x. 3 apache apache 76 Aug 31 20:55 46
drwxr-xr-x. 3 apache apache 76 Aug 31 20:53 4a
drwxr-xr-x. 3 apache apache 76 Aug 31 20:59 59
drwxr-xr-x. 3 apache apache 76 Aug 31 20:54 68
drwxr-xr-x. 3 apache apache 76 Aug 31 20:55 72
drwxr-xr-x. 3 apache apache 76 Aug 31 20:44 83
drwxr-xr-x. 3 apache apache 76 Aug 31 20:43 a5
drwxr-xr-x. 3 apache apache 76 Aug 31 20:53 af
drwxr-xr-x. 3 apache apache 76 Aug 31 20:54 c2
drwxr-xr-x. 3 apache apache 76 Aug 31 20:44 c9

[root@ibm-x3550m3-07 iso]# du -hd 1
247M	./27
264M	./14
431M	./a5
445M	./83
447M	./c9
744K	./af
612K	./4a
361M	./32
464M	./68
424K	./c2
470M	./72
465M	./30
453M	./46
3.8G	./59
7.8G	.

[root@ibm-x3550m3-07 iso]# file 14/4fcf84265f5b59f70d157e40aa784282447af3bf29fcfeecb295f244d71e8b/supp-server-7.1-rhel-7-x86_64-dvd.iso 
14/4fcf84265f5b59f70d157e40aa784282447af3bf29fcfeecb295f244d71e8b/supp-server-7.1-rhel-7-x86_64-dvd.iso: # ISO 9660 CD-ROM filesystem data 'Supp-7.1 RHEL-7 Server.x86_64'


capsule 2
---------
-bash-4.2# ll
total 0
drwxr-xr-x. 3 apache apache  76 Aug 31 15:14 09
drwxr-xr-x. 3 apache apache  76 Aug 31 15:16 29
drwxr-xr-x. 3 apache apache  76 Aug 31 15:17 30
drwxr-xr-x. 3 apache apache  76 Aug 31 15:14 32
drwxr-xr-x. 3 apache apache  76 Aug 31 15:14 3b
drwxr-xr-x. 3 apache apache  76 Aug 31 15:10 4a
drwxr-xr-x. 3 apache apache  76 Aug 31 15:10 64
drwxr-xr-x. 3 apache apache  76 Aug 31 15:17 68
drwxr-xr-x. 3 apache apache  76 Aug 31 15:16 6c
drwxr-xr-x. 3 apache apache  76 Aug 31 15:17 72
drwxr-xr-x. 3 apache apache  76 Aug 31 15:10 78
drwxr-xr-x. 3 apache apache  76 Aug 31 15:10 90
drwxr-xr-x. 3 apache apache  76 Aug 31 15:16 a7
drwxr-xr-x. 3 apache apache  76 Aug 31 15:10 a9
drwxr-xr-x. 3 apache apache  76 Aug 31 15:10 ad
drwxr-xr-x. 3 apache apache  76 Aug 31 15:10 af
drwxr-xr-x. 4 apache apache 146 Aug 31 15:17 c2
drwxr-xr-x. 3 apache apache  76 Aug 31 15:10 cb
drwxr-xr-x. 3 apache apache  76 Aug 31 15:10 e4
drwxr-xr-x. 3 apache apache  76 Aug 31 15:16 ff

-bash-4.2# file 29/6b17ed5088753377f4abb0d9067c69a6425bb4f8021525fc075f81ca915933/dd-megaraid_sas.el7_2.iso 
29/6b17ed5088753377f4abb0d9067c69a6425bb4f8021525fc075f81ca915933/dd-megaraid_sas.el7_2.iso: # ISO 9660 CD-ROM filesystem data 'OEMDRV'

Comment 27 errata-xmlrpc 2017-09-25 18:59:44 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2803


Note You need to log in before you can comment on or make changes to this bug.