Bug 1017404

Summary: Not able to sync cdn repos when using proxy setting
Product: [Retired] Pulp Reporter: Sayli Karmarkar <skarmark>
Component: z_otherAssignee: Sayli Karmarkar <skarmark>
Status: CLOSED CURRENTRELEASE QA Contact: Preethi Thomas <pthomas>
Severity: unspecified Docs Contact:
Priority: high    
Version: MasterCC: cperry, paul.jochum, pthomas, skarmark
Target Milestone: ---Keywords: Triaged
Target Release: 2.3.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1014368 Environment:
Last Closed: 2013-12-09 14:32:00 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1014368    
Bug Blocks:    

Description Sayli Karmarkar 2013-10-09 19:53:56 UTC
+++ This bug was initially created as a clone of Bug #1014368 +++

Description of problem:
if I use the --feed-cert options with proxy, repo sync fails

Version-Release number of selected component (if applicable):
[root@ibm-ls22-05 ~]# rpm -qa |grep nectar
python-nectar-1.1.2-1.el6.noarch
[root@ibm-ls22-05 ~]# 



Tested in both 2.2 & 2.3 with the above nectar

How reproducible:


Steps to Reproduce:
1. Create a repo with proxy-host & port and feed certs/keys (eg: CDN repo)
2. Try to sync the repo

Actual results:

[root@ibm-ls22-05 ~]# pulp-admin rpm repo create --repo-id rhel6-4 --feed https://cdn.redhat.com/content/dist/rhel/rhui/server/6/6.4/i386/os/ --feed-ca-cert CDN/cdn.redhat.com-chain.crt --feed-cert CDN/1359391926_4512.crt  --feed-key CDN/1359391926_4512.key --proxy-host=http://pulp-proxy.usersys.redhat.com --proxy-port 3128 
Successfully created repository [rhel6-4]



[root@ibm-ls22-05 ~]# pulp-admin rpm repo sync run --repo-id rhel6-4 
+----------------------------------------------------------------------+
                   Synchronizing Repository [rhel6-4]
+----------------------------------------------------------------------+

This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.

Downloading metadata...
[|]
... failed

Service Unavailable

[root@ibm-ls22-05 ~]# 



2013-10-01 14:56:16,963 urllib3.connectionpool:INFO: Starting new HTTP connection (1): pulp-proxy.usersys.redhat.com
2013-10-01 14:56:18,885 nectar.downloaders.threaded:ERROR: Download of https://cdn.redhat.com/content/dist/rhel/rhui/server/6/6.4/i386/os/repodata/repomd.xml failed with code 503: Service Unavailable
2013-10-01 14:56:18,888 pulp_rpm.plugins.importers.yum.sync:ERROR: sync failed
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 109, in run
    metadata_files = self.get_metadata()
  File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 194, in get_metadata
    raise FailedException(str(e))
FailedException: Service Unavailable
2013-10-01 14:56:18,900 pulp.server.dispatch.task:ERROR: Importer indicated a failed response
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/pulp/server/dispatch/task.py", line 138, in _run
    result = call(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/pulp/server/managers/repo/sync.py", line 117, in sync
    raise PulpExecutionException(_('Importer indicated a failed response'))
PulpExecutionException: Importer indicated a failed response
2013-10-01 14:56:18,902 pulp.server.dispatch.task:INFO: FAILURE: Task 9fc2f4ee-aa28-4453-b703-0255a35f358c: CallRequest: RepoSyncManager.sync(u'rhel6-4', sync_config_override=None)
Expected results:


Additional info:

--- Additional comment from Sayli Karmarkar on 2013-10-08 17:34:11 EDT ---

https://github.com/pulp/pulp/pull/656

Added python-requests-2.0.0 package to pulp dependencies which also contains latest urllib3 in order to support proxy with https.

--- Additional comment from Sayli Karmarkar on 2013-10-09 14:10:37 EDT ---



--- Additional comment from Sayli Karmarkar on 2013-10-09 15:49:43 EDT ---

Merged and tagged all the relevant packages and dependencies.

Comment 1 Sayli Karmarkar 2013-10-09 19:58:02 UTC
This has been fixed and merged in 2.2. It will be moves to modified when 2.2 is merged into master.

Comment 2 Jeff Ortel 2013-10-11 15:26:11 UTC
build: 2.3.0-0.18.beta

Comment 3 Preethi Thomas 2013-10-22 12:43:53 UTC
verified

[root@qe-blade-03 ~]# rpm -qa pulp-server
pulp-server-2.3.0-0.23.beta.fc19.noarch
[root@qe-blade-03 ~]# 

[root@qe-blade-03 ~]#  pulp-admin rpm repo create --repo-id rhel63-os --feed https://cdn.redhat.com/content/dist/rhel/rhui/server/6/6.3/x86_64/os/ --feed-ca-cert CDN/cdn.redhat.com-chain.crt --feed-cert CDN/1359391926_4512.crt  --feed-key CDN/1359391926_4512.key --proxy-host http://pulp-proxy.usersys.redhat.com --proxy-port 8080 --proxy-user admin --proxy-pass admin
Successfully created repository [rhel63-os]


[root@qe-blade-03 ~]# pulp-admin rpm repo sync run --repo-id rhel63-os
+----------------------------------------------------------------------+
                  Synchronizing Repository [rhel63-os]
+----------------------------------------------------------------------+

This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.

Downloading metadata...
[/]
... completed

Downloading repository content...
[==================================================] 100%
RPMs:       7/7 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 7/7 items
... completed

Importing errata...
[|]
... completed

Importing package groups/categories...
[\]
... completed

Publishing packages...
[==================================================] 100%
Packages: 8921/8921 items
... completed

Publishing distributions...
[==================================================] 100%
Distributions: 7/7 items
... completed

Generating metadata
[/]
... completed

Publishing repository over HTTPS
[-]
... completed



[root@qe-blade-03 ~]# pulp-admin puppet repo update --repo-id puppet-repo --proxy-host http://pulp-upgrade-testing.usersys.redhat.com --proxy-port 3128
Repository [puppet-repo] successfully updated

[root@qe-blade-03 ~]# pulp-admin puppet repo sync run --repo-id puppet-repo+----------------------------------------------------------------------+
                 Synchronizing Repository [puppet-repo]
+----------------------------------------------------------------------+

This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.

Downloading metadata...
[==================================================] 100%
Metadata Query: 2/2 items
... completed

Downloading new modules...
[==================================================] 100%
Module: 38/38 items
... completed

Publishing modules...
[==================================================] 100%
Module: 38/38 items
... completed

Generating repository metadata...
[-]
... completed

Publishing repository over HTTP...
... completed

Publishing repository over HTTPS...
... skipped
[root@qe-blade-03 ~]#

Comment 4 Preethi Thomas 2013-12-09 14:32:00 UTC
Pulp 2.3 released.