Bug 1021662 - Pulp http proxy with authentication seems broken
Summary: Pulp http proxy with authentication seems broken
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Pulp
Classification: Retired
Component: rpm-support
Version: Master
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: 2.3.0
Assignee: Sayli Karmarkar
QA Contact: Preethi Thomas
URL:
Whiteboard:
Depends On:
Blocks: 950743 1017348 1021669 1022662
TreeView+ depends on / blocked
 
Reported: 2013-10-21 18:19 UTC by Justin Sherrill
Modified: 2015-03-23 01:12 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 1021669 1022662 (view as bug list)
Environment:
Last Closed: 2013-12-09 14:30:57 UTC
Embargoed:


Attachments (Terms of Use)

Description Justin Sherrill 2013-10-21 18:19:10 UTC
Description of problem:

Attempting to configure pulp using the global auth file for yum content:

/etc/pulp/server/plugins.conf.d/yum_importer.json

With either:

{
    "num_threads" : 4,
    "proxy_host" : "http://1.2.3.4",
    "proxy_username": "admin",
    "proxy_password": "redhat",
    "proxy_port" : 8888
}

or


{
    "num_threads" : 4,
    "proxy_host" : "http://admin:redhat@1.2.3.4",
    "proxy_port" : 8888
}


or


{
    "num_threads" : 4,
    "proxy_host" : "http://admin:redhat@1.2.3.4:8888",
    "proxy_port" : 8888
}


Does not seem to work. In most cases I get:

Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py"
, line 109, in run
    metadata_files = self.get_metadata()
  File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py"
, line 194, in get_metadata
    raise FailedException(str(e))
FailedException: Proxy Authentication Required


According to my squid log, pulp isn't sending the auth user name

1382375504.460      0 1.2.3.4 TCP_DENIED/407 3969 GET http://lzap.fedorapeople.org/fakerepos/zoo5/repodata/repomd.xml - NONE/- text/html


whereas with curl (passing an incorrect password), i see:


1382375255.594      0 1.2.3.4 TCP_DENIED/407 4105 GET http://lzap.fedorapeople.org/fakerepos/zoo5/repodata/repomd.xml admin NONE/- text/html


Note 'admin' (the user) is missing when pulp tries to access it.

Note that it works fine using curl with the correct password.  

curl -X GET http://google.com/  --proxy http://admin:redhat@1.2.3.4:8888

It very well could be my proxy configuration, however I would expect it to work with basic or digest auth. 



If it 'works', it would be great to get some instructions for setting up a proxy requiring authentication that works with pulp.

Comment 1 Sayli Karmarkar 2013-10-23 18:10:37 UTC
This was a bug in python-requests when using the documented way to specify proxy authentication. They are working on fixing the issue, but meanwhile, we found an alternate way of handling this and implemented it.  https://github.com/pulp/nectar/pull/12/

Comment 2 Jeff Ortel 2013-10-23 23:27:14 UTC
build: 2.3.0-0.24.beta

Comment 3 Preethi Thomas 2013-10-24 20:40:18 UTC
seems like this is still an issue

fails-qa

[root@qe-blade-03 ~]#  pulp-admin rpm repo create --repo-id rhel64-ks --feed https://cdn.redhat.com/content/dist/rhel/rhui/server/6/6.4/x86_64/kickstart/ --feed-ca-cert CDN/cdn.redhat.com-chain.crt --feed-cert CDN/1359391926_4512.crt  --feed-key CDN/1359391926_4512.key --skip rpm,erratum  --proxy-host http://pulp-proxy.usersys.redhat.com --proxy-port 8080
Successfully created repository [rhel64-ks]

[root@qe-blade-03 ~]# 
[root@qe-blade-03 ~]# 
[root@qe-blade-03 ~]# pulp-admin rpm repo sync run --repo-id rhel64-ks
+----------------------------------------------------------------------+
                  Synchronizing Repository [rhel64-ks]
+----------------------------------------------------------------------+

This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.

Downloading metadata...
[-]
... failed

Cannot connect to proxy. Socket error: Tunnel connection failed: 407 Proxy
Authentication Required.

[root@qe-blade-03 ~]# 


Without proxy auth

[root@qe-blade-03 ~]#  pulp-admin rpm repo create --repo-id rhel64-os --feed https://cdn.redhat.com/content/dist/rhel/rhui/server/6/6.4/x86_64/os/ --feed-ca-cert CDN/cdn.redhat.com-chain.crt --feed-cert CDN/1359391926_4512.crt  --feed-key CDN/1359391926_4512.key --skip rpm,erratum  --proxy-host http://pulp-upgrade-testing.usersys.redhat.com --proxy-port 3128
Successfully created repository [rhel64-os]

[root@qe-blade-03 ~]# 
[root@qe-blade-03 ~]# 
[root@qe-blade-03 ~]# pulp-admin rpm repo sync run --repo-id rhel64-os+----------------------------------------------------------------------+
                  Synchronizing Repository [rhel64-os]
+----------------------------------------------------------------------+

This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.

Downloading metadata...
[/]
... completed

Downloading repository content...
[==================================================] 100%
RPMs:       0/0 items
Delta RPMs: 0/0 items

^C[root@qe-blade-03 ~]#

Comment 4 Jeff Ortel 2013-10-29 15:29:46 UTC
build: 2.3.0-0.26.beta

Comment 5 Preethi Thomas 2013-10-30 20:38:01 UTC
[root@dell-pe2900-01 ~]# pulp-admin rpm repo create --repo-id rhel59-os --feed https://cdn.redhat.com/content/dist/rhel/rhui/server/5/5.9/x86_64/os/ --feed-ca-cert CDN/cdn.redhat.com-chain.crt --feed-cert CDN/1359391926_4512.crt  --feed-key CDN/1359391926_4512.key  --proxy-host http://pulp-proxy.usersys.redhat.com --proxy-port 8888 --proxy-user admin --proxy-pass admin 
Successfully created repository [rhel59-os]

[root@hp-sl2x170zg6-01 ~]# pulp-admin rpm repo sync run --repo-id rhel59-os
+----------------------------------------------------------------------+
                  Synchronizing Repository [rhel59-os]
+----------------------------------------------------------------------+


This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.

Downloading metadata...
[|]
... completed

Downloading repository content...
[==================================================] 100%
RPMs:       1623/1623 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 19/19 items
... completed

Importing errata...
[|]
... completed

Importing package groups/categories...
[|]
... completed

Publishing distributions...
[==================================================] 100%
Distributions: 19/19 items
... completed

Publishing packages...
[==================================================] 100%
Packages: 15362/15362 items
... completed

Generating metadata
[\]
... completed

Publishing repository over HTTPS
[-]
... completed

[root@hp-sl2x170zg6-01 ~]# 
[root@hp-sl2x170zg6-01 ~]#

Comment 6 Preethi Thomas 2013-10-31 14:25:13 UTC
verified

[root@hp-sl2x170zg6-01 ~]# rpm -qa pulp-server
pulp-server-2.3.0-0.26.beta.el6.noarch
[root@hp-sl2x170zg6-01 ~]# 

[root@hp-sl2x170zg6-01 ~]# pulp-admin rpm repo create --repo-id rhel62-os --feed https://cdn.redhat.com/content/dist/rhel/rhui/server/6/6.2/x86_64/os/ --feed-ca-cert CDN/cdn.redhat.com-chain.crt --feed-cert CDN/1359391926_4512.crt  --feed-key CDN/1359391926_4512.key
Successfully created repository [rhel62-os]


[root@hp-sl2x170zg6-01 ~]# pulp-admin rpm repo sync run --repo-id rhel62-os+----------------------------------------------------------------------+
                  Synchronizing Repository [rhel62-os]
+----------------------------------------------------------------------+

This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.

Downloading metadata...
[\]
... failed

Cannot connect to proxy. Socket error: Tunnel connection failed: 407 Proxy
Authentication Required.



[root@hp-sl2x170zg6-01 ~]# pulp-admin rpm repo sync run --repo-id rhel62-os
+----------------------------------------------------------------------+
                  Synchronizing Repository [rhel62-os]
+----------------------------------------------------------------------+

This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.

Downloading metadata...
[-]
... completed

Downloading repository content...
[==================================================] 100%
RPMs:       7281/7281 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 6/6 items
... completed

Importing errata...
[/]
... completed

Importing package groups/categories...
[|]
... completed

Publishing distributions...
[==================================================] 100%
Distributions: 6/6 items
... completed

Publishing packages...
[==================================================] 100%
Packages: 7281/7281 items
... completed

Generating metadata
[/]
... completed

Publishing repository over HTTPS
[-]
... completed

[root@hp-sl2x170zg6-01 ~]# cat /etc/pulp/server/plugins.conf.d/yum_importer.json 

{
    "num_threads" : 4,
    "proxy_host" : "http://pulp-proxy.usersys.redhat.com",
    "proxy_username": "admin",
    "proxy_password": "admin",
    "proxy_port" : 8888
}
[root@hp-sl2x170zg6-01 ~]# 



[root@hp-sl2x170zg6-01 ~]# pulp-admin rpm repo create --repo-id zoo --feed http://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/zoo/
Successfully created repository [zoo]

[root@hp-sl2x170zg6-01 ~]# 
[root@hp-sl2x170zg6-01 ~]# 
[root@hp-sl2x170zg6-01 ~]# 
[root@hp-sl2x170zg6-01 ~]# pulp-admin rpm repo sync run --repo-id zoo +----------------------------------------------------------------------+
                     Synchronizing Repository [zoo]
+----------------------------------------------------------------------+

This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.

Downloading metadata...
[\]
... completed

Downloading repository content...
[==================================================] 100%
RPMs:       32/32 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed

Importing errata...
[-]
... completed

Importing package groups/categories...
[-]
... completed

Publishing packages...
[==================================================] 100%
Packages: 32/32 items
... completed

Publishing distributions...
[==================================================] 100%
Distributions: 0/0 items
... completed

Generating metadata
[\]
... completed

Publishing repository over HTTPS
[-]
... completed

[root@hp-sl2x170zg6-01 ~]#

Comment 7 Preethi Thomas 2013-12-09 14:30:57 UTC
Pulp 2.3 released.


Note You need to log in before you can comment on or make changes to this bug.