Bug 1021662 - Pulp http proxy with authentication seems broken
Pulp http proxy with authentication seems broken
Status: CLOSED CURRENTRELEASE
Product: Pulp
Classification: Community
Component: rpm-support (Show other bugs)
Master
Unspecified Unspecified
unspecified Severity high
: ---
: 2.3.0
Assigned To: Sayli Karmarkar
Preethi Thomas
: Triaged
Depends On:
Blocks: 950743 1017348 1021669 1022662
  Show dependency treegraph
 
Reported: 2013-10-21 14:19 EDT by Justin Sherrill
Modified: 2015-03-22 21:12 EDT (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1021669 1022662 (view as bug list)
Environment:
Last Closed: 2013-12-09 09:30:57 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Justin Sherrill 2013-10-21 14:19:10 EDT
Description of problem:

Attempting to configure pulp using the global auth file for yum content:

/etc/pulp/server/plugins.conf.d/yum_importer.json

With either:

{
    "num_threads" : 4,
    "proxy_host" : "http://1.2.3.4",
    "proxy_username": "admin",
    "proxy_password": "redhat",
    "proxy_port" : 8888
}

or


{
    "num_threads" : 4,
    "proxy_host" : "http://admin:redhat@1.2.3.4",
    "proxy_port" : 8888
}


or


{
    "num_threads" : 4,
    "proxy_host" : "http://admin:redhat@1.2.3.4:8888",
    "proxy_port" : 8888
}


Does not seem to work. In most cases I get:

Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py"
, line 109, in run
    metadata_files = self.get_metadata()
  File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py"
, line 194, in get_metadata
    raise FailedException(str(e))
FailedException: Proxy Authentication Required


According to my squid log, pulp isn't sending the auth user name

1382375504.460      0 1.2.3.4 TCP_DENIED/407 3969 GET http://lzap.fedorapeople.org/fakerepos/zoo5/repodata/repomd.xml - NONE/- text/html


whereas with curl (passing an incorrect password), i see:


1382375255.594      0 1.2.3.4 TCP_DENIED/407 4105 GET http://lzap.fedorapeople.org/fakerepos/zoo5/repodata/repomd.xml admin NONE/- text/html


Note 'admin' (the user) is missing when pulp tries to access it.

Note that it works fine using curl with the correct password.  

curl -X GET http://google.com/  --proxy http://admin:redhat@1.2.3.4:8888

It very well could be my proxy configuration, however I would expect it to work with basic or digest auth. 



If it 'works', it would be great to get some instructions for setting up a proxy requiring authentication that works with pulp.
Comment 1 Sayli Karmarkar 2013-10-23 14:10:37 EDT
This was a bug in python-requests when using the documented way to specify proxy authentication. They are working on fixing the issue, but meanwhile, we found an alternate way of handling this and implemented it.  https://github.com/pulp/nectar/pull/12/
Comment 2 Jeff Ortel 2013-10-23 19:27:14 EDT
build: 2.3.0-0.24.beta
Comment 3 Preethi Thomas 2013-10-24 16:40:18 EDT
seems like this is still an issue

fails-qa

[root@qe-blade-03 ~]#  pulp-admin rpm repo create --repo-id rhel64-ks --feed https://cdn.redhat.com/content/dist/rhel/rhui/server/6/6.4/x86_64/kickstart/ --feed-ca-cert CDN/cdn.redhat.com-chain.crt --feed-cert CDN/1359391926_4512.crt  --feed-key CDN/1359391926_4512.key --skip rpm,erratum  --proxy-host http://pulp-proxy.usersys.redhat.com --proxy-port 8080
Successfully created repository [rhel64-ks]

[root@qe-blade-03 ~]# 
[root@qe-blade-03 ~]# 
[root@qe-blade-03 ~]# pulp-admin rpm repo sync run --repo-id rhel64-ks
+----------------------------------------------------------------------+
                  Synchronizing Repository [rhel64-ks]
+----------------------------------------------------------------------+

This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.

Downloading metadata...
[-]
... failed

Cannot connect to proxy. Socket error: Tunnel connection failed: 407 Proxy
Authentication Required.

[root@qe-blade-03 ~]# 


Without proxy auth

[root@qe-blade-03 ~]#  pulp-admin rpm repo create --repo-id rhel64-os --feed https://cdn.redhat.com/content/dist/rhel/rhui/server/6/6.4/x86_64/os/ --feed-ca-cert CDN/cdn.redhat.com-chain.crt --feed-cert CDN/1359391926_4512.crt  --feed-key CDN/1359391926_4512.key --skip rpm,erratum  --proxy-host http://pulp-upgrade-testing.usersys.redhat.com --proxy-port 3128
Successfully created repository [rhel64-os]

[root@qe-blade-03 ~]# 
[root@qe-blade-03 ~]# 
[root@qe-blade-03 ~]# pulp-admin rpm repo sync run --repo-id rhel64-os+----------------------------------------------------------------------+
                  Synchronizing Repository [rhel64-os]
+----------------------------------------------------------------------+

This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.

Downloading metadata...
[/]
... completed

Downloading repository content...
[==================================================] 100%
RPMs:       0/0 items
Delta RPMs: 0/0 items

^C[root@qe-blade-03 ~]#
Comment 4 Jeff Ortel 2013-10-29 11:29:46 EDT
build: 2.3.0-0.26.beta
Comment 5 Preethi Thomas 2013-10-30 16:38:01 EDT
[root@dell-pe2900-01 ~]# pulp-admin rpm repo create --repo-id rhel59-os --feed https://cdn.redhat.com/content/dist/rhel/rhui/server/5/5.9/x86_64/os/ --feed-ca-cert CDN/cdn.redhat.com-chain.crt --feed-cert CDN/1359391926_4512.crt  --feed-key CDN/1359391926_4512.key  --proxy-host http://pulp-proxy.usersys.redhat.com --proxy-port 8888 --proxy-user admin --proxy-pass admin 
Successfully created repository [rhel59-os]

[root@hp-sl2x170zg6-01 ~]# pulp-admin rpm repo sync run --repo-id rhel59-os
+----------------------------------------------------------------------+
                  Synchronizing Repository [rhel59-os]
+----------------------------------------------------------------------+


This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.

Downloading metadata...
[|]
... completed

Downloading repository content...
[==================================================] 100%
RPMs:       1623/1623 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 19/19 items
... completed

Importing errata...
[|]
... completed

Importing package groups/categories...
[|]
... completed

Publishing distributions...
[==================================================] 100%
Distributions: 19/19 items
... completed

Publishing packages...
[==================================================] 100%
Packages: 15362/15362 items
... completed

Generating metadata
[\]
... completed

Publishing repository over HTTPS
[-]
... completed

[root@hp-sl2x170zg6-01 ~]# 
[root@hp-sl2x170zg6-01 ~]#
Comment 6 Preethi Thomas 2013-10-31 10:25:13 EDT
verified

[root@hp-sl2x170zg6-01 ~]# rpm -qa pulp-server
pulp-server-2.3.0-0.26.beta.el6.noarch
[root@hp-sl2x170zg6-01 ~]# 

[root@hp-sl2x170zg6-01 ~]# pulp-admin rpm repo create --repo-id rhel62-os --feed https://cdn.redhat.com/content/dist/rhel/rhui/server/6/6.2/x86_64/os/ --feed-ca-cert CDN/cdn.redhat.com-chain.crt --feed-cert CDN/1359391926_4512.crt  --feed-key CDN/1359391926_4512.key
Successfully created repository [rhel62-os]


[root@hp-sl2x170zg6-01 ~]# pulp-admin rpm repo sync run --repo-id rhel62-os+----------------------------------------------------------------------+
                  Synchronizing Repository [rhel62-os]
+----------------------------------------------------------------------+

This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.

Downloading metadata...
[\]
... failed

Cannot connect to proxy. Socket error: Tunnel connection failed: 407 Proxy
Authentication Required.



[root@hp-sl2x170zg6-01 ~]# pulp-admin rpm repo sync run --repo-id rhel62-os
+----------------------------------------------------------------------+
                  Synchronizing Repository [rhel62-os]
+----------------------------------------------------------------------+

This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.

Downloading metadata...
[-]
... completed

Downloading repository content...
[==================================================] 100%
RPMs:       7281/7281 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 6/6 items
... completed

Importing errata...
[/]
... completed

Importing package groups/categories...
[|]
... completed

Publishing distributions...
[==================================================] 100%
Distributions: 6/6 items
... completed

Publishing packages...
[==================================================] 100%
Packages: 7281/7281 items
... completed

Generating metadata
[/]
... completed

Publishing repository over HTTPS
[-]
... completed

[root@hp-sl2x170zg6-01 ~]# cat /etc/pulp/server/plugins.conf.d/yum_importer.json 

{
    "num_threads" : 4,
    "proxy_host" : "http://pulp-proxy.usersys.redhat.com",
    "proxy_username": "admin",
    "proxy_password": "admin",
    "proxy_port" : 8888
}
[root@hp-sl2x170zg6-01 ~]# 



[root@hp-sl2x170zg6-01 ~]# pulp-admin rpm repo create --repo-id zoo --feed http://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/zoo/
Successfully created repository [zoo]

[root@hp-sl2x170zg6-01 ~]# 
[root@hp-sl2x170zg6-01 ~]# 
[root@hp-sl2x170zg6-01 ~]# 
[root@hp-sl2x170zg6-01 ~]# pulp-admin rpm repo sync run --repo-id zoo +----------------------------------------------------------------------+
                     Synchronizing Repository [zoo]
+----------------------------------------------------------------------+

This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.

Downloading metadata...
[\]
... completed

Downloading repository content...
[==================================================] 100%
RPMs:       32/32 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed

Importing errata...
[-]
... completed

Importing package groups/categories...
[-]
... completed

Publishing packages...
[==================================================] 100%
Packages: 32/32 items
... completed

Publishing distributions...
[==================================================] 100%
Distributions: 0/0 items
... completed

Generating metadata
[\]
... completed

Publishing repository over HTTPS
[-]
... completed

[root@hp-sl2x170zg6-01 ~]#
Comment 7 Preethi Thomas 2013-12-09 09:30:57 EST
Pulp 2.3 released.

Note You need to log in before you can comment on or make changes to this bug.