Bug 1387622

Summary: add proxy auth method option
Product: [Fedora] Fedora Reporter: Alexander Kabakaev <kabakaev+bugzilla+redhat>
Component: dnfAssignee: rpm-software-management
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: medium    
Version: 26CC: arrarexcaravels, bilal05beny, fabio.venturi.linux, formisc, jan.brummer, jmracek, mblaha, mmraka, packaging-team-maint, rpm-software-management, vmukhame
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-05-29 12:00:57 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Description Alexander Kabakaev 2016-10-21 11:54:47 UTC
Description of problem:

Recently librepo introduced new option librepo.LRO_PROXYAUTHMETHODS (see link [1]) and deprecated the librepo.LRO_PROXYAUTH option (see link [2]).

Still, dnf uses [3] the LRO_PROXYAUTH=True, which does not work for me.
Even with configured proxy settings, dnf sends first request without auth header, receives "407 Proxy Authentication Required" response from the proxy server and closes connection.

For some reason, dnf/librepo/curl do not retry with any of the auth methods suggested by the proxy server.

It may be also an issue of librepo or curl, not sure. The very same symptoms were reported on Bluecoat proxy server and PHP: https://core.trac.wordpress.org/ticket/31232

I'm posting the bug against dnf, because I was able to fix auth with one line in dnf code, and this dirty fix can be used by anyone else struggling to connect behind similar proxy.
But apparently the root cause lies far beyond dnf, somewhere in between curl and the proxy server implementation. Any suggestions?

An obvious solution would be to add a new dnf option "proxy_auth_method", to allow direct configuration of LRO_PROXYAUTHMETHODS through dnf.conf.


Version-Release number of selected component (if applicable):
The issue is reproducible in fedora-24 (dnf-1.1.9-2.fc24) and fedora-rawhide (dnf-1.1.10-1.fc26).

How reproducible:
Always reproducible.

Steps to Reproduce:
1. Add proxy configuration in dnf config:
  # cat /etc/dnf/dnf.conf
  [main]
  gpgcheck=1
  installonly_limit=3
  clean_requirements_on_remove=True
  # proxy server IP = 10.11.12.13
  proxy=http://cisco-wsa.internal.net:3128
  proxy_username=myusername
  proxy_password=VeryStrongPassword

2. Run dnf under strace:
  # strace -s2000 dnf install vim > dnf-out-1 2>&1

3. Search ip address of the proxy server and find that unauthenticated CONNECT request is sent:
  # less dnf-out-1
connect(11, {sa_family=AF_INET, sin_port=htons(3128), sin_addr=inet_addr("10.11.12.13")}, 16) = -1 EINPROGRESS (Operation now in progress)
sendto(11, "CONNECT mirrors.fedoraproject.org:443 HTTP/1.1\r\nHost: mirrors.fedoraproject.org:443\r\nUser-Agent: dnf/1.1.10\r\n\r\n", 111, MSG_NOSIGNAL, NULL, 0) = 111
recvfrom(11, "HTTP/1.1 407 Proxy Authentication Required\r\nMime-Version: 1.0\r\nDate: Fri, 21 Oct 2016 08:34:02 GMT\r\nVia: 1.1 cisco-wsa.internal.net:80 (Cisco-WSA/9.1.1-074)\r\nContent-Type: text/html\r\nProxy-Authenticate: Negotiate\r\nProxy-Authenticate: NTLM\r\nProxy-Authenticate: Basic realm=\"Cisco IronPort Web Security Appliance\"\r\nConnection: close\r\nProxy-Connection: close\r\nContent-Length: 2088\r\n\r\nHere goes HTML error message"..., 16384, 0, NULL, NULL) = 2477
close(11)                               = 0


Actual results: authentication header is not sent, authentication is not even attempted.

Expected results: authenticated connection is attempted.

Now, let's specify explicitly in dnf code that we want to use Basic Auth.
The value of LRO_PROXYAUTHMETHODS is described in link [1].

[root@localhost ~]# diff -U3 /usr/lib/python3.5/site-packages/dnf/repo_orig.py /usr/lib/python3.5/site-packages/dnf/repo.py
--- /usr/lib/python3.5/site-packages/dnf/repo_orig.py   2016-10-21 08:42:45.261000000 +0000
+++ /usr/lib/python3.5/site-packages/dnf/repo.py        2016-10-21 08:47:07.186000000 +0000
@@ -664,7 +664,8 @@
         # apply repo options
         h.maxspeed = self.throttle if type(self.throttle) is int \
                      else int(self.bandwidth * self.throttle)
-        h.setopt(librepo.LRO_PROXYAUTH, True)
+        # h.setopt(librepo.LRO_PROXYAUTH, True)
+        h.setopt(librepo.LRO_PROXYAUTHMETHODS, 1)
         h.proxy = self.proxy
         h.lowspeedlimit = self.minrate
         if self.timeout > 0:

with the same proxy configuration in /etc/dnf/dnf.conf,
runnig dnf under strace is successful:
  # strace -s2000 dnf install vim > dnf-out-2 2>&1
  # less dnf-out-2
connect(10, {sa_family=AF_INET, sin_port=htons(3128), sin_addr=inet_addr("10.11.12.13")}, 16) = -1 EINPROGRESS (Operation now in progress)
sendto(10, "CONNECT alt.fedoraproject.org:443 HTTP/1.1\r\nHost: alt.fedoraproject.org:443\r\nProxy-Authorization: Basic BASE64-encoded-myusername:VeryStrongPassword\r\nUser-Agent: dnf/1.1.10\r\n\r\n", 180, MSG_NOSIGNAL, NULL, 0) = 180
recvfrom(10, "HTTP/1.1 200 Connection established\r\nDate: Fri, 21 Oct 2016 08:40:17 GMT\r\nVia: 1.1 cisco-wsa.internal.net:80 (Cisco-WSA/9.1.1-074)\r\n\r\n", 16384, 0, NULL, NULL) = 142

It works!

Now, let's try to use NTLM auth:

[root@localhost ~]# cat /etc/dnf/dnf.conf
[main]
gpgcheck=1
installonly_limit=3
clean_requirements_on_remove=True
proxy=http://cisco-wsa.internal.net:3128
proxy_username=windowsdomain\myusername
proxy_password=VeryStrongPassword

[root@localhost ~]# diff -U3 /usr/lib/python3.5/site-packages/dnf/repo_orig.py /usr/lib/python3.5/site-packages/dnf/repo.py
--- /usr/lib/python3.5/site-packages/dnf/repo_orig.py   2016-10-21 08:42:45.261000000 +0000
+++ /usr/lib/python3.5/site-packages/dnf/repo.py        2016-10-21 08:47:07.186000000 +0000
@@ -664,7 +664,8 @@
         # apply repo options
         h.maxspeed = self.throttle if type(self.throttle) is int \
                      else int(self.bandwidth * self.throttle)
-        h.setopt(librepo.LRO_PROXYAUTH, True)
+        # h.setopt(librepo.LRO_PROXYAUTH, True)
+        h.setopt(librepo.LRO_PROXYAUTHMETHODS, 8)
         h.proxy = self.proxy
         h.lowspeedlimit = self.minrate
         if self.timeout > 0:

[root@localhost ~]# strace -s2000 dnf -y install vim > dnf-out-3 2>&1
connect(10, {sa_family=AF_INET, sin_port=htons(3128), sin_addr=inet_addr("10.11.12.13")}, 16) = -1 EINPROGRESS (Operation now in progress)
sendto(10, "CONNECT alt.fedoraproject.org:443 HTTP/1.1\r\nHost: alt.fedoraproject.org:443\r\nProxy-Authorization: NTLM TlRMTVNTUAABAAAABoIIAAAAAAAAAAAAAAAAAAAAAAA=\r\nUser-Agent: dnf/1.1.10\r\n\r\n", 175, MSG_NOSIGNAL, NULL, 0) = 175
recvfrom(10, "HTTP/1.1 407 Proxy Authentication Required\r\nMime-Version: 1.0\r\nDate: Fri, 21 Oct 2016 08:50:18 GMT\r\nVia: 1.1 cisco-wsa.internal.net:80 (Cisco-WSA/9.1.1-074)\r\nContent-Type: text/html\r\nProxy-Authenticate: NTLM BASE64-encoded-challange\r\nConnection: keep-alive\r\nProxy-Connection: keep-alive\r\nContent-Length: 0\r\n\r\n", 16384, 0, NULL, NULL) = 557
uname({sysname="Linux", nodename="localhost.localdomain", ...}) = 0
sendto(10, "CONNECT alt.fedoraproject.org:443 HTTP/1.1\r\nHost: alt.fedoraproject.org:443\r\nProxy-Authorization: NTLM BASE64-encoded-response\r\nUser-Agent: dnf/1.1.10\r\n\r\n", 543, MSG_NOSIGNAL, NULL, 0) = 543
recvfrom(10, "HTTP/1.1 200 Connection established\r\nDate: Fri, 21 Oct 2016 08:50:18 GMT\r\nVia: 1.1 cisco-wsa.internal.net:80 (Cisco-WSA/9.1.1-074)\r\n\r\n", 16384, 0, NULL, NULL) = 142

Works again!

To sum up,
  * dnf does not authenticate against proxy, even if auth options are set;
  * it would be nice to add new dnf.conf option to be able to specify explicitly the proxy auth method.


Additional info:
[1] https://github.com/rpm-software-management/librepo/commit/bfc05df2f74ad7038977ce4c6b1fd367ec1da430#diff-963eed69ec97dd555106594053f52cf2R69

[2] https://github.com/rpm-software-management/librepo/commit/bfc05df2f74ad7038977ce4c6b1fd367ec1da430#diff-9cff5beb4a457ecb5636a70db528d172R159

[3] https://github.com/rpm-software-management/dnf/blob/0c1908cfafa5961924e48dc66225b31cdc34186e/dnf/repo.py#L668

Comment 1 Michael Mráka 2016-10-24 11:30:33 UTC

Some proxies won't authorize correctly with general setting, so ideally you'll want to specify the authentication method that your proxy uses.
A new option to specify proxy auth type should solve it.

Comment 2 Fedora End Of Life 2017-02-28 10:29:24 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 26 development cycle.
Changing version to '26'.

Comment 3 andrew 2017-04-27 18:50:23 UTC
What are the chances to see in in version 26?

Comment 4 arrarexcaravels 2017-09-22 09:10:37 UTC
I still get this error in fedora 26. The only thing that solves it for me is editing /usr/lib/python3.6/site-packages/dnf/repo.py and changing the following as described by Alexander: 

-        h.setopt(librepo.LRO_PROXYAUTH, True)
+        h.setopt(librepo.LRO_PROXYAUTHMETHODS, 8)

Adding the option to specify an authentication method, as Michael suggests, sounds like a good idea. 

Any ideas when this will be fixed?

Comment 5 arrarexcaravels 2017-10-23 12:45:02 UTC
I have to change this manually after every dnf update. I would assume all users behind an ntlm proxy are affected. 

When will this be fixed?

Comment 6 b1ng0 2017-12-21 09:26:13 UTC
dnf doesn't work  with ISA server ( NTLM ) in Fedora 27 !

Comment 7 Fabio Venturi 2017-12-21 16:30:45 UTC
Same problem with Fedora release 26,
DNF behind proxy is not working anymore
with proxy/credential as env variables or in yum.conf or in dnf.conf

Same credential works on Firefox to browse the web.

It feel like DNF is very very bad compared to YUM...

 - Fabio

Comment 8 Jan-Michael Brummer 2018-04-30 17:42:42 UTC
What's the recommended way to patch this issue? Do you want a pull request with this small change?

I've tested this patch on a private (no proxy) and on my company (proxy) machine, there are no obvious regression.

Comment 9 Jan-Michael Brummer 2018-04-30 18:08:17 UTC
Created PR: https://github.com/rpm-software-management/dnf/pull/1076

Comment 10 Fedora End Of Life 2018-05-03 08:47:01 UTC
This message is a reminder that Fedora 26 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 26. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '26'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 26 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged  change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

Comment 12 Fedora End Of Life 2018-05-29 12:00:57 UTC
Fedora 26 changed to end-of-life (EOL) status on 2018-05-29. Fedora 26
is no longer maintained, which means that it will not receive any
further security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.

Comment 13 Jaroslav Mracek 2018-06-28 08:15:55 UTC
The issue is solved by dnf-3.0.1-1 that was released into rawhide.