Bug 589132

Summary: File scheme breaks with proxy environment vars set
Product: Red Hat Enterprise Linux 6 Reporter: Simon Matter <simon.matter>
Component: curlAssignee: Kamil Dudka <kdudka>
Status: CLOSED CURRENTRELEASE QA Contact: Milos Malik <mmalik>
Severity: high Docs Contact:
Priority: high    
Version: 6.0CC: ebenes, jhutar, ovasik, syeghiay
Target Milestone: betaKeywords: Regression
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: curl-7.19.7-15.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-07-02 18:57:36 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
backport of upstream commit 3111701c38ee4d15df8e
ovasik: review+
backport of upstream commit 7603a29fc3db05d354b5 ovasik: review+

Description Simon Matter 2010-05-05 13:25:52 UTC
Description of problem:
Import of GPG keys fail with the first yum call if certain proxy environment vars are set.

Version-Release number of selected component (if applicable):
yum-3.2.25-12.el6.noarch

How reproducible:
Always

Steps to Reproduce:
1. Make sure you are using http scheme in your repos (RHEL6 defaults to ftp), like

baseurl=http://ftp.redhat.com/pub/redhat/rhel/beta/$releasever/$basearch/os/

2. Set proxy vars:

export http_proxy="http://proxy:8080"
export all_proxy="http://proxy:8080"

3. yum clean all
4. rpm -e gpg-pubkey-f21541eb-4a5233e7
5. rpm -e xterm
6. yum install xterm
  
Actual results:
GPG key retrieval failed: [Errno 14] Unknown Error: URL=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-beta-2 , scheme=file

Expected results:
The GPG key should install as well as the xterm package

Additional info:
Calling "yum install xterm" a second time makes it install without error.
It's also seems that it only happens with more than one environment var. We set a number of vars for a number of different tools and that's where yum has a problem. If only http_proxy is set then it works.

Comment 2 RHEL Program Management 2010-05-05 15:00:00 UTC
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux major release.  Product Management has requested further
review of this request by Red Hat Engineering, for potential inclusion in a Red
Hat Enterprise Linux Major release.  This request is not yet committed for
inclusion.

Comment 3 James Antill 2010-06-08 03:50:13 UTC
I'm going to move this to curl, as IIRC yum and urlgrabber don't do anything with the env. vars.

Comment 4 Kamil Dudka 2010-06-08 10:41:35 UTC
Please provide a (lib)curl based minimal example.  I have really no idea what '[Errno 14] Unknown Error' means and how to debug yum.  I've been playing with that variables using curl(1) and they seemed to work as documented:

http://curl.haxx.se/libcurl/c/curl_easy_setopt.html#CURLOPTPROXY

http://curl.haxx.se/libcurl/c/libcurl-tutorial.html#Proxies

Comment 5 seth vidal 2010-06-08 13:13:02 UTC
Kamil,
 if you want to get debug output from urlgrabber (which is what yum uses) then just set URLGRABBER_DEBUG=1
 on the shell and run the command, you'll get detailed results.

Also - if you want the error codes from urlgrabber to be better then I'd ask you to improve those in pycurl.

thanks

Comment 6 Kamil Dudka 2010-06-08 13:29:10 UTC
Simon, please run yum with the environment variable URLGRABBER_DEBUG set to 1 as Seth suggests and attach the verbose output to this bug.  I'll have a look.  Thanks!

(In reply to comment #5)
> Also - if you want the error codes from urlgrabber to be better then I'd ask
> you to improve those in pycurl.

You're probably asking at wrong place.  I've never used the python binding myself and can't speak python.  I can imagine some people which would do this job better than me ;-)

Comment 7 Simon Matter 2010-06-14 06:12:47 UTC
Kamil, could you try to reproduce this on a machine in your environment? Would be really nice because I don't have the box anymore where I have made the RHEL6 beta installation and I can't just do it again for now.

Comment 8 Kamil Dudka 2010-06-14 06:52:42 UTC
Simon, if you're not able to reproduce it, I doubt anybody else is.  The steps to reproduce are a bit vague.  I can of course give it a try, but need at least some more info from you:

1. NVR of libcurl

2. What exactly did you change in yum's configuration?

3. Is "http://proxy:8080" a working proxy?  If so, where is it placed?  What software is running in there?

Comment 9 Simon Matter 2010-06-14 07:18:34 UTC
Kamil, you should be able to reproduce it without problems. I have spent hours [ yes really :) ] to find out how exactly it fails. Because usually when something fails that way you just try it again - and in this case it works then - and forget about it.

All versions of the software are the packages from RHEL6 beta1.

For your questions:

1) This libcurl was used
ftp://ftp.redhat.com/pub/redhat/rhel/beta/6/i386/os/Packages/libcurl-7.19.7-6.el6.i686.rpm

2) I don't have the yum config anymore but I know I have only changed the repo file of the base repo to use http instead of ftp scheme, like so

- baseurl=ftp://ftp.redhat.com/pub/redhat/rhel/beta/$releasever/$basearch/os/
+ baseurl=http://ftp.redhat.com/pub/redhat/rhel/beta/$releasever/$basearch/os/

no other changes have been made to yum config. Also no proxy has been set on yum.conf because we have all our proxy settings in env vars.

3) That's correct, "http://proxy:8080" is our inhouse working proxy (which is why the name is not FQDN). It's placed in one of our internal RFC918 networks, clients do not have any direct internet connection, they can only use the proxy. The proxy is running squid on EL4 in a quite standard configuration.

The problem doesn't show up with downloading rpm packages or yum metadata but only with the initial import of the GPG key which happens automatically.
I think you should be able to reproduce the issue even with no running proxy available by just doing the steps 1 to 6 as described in the how to reproduce section.

Comment 10 Kamil Dudka 2010-06-16 14:36:54 UTC
Indeed, I can reproduce it independently on yum:

$ all_proxy="http://proxy:8080" curl file:///etc/yum.conf
curl: (5) Couldn't resolve proxy 'proxy'

It's a bug, which was fixed in libcurl-7.20.0:

http://github.com/bagder/curl/commit/3111701c

I'll prepare a backport...

Comment 11 Kamil Dudka 2010-06-17 08:32:31 UTC
Created attachment 424720 [details]
backport of upstream commit 3111701c38ee4d15df8e

Comment 12 Kamil Dudka 2010-06-17 09:34:09 UTC
still not perfect, test1101 is broken by the patch:

http://github.com/bagder/curl/commit/7603a29

Comment 13 Kamil Dudka 2010-06-17 11:30:21 UTC
Created attachment 424777 [details]
backport of upstream commit 7603a29fc3db05d354b5

With both patches applied, the test-suite runs fine, including the new test-case 1106.  The test from comment #10 still works.

Comment 14 Fedora Update System 2010-06-17 11:34:45 UTC
curl-7.19.7-11.fc12 has been submitted as an update for Fedora 12.
http://admin.fedoraproject.org/updates/curl-7.19.7-11.fc12

Comment 17 Kamil Dudka 2010-06-18 19:25:01 UTC
built as curl-7.19.7-15.el6

Comment 19 releng-rhel@redhat.com 2010-07-02 18:57:36 UTC
Red Hat Enterprise Linux Beta 2 is now available and should resolve
the problem described in this bug report. This report is therefore being closed
with a resolution of CURRENTRELEASE. You may reopen this bug report if the
solution does not work for you.

Comment 20 Fedora Update System 2010-07-06 17:29:47 UTC
curl-7.20.1-3.fc13 has been pushed to the Fedora 13 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 21 Fedora Update System 2010-07-14 23:05:40 UTC
curl-7.19.7-12.fc12 has been pushed to the Fedora 12 stable repository.  If problems still persist, please make note of it in this bug report.