Created attachment 331192 [details] File: /etc/yum.repos.d/fedora-rawhide.repo Description of problem: Fedora 11 Alpha by default specifies an https url for metalink in fedora-rawhide.repo. However, yum seems not to support https links when an http proxy is specified (Proxy server is https too, ie allows CONNECT method). As a result the following error message is shown: Error: Cannot retrieve repository metadata (repomd.xml) for repository: rawhide. Please verify its path and try again Version-Release number of selected component (if applicable): Tested on yum-3.2.20-5.fc8 and yum-3.2.21-8.fc11.noarch How reproducible: always Steps to Reproduce: 1.Specify http/https proxy in yum in fc11 Alpha default install (I used LiveCD). 2.Do "yum search xyz" etc. 3.Tried same in Fedora 10 after changing http:// to https://. Same error for metalink url. Did the same for baseurl: this time an additional error was generated along with the previous one: [Errno 14] HTTP Error 501: Not Implemented. Actual results: Yum error: Error: Cannot retrieve repository metadata (repomd.xml) for repository: rawhide. Please verify its path and try again Expected results: yum shuold have worked. Additional info: Problem can be resolved just by changing all https links to http.
What is in your yum.conf and/or set for the *_proxy env. variables?
Created attachment 331197 [details] yum.conf
Attached yum.conf. I haven't set any enviroment variables for proxy.
I'm pretty sure this is related to bug #487101
Just my 2 cents: Regardless of configuring a proxy in either yum.conf or the environment, stracing yum on my current FC11 (preview) reveals, that it opens a connection to the proxy and then issues a GET https://mirrors.fedoraproject.org/metalink?repo=fedora-11&arch=i386 HTTP/1.1 which is totally wrong for https URLs (needs to issue a CONNECT isntead of a GET, then switch to SSL after CONNECTED response ...). Therefore, the proxy (squid in this case) responds with HTTP/1.0 400 Bad Request Reason is: yum (in yumRepo.py) uses urlgrabber which uses urllib2 which simply doesn't support https over proxies out of the box. See bzrlib's transport/http/_urllib2_wrappers.py for an example how that's done the right way. Cheers -Fritz
Same problem here.. At work we're behind a forced proxy, and I was having trouble figuring out what was going on until I found this bug.. Changing the mirror metalink URI from https to http allows me to update here. Any more info to provide to get this fixed? This is on an up-to-date F11 image as of this date.
BTW, even per bug #487101 I made sure that https_proxy (and HTTPS_PROXY) were set, no joy... Yum will not update unless I change mirrorlist=https:// to http://
At the moment I don't have a good answer for you. There are a couple of places where urlgrabber and python in general cannot handle certain kinds of proxies.
This bug appears to have been reported against 'rawhide' during the Fedora 11 development cycle. Changing version to '11'. More information and reason for this action is here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping
I was hit by this issue too. Fritz has the best description of the problem. When you specify a proxy server in the yum.conf configuration, It should use the CONNECT method to establish an https connection through a proxy. Changing the https to http worked around the problem. https would be better to use from a security standpoint though. We use the yum.conf settings: proxy=http://10.x.x.x/3128 (where 10.x.x.x is replaced with the real squid proxy server IP of course in our config)
I was also hit by this. As a quick fix would it not be possible to change all https URLs to http when a proxy variable is set? As a side issue I also found it odd that yum -d 10 didn't spit out any information about the URL it was trying to retrieve, or the proxy it was using before dying with the error about failing to retrieve repomd.xml. Seeing exactly what it was trying and at what stage it was failing would have made it easier to track down the problem. This bug title should also be changed to reflect the fact that this is now a bug in the released F11, not just the alpha.
This is the bug, and I assume the fix. Looks like 2.6.2 will not be broken. I guess we could backport it. Anyone who is having the problem want to try it? http://bugs.python.org/issue1424152
Re: Comment #12 Jemes, I can try it out. Please show me how. Seth, Since the python we have does not handle https proxy well. how about change *.repo back to http instead of https? We can move to https when https proxy in python of F11 is fixed properly.
*** Bug 501851 has been marked as a duplicate of this bug. ***
(In reply to comment #13) > Since the python we have does not handle https proxy well. how about > change *.repo back to http instead of https? We can move to https > when https proxy in python of F11 is fixed properly. Sounds like a good solution. In fact, this worked fairly well for me.
If anyone here is running on rawhide-ish and has a proxy that can test something new - please comment on this bug and I'll send you a link to something to test.
I can give it a try. (in the meantime i'm on official F11, but setting up a rawhide-ish VM, shouldn't take long) -Fritz
Grab the pkg from: http://lists.baseurl.org/pipermail/yum/2009-July/022868.html it is built for rawhide. Test it out. Let me know if anything changes for this problem. thanks
Works like a charm :-) I even got bold and rebuilt the src.rpm on a regular F11, installed it manually, changed the URLs in fedora.repo back to their original https://.., ran yum clean all and after that a yum update without any problems. Logs at at our proxy show, that yum now correctly uses CONNECT when requesting https URLs. Nice Job! -Fritz
Fritz, This is excellent news. Thanks, -sv
This package in koji (soon to be in rawhide, I hope) http://koji.fedoraproject.org/koji/buildinfo?buildID=125399 should solve it - I'm going to close this rawhide. Please reopen if this is still broken.
Can this fix not be released as an update for F11? There's a long way to go to EOL and it would be good to not have to manually change the repo definitions every time we install a new machine.
it might be able to be backported - but it needs more testing in rawhide first, I think. It brings in a new dependency (pycurl) so it does make things a bit harder for backporting.
Oh well, while working nicely for a few updates, meanwhile a strange bug has shown up: When fetching *big* files (> 20Mb), it fails with the following message and yum goes into a retry loop, fetching the same file over and over again from different mirrors ...: http://ftp.uni-kl.de/pub/linux/fedora/linux/updates/11/i386/kdegames-4.3.1-4.fc11.i586.rpm: [Errno 14] HTTP Error 200 : http://ftp.uni-kl.de/pub/linux/fedora/linux/updates/11/i386/kdegames-4.3.1-4.fc11.i586.rpm Trying other mirror. I tried disabling the proxy, but that does not seem to help. Currently, when this happens, i have to interrupt yum and download the relevant file manually with wget into /var/cache/yum/<reponame>/packages, then restart yum update .... Errno 14 is an EFAULT, so this seems to be some buffer overflow?! .... -Fritz PS: Can't reopen this bug, 'cause i'm not the owner ...
This is on python-urlgrabber of what version?
I'm using python-urlgrabber-3.9.0-3.fc11 (rebuild on F11 of your previously posted src.rpm). Actually, i just looked into grabber.py and realized, that Errno isn't meant to be a "standard" unix error code (as defined in errno.h), but some generic code of your grabber lib. So: Is there any way to convince yum to produce more verbose stack traces so i can find out the real reason? -Fritz
That particular bug is fixed by the newer python-urlgrabber in rawhide. errno is coming from pycurl and it is about a timeout that is a fixed length, rather than being based on inactivity. update to urlgrabber 3.9.0-8 and it goes away.
Thanks for the hint. Works well now.