From Bugzilla Helper: User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.4) Gecko/20030703 Description of problem: I am testing the new up2date apt repository feature against arjan's 2.6 kernel apt repository and get the following when I run up2date -l Fetching package list for channel: redhat-linux-severn-i386-9.0.93... ######################################## Fetching package list for channel: arjan-2.6-kernel-i386-kernel... Fetching http://people.redhat.com/~arjanv/2.5//base/pkglist.kernel.bz2... {'User-Agent': 'Up2date 3.9/Apt'} There was some sort of I/O error: <urlopen error (110, 'Connection timed out')> My guess would be that the problem is because we have non-transparent http proxying (which works fine for up2date normally), and the new up2date features aren't trying to use the proxy settings - I can retrieve the pkglist.kernel.bz2 file directly with a browser.
Yup, the new stuff isnt hooked into the proxy config stuff yet... you should be able to work around it by setting a HTTP_PROXY enviroment variable (the apt/yum code uses normal python urllib2 stuff, so this should work)
This is still an issue with 4.1.5-1 , your workaround of setting http_proxy only works when up2date is run directly as root, not as an ordinary user; presumably consolehelper strips the environment for security reasons.
Just a note to confirm it seems to be console helper export http_proxy=http://some.proxy:80 sudo /usr/sbin/up2date -l works sudo up2date -l fails with I/O error (connection refused) as it's proxy only setup here.
Still there in up2date-4.1.16-1
up2date-4.1.16-1 : I am having problems with the Fedora release up2date versus the RedHat 9 up2date concerning a proxy with authentication. The RedHat 9 up2date works with an authenticated proxy, and the Fedora 1 up2date does not. [root@josiahdesk jroyse]# up2date --list Fetching package list for channel: fedora-core-1... Fetching http://fedora.redhat.com/releases/fedora-core-1/headers/header.info... There was some sort of I/O error: <urlopen error (-2, 'Name or service not known')> I copied the proxy settings from the /etc/sysconfig/rhn/up2date files from RH9 to FC1 to verify there wasn't a typo. The Proxy hasn't changed settings..
Is anyone working on this? It's been nearly three months since it was first reported, and no action has been taken. Does anyone have any ideas where to start?
Yes it's the repo specific code in repoBackends/ (either in src.rpm or in /usr/share/rhn/up2date_client. This really needs to be changed to not use urllib directly and use the wrapped httplib. Adrian is aware of this and I think he knows what to change, I imagine it's just a time issue. The yum/apt repo needs to grab the proxy from the config an use it in urllib is a short term fix. Note up2date on rhn still works with proxies so this is only on other repos. I believe that export http_proxy on the environment also fixes this which could be another hacky fix within up2date itself.
Created attachment 95866 [details] make yumRepo.py use (proxy-friendly) fetchURLAndWriteFd I have hacked the code so that yum works when going through a proxy that requires authentication. My settings: ======== [root@porta-pc root]# grep -i proxy /etc/sysconfig/rhn/up2date enableProxyAuth[comment]=To use an authenticated proxy or not enableProxyAuth=1 enableProxy[comment]=Use a HTTP Proxy enableProxy=1 proxyPassword[comment]=The password to use for an authenticated proxy proxyPassword=PASSWORD proxyUser[comment]=The username for an authenticated proxy proxyUser=johnm httpProxy[comment]=HTTP proxy in host:port format, e.g. squid.redhat.com:3128 httpProxy=http://proxy.monash.edu:80 ======== When I run it: ======== [root@porta-pc root]# up2date -u Fetching package list for channel: fedora-core-1... Fetching http://fedora.redhat.com/releases/fedora-core-1/headers/header.info... ######################################## yum http://fedora.redhat.com/releases/fedora-core-1/headers/xcin-0-2.5.3.pre3-17 .i386.hdr /var/spool/up2date/xcin-2.5.3.pre3-17.i386.hdr ####################################### Fetching package list for channel: updates-released... Fetching http://fedora.redhat.com/updates/released/fedora-core-1/headers/header. info... Fetching Obsoletes list for channel: fedora-core-1... Fetching obsoletes list for http://fedora.redhat.com/releases/fedora-core-1... ####################################### Fetching Obsoletes list for channel: updates-released... Fetching obsoletes list for http://fedora.redhat.com/updates/released/fedora-cor e-1... Fetching rpm headers... Name Version Rel ---------------------------------------------------------- All packages are currently up to date ======== I have no idea why it fetches the xcin header -- which is why I left the debug print in the patch. Different Bug?
triage->easyfix Looks sane to me - Adrian?
Working behind an ISA (Microsoft) auth proxy, the above patch was applied, and does not improve my situation. # up2date -l: There was some sort of I/O error: <urlopen error (-2, 'Name or service not known')> # yum list: [Errno 7] HTTP Error (CannotSendRequest)
*** Bug 107036 has been marked as a duplicate of this bug. ***
*** Bug 110989 has been marked as a duplicate of this bug. ***
bug 107313 and bug 110168 are also duplicates of this bug. bug 109195 and bug 110284 are the same, but with authentication added.
*** Bug 107313 has been marked as a duplicate of this bug. ***
*** Bug 110168 has been marked as a duplicate of this bug. ***
*** Bug 109195 has been marked as a duplicate of this bug. ***
*** Bug 110284 has been marked as a duplicate of this bug. ***
*** Bug 111523 has been marked as a duplicate of this bug. ***
As an aside here, I've been experiencing really flaky up2date downloads lately, especially with sizable packages like kernel-source. The sooner this proxy thing is fixed, the sooner the update server will benefit from those of us who can use a caching proxy to lower demands on fedora bandwidth!
You can already lower fedora bandwidth and probably get better performance by configuring up2date to use a mirror.
Is there another solution (apart from the supplied patch) if the proxy requires authentication - are there env vars that can be set for proxy user and proxy password?
Try to set these env vars : export http_proxy=http://proxy:3128 export proxy_user=username export proxy_passwd=password on my case this solution works fine!!! If u need further help, I'm really pleased to help U. Have a nice day Ale
*** Bug 114154 has been marked as a duplicate of this bug. ***
I tried to reproduce the suggestions from bug 102272 comment #3 - running up2date as root: -- up2date -l Fetching package list for channel: redhat-linux-i386-9... ######################################## Fetching package list for channel: fedora-core-1... Fetching http://fedora.redhat.com/releases/fedora-core-1/headers/header.info... There was some sort of I/O error: <urlopen error (-3, 'Temporary failure in name resolution')> --- Running with full pathname results in: --- /usr/sbin/up2date -l Fetching package list for channel: redhat-linux-i386-9... ######################################## Fetching package list for channel: fedora-core-1... Fetching http://fedora.redhat.com/releases/fedora-core-1/headers/header.info... Traceback (most recent call last): File "/usr/sbin/up2date", line 1165, in ? sys.exit(main() or 0) File "/usr/sbin/up2date", line 745, in main fullUpdate, dryRun=options.dry_run)) File "/usr/sbin/up2date", line 1028, in batchRun batch.run() File "up2dateBatch.py", line 57, in run File "up2dateBatch.py", line 96, in __findPackagesToUpdate File "packageList.py", line 117, in run File "rhnPackageInfo.py", line 316, in getAvailableAllArchPackageList File "rhnPackageInfo.py", line 156, in availablePackageList File "rpcServer.py", line 111, in doCall File "repoDirector.py", line 26, in listPackages File "rpmSource.py", line 226, in listPackages File "/usr/share/rhn/up2date_client/repoBackends/yumRepo.py", line 197, in lis tPackages agent = "Up2date 3.9/Yum") File "/usr/share/rhn/up2date_client/repoBackends/urlUtils.py", line 257, in fe tchUrl agent = agent) File "/usr/share/rhn/up2date_client/repoBackends/urlUtils.py", line 98, in ope n_resource return opener.open(request) File "/usr/lib/python2.2/urllib2.py", line 328, in open '_open', req) File "/usr/lib/python2.2/urllib2.py", line 307, in _call_chain result = func(*args) File "/usr/lib/python2.2/urllib2.py", line 482, in <lambda> lambda r, proxy=url, type=type, meth=self.proxy_open: \ File "/usr/lib/python2.2/urllib2.py", line 489, in proxy_open if '@' in host: TypeError: iterable argument required ---
*** Bug 115618 has been marked as a duplicate of this bug. ***
Is this going to be fixed for FC2? Has anyone looked at Comment #7 about using httplib instead of urllib? I'd love to see this fixed for FC2, so if anyone has any suggestions, let me know...
Regarding comment 24, it seems you have forgotten to specify the protocol in the proxy setting. It should be e.g.: export http_proxy=http://proxy.domain.com:8080 and not export http_proxy=proxy.domain.com:8080 I also just noticed that up2date only fails on me when I call it as the normal user, i.e. when run through consolehelper. When I run it from a root-shell, it uses the proxy just fine. I guess this is due to *_proxy being set in the root-shell, but not when run through consolehelper. --> Somehow up2date (the underlying lib) uses the proxy correctly when *_proxy is set, but misses something (does one DNS lookup too much?) if it only has proxy settings in /etc/sysconfig/rhn/up2date.
Regarding comment 27: Hmmmmmmmmmmmm.....................no! --- $# export http_proxy=http://proxy:8080 $# /usr/sbin/up2date -l Fetching package list for channel: redhat-linux-i386-9... ######################################## Fetching package list for channel: fedora-core-1... Fetching http://fedora.redhat.com/releases/fedora-core-1/headers/header.info... Traceback (most recent call last): File "/usr/sbin/up2date", line 1165, in ? sys.exit(main() or 0) File "/usr/sbin/up2date", line 745, in main fullUpdate, dryRun=options.dry_run)) File "/usr/sbin/up2date", line 1028, in batchRun batch.run() File "up2dateBatch.py", line 57, in run File "up2dateBatch.py", line 96, in __findPackagesToUpdate File "packageList.py", line 117, in run File "rhnPackageInfo.py", line 316, in getAvailableAllArchPackageList File "rhnPackageInfo.py", line 156, in availablePackageList File "rpcServer.py", line 111, in doCall File "repoDirector.py", line 26, in listPackages File "rpmSource.py", line 226, in listPackages File "/usr/share/rhn/up2date_client/repoBackends/yumRepo.py", line 197, in lis tPackages agent = "Up2date 3.9/Yum") File "/usr/share/rhn/up2date_client/repoBackends/urlUtils.py", line 270, in fe tchUrl buffer = readFDBuf(fh, progressCallback) File "/usr/share/rhn/up2date_client/repoBackends/urlUtils.py", line 240, in re adFDBuf size = get_size(fd) File "/usr/share/rhn/up2date_client/repoBackends/urlUtils.py", line 231, in ge t_size return long(size) TypeError: long() argument must be a string or a number
in addition co comment 28: Note, that these errors only occur while trying to access the fedora core. I still have a "backup" sources file (/etc/sysconfig/rhn/resources) which differs from the fedora sources file only in: --- 11c11,12 < #up2date default --- > ### This is required. > up2date default 30,32c31,32 < yum fedora-core-1 http://fedora.redhat.com/releases/fedora-core-1 < yum updates-released http://fedora.redhat.com/updates/released/fedora-core-1 < yum updates-testing http://fedora.redhat.com/updates/testing/fedora-core-1 --- > #yum fedora-9-i386-stable http://download.fedora.us/fedora/redhat/9/i386/yum/stable > --- So accessing the redhat-9 channel still works, even without specifying http_proxy. Volker.
I applied John Mann's patch to the yumRepo.py file that came with up2date-4.1.21-3 on Fedora Core 1 and am still unable to use an authenticating proxy. A tcpdump trace shows that the required Proxy-Authorization: header for Basic authentication is missing from the HTTP GET: statement. I have the username/password defined in both the up2date config file as well as in environment variables and have also tried on the command line. It looks as though the user/pass are simply getting ignored by up2date. Any suggestions from fellow bug-watchers? I plan to explore how the fetchUrlAndWriteFD call gets its proxy login credentials. -- Steve
After a nice tour of the python code I'm pretty convinced that the Yum backend in urlUtils.py simply does not support Basic HTTP authentication. The open_resource call (where fetchUrlAndWriteFD ends up) does not have any facility for a Proxy-Authentication: header. (In fact, a grep through the whole up2date and yum files shows that this string doesn't appear anywhere.) There may be some hope for getting urllib2 working since it supports several forms of authentication: http://www.python.org/doc/current/lib/module-urllib2.html For those browsing through this bug who just want to get their updates, head over to http://www.freshrpms.net/apt and install the RPM version of apt. Edit the /etc/apt/apt.conf file's "Proxy" line, unset any http_proxy environment variable, and "apt-get upgrade" should magically work. -- Steve
*** Bug 118672 has been marked as a duplicate of this bug. ***
*** Bug 123945 has been marked as a duplicate of this bug. ***
I'm not sure if I have the "same" error or not. Yum worked fine with my proxy until the repo.d change (2.1.11?). Now, it doesn't seem to even try to authenticate, despite correctly set environment variables. If the "mirrors" lines are active, it hangs with no output at all, while if I set a baseurl instead, I get a printout of the first repo. Despite update to FC3, the problem still persists.
In my case, /usr/sbin/userhelper doesn't pass http_proxy to up2date.
Hi, Seems I'm suffering the same (or a similar problem). I'm using 2 PCs with clean installs of FC3. One has a modem, and can "up2date" happily when dialled up. Neither can "up2date" over my proxy (the AnalogX proxy running on win2k) - which, incidentally, I'm submitting this comment over - FireFox is happy with it. I've tried all the obvious combinations of RHN GUI settings, environment vars and command line args - to no avail. I get as far as the "Fetching http://..../header.info" message, and up2date freezes. I've had a go at using urllib2 on the Python command line to fetch a file or 2 over http via the proxy - that works OK. Any suggestions (other than apt/Synaptic)? cheers John Dubery jdubery.uk p.s. In case it's of use, here's a typical Python traceback... [root@fedora1 john]# export http_proxy=http://192.168.1.3:6588 [root@fedora1 john]# up2date /usr/share/rhn/up2date_client/checklist.py:21: DeprecationWarning: use gtk.TreeView gtk.CList.__init__ (self, columns+1) /usr/lib/python2.3/site-packages/gtk-2.0/gtk/__init__.py:90: GtkDeprecationWarning: gtk.mainloop is deprecated, use gtk.main instead self.warn(message, DeprecationWarning) http://fedora.redhat.com/download/up2date-mirrors/fedora-core-3 http://fedora.redhat.com/download/up2date-mirrors/updates-released-fc3 /usr/lib/python2.3/site-packages/gtk-2.0/gtk/__init__.py:90: GtkDeprecationWarning: gtk.create_pixmap is deprecated, use gtk.gdk.Pixmap instead self.warn(message, DeprecationWarning) /usr/lib/python2.3/site-packages/gtk-2.0/gtk/__init__.py:90: GtkDeprecationWarning: gtk.mainiteration is deprecated, use gtk.main_iteration instead self.warn(message, DeprecationWarning) Traceback (most recent call last): File "/usr/share/rhn/up2date_client/gui.py", line 1434, in onChannelsPageNext self.pList.run() File "/usr/share/rhn/up2date_client/packageList.py", line 122, in run self.progressCallback) File "/usr/share/rhn/up2date_client/rhnPackageInfo.py", line 315, in getAvailableAllArchPackageList package_list = availablePackageList( File "/usr/share/rhn/up2date_client/rhnPackageInfo.py", line 153, in availablePackageList progressCallback = progressCallback) File "/usr/share/rhn/up2date_client/rpcServer.py", line 310, in doCall ret = apply(method, args, kwargs) File "/usr/share/rhn/up2date_client/repoDirector.py", line 20, in listPackages return self.handlers[channel['type']].listPackages(channel, msgCallback, progressCallback) File "/usr/share/rhn/up2date_client/rpmSource.py", line 226, in listPackages msgCallback, progressCallback) File "/usr/share/rhn/up2date_client/repoBackends/yumRepo.py", line 201, in listPackages agent = "Up2date %s/Yum" % up2dateUtils.version()) File "/usr/share/rhn/up2date_client/repoBackends/urlUtils.py", line 324, in fetchUrl endRange=end) File "/usr/share/rhn/up2date_client/repoBackends/urlUtils.py", line 117, in open_resource return opener.open(request) File "/usr/lib/python2.3/urllib2.py", line 326, in open '_open', req) File "/usr/lib/python2.3/urllib2.py", line 306, in _call_chain result = func(*args) File "/usr/lib/python2.3/urllib2.py", line 901, in http_open return self.do_open(httplib.HTTP, req) File "/usr/lib/python2.3/urllib2.py", line 886, in do_open raise URLError(err) urllib2.URLError: <urlopen error (-3, 'Temporary failure in name resolution')> Traceback (most recent call last): File "/usr/share/rhn/up2date_client/gui.py", line 1500, in onSkippedPagePrepare self.__preparePackageList() File "/usr/share/rhn/up2date_client/gui.py", line 1386, in __preparePackageList self.pList.run() File "/usr/share/rhn/up2date_client/packageList.py", line 122, in run self.progressCallback) File "/usr/share/rhn/up2date_client/rhnPackageInfo.py", line 315, in getAvailableAllArchPackageList package_list = availablePackageList( File "/usr/share/rhn/up2date_client/rhnPackageInfo.py", line 153, in availablePackageList progressCallback = progressCallback) File "/usr/share/rhn/up2date_client/rpcServer.py", line 310, in doCall ret = apply(method, args, kwargs) File "/usr/share/rhn/up2date_client/repoDirector.py", line 20, in listPackages return self.handlers[channel['type']].listPackages(channel, msgCallback, progressCallback) File "/usr/share/rhn/up2date_client/rpmSource.py", line 226, in listPackages msgCallback, progressCallback) File "/usr/share/rhn/up2date_client/repoBackends/yumRepo.py", line 201, in listPackages agent = "Up2date %s/Yum" % up2dateUtils.version()) File "/usr/share/rhn/up2date_client/repoBackends/urlUtils.py", line 324, in fetchUrl endRange=end) File "/usr/share/rhn/up2date_client/repoBackends/urlUtils.py", line 117, in open_resource return opener.open(request) File "/usr/lib/python2.3/urllib2.py", line 326, in open '_open', req) File "/usr/lib/python2.3/urllib2.py", line 306, in _call_chain result = func(*args) File "/usr/lib/python2.3/urllib2.py", line 901, in http_open return self.do_open(httplib.HTTP, req) File "/usr/lib/python2.3/urllib2.py", line 886, in do_open raise URLError(err) urllib2.URLError: <urlopen error (-3, 'Temporary failure in name resolution')>
Updating the OS from fc1 to fc4t2 as it is still broken.
who should be prodded to fix this (going on 2 years) old bug?
Created attachment 114116 [details] YUM repo proxy fix for up2date Contains 3 patched source files that replace the existing up2date source code.
Created attachment 114117 [details] YUM repo proxy fix for up2date Contains 3 patched source files that replace the existing up2date source code.
Comment on attachment 114116 [details] YUM repo proxy fix for up2date Sorry, wrong MIME type - I've re-attached it with the correct type. Can't see a way to delete this one.
Hi, I've become sufficiently annoyed by this one to do the fix for YUM repos (time and a lack of need have resulted in APT being unchanged). I've tried to make it un-hack-ish enough to use as-is. Three files have been changed - urlUtils.py, urlMirrors.py, yumRepo.py - I've just attached a ZIP with them in (sorry, no means of doing a diff from here). I've also addressed a couple of FIXMEs around the code I've changed. cheers, John Dubery
Created attachment 115057 [details] Standard patch file Attachment is a patch file generated by running: diff -ur old new Patch can be applied by running: patch -p1 -b -d /usr/share/rhn/up2date_client < patch.diff (the -b creates backup files)
Created attachment 115059 [details] Clean patch without print statements I attached a clean version of the patch which removes the print statements John added. This is the patch I would use for a production system. As before, to apply the patch run: patch -p1 -b -d /usr/share/rhn/up2date_client < patch.diff
The issue we are seeing is that since our internal DNS servers do not resolve outside addresses, the DNS lookup that yum does to resolve the repository name fails. I would think that yum should be able to be configured to use the proxy for DNS lookups. That would mitigate our issue.
(In reply to comment #45) > The issue we are seeing is that since our internal DNS servers do not resolve > outside addresses, the DNS lookup that yum does to resolve the repository name > fails. I would think that yum should be able to be configured to use the proxy > for DNS lookups. That would mitigate our issue. Sorry, forgot to include the error output... yum update Setting up Update Process Setting up Repo: rhel4u3ws repomd.xml 100% |=========================| 951 B 00:00 Setting up Repo: dag http://apt.sw.be/redhat/el4/en/i386/dag/repodata/repomd.xml: [Errno 4] IOError: <urlopen error (-2, 'Name or service not known')> Trying other mirror. Cannot open/read repomd.xml file for repository: dag failure: repodata/repomd.xml from dag: [Errno 256] No more mirrors to try.
Note that FC2 is no longer supported even by Fedora Legacy. Also, up2date has been replaced by pirut and pup since FC5. FC3 and FC4 are supported by Fedora Legacy for security issues only. If this still occurs on FC3 or FC4 and is a security issue, please reopen and assign to that version and Fedora Legacy. If it occurs on RHEL 3 or 4, please reassign or refile against that product. The codebase for pirut and pup is quite different, so existing bugs do not apply, but please continue testing them on the still supported versions of Fedora Core and file bugs as necessary.