From Bugzilla Helper: User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.3) Gecko/20041104 Firefox/1.0RC1 Description of problem: Yum crashes with python traceback. this seems to be an issue with python 2.4 rather than yum itself Version-Release number of selected component (if applicable): yum-2.1.11-4, python-2.4-0.b2.3 How reproducible: Always Steps to Reproduce: 1. yum check-update 2. 3. Actual Results: traceback error Expected Results: show yum rpm updates Additional info: return apply(func, (opts,) + args, {}) File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 554, in retryfunc return URLGrabberFileObject(url, filename=None, opts=opts) File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 703, in __init__ self._do_open() File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 747, in _do_open fo, hdr = self._make_request(req, opener) File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 823, in _make_request fo = opener.open(req) File "/usr/lib/python2.4/urllib2.py", line 364, in open response = meth(req, response) File "/usr/lib/python2.4/urllib2.py", line 468, in http_response code, msg, hdrs = response.code, response.msg, response.info() AttributeError: HTTPResponse instance has no attribute 'code' running python version python-2.4-0.b2.3
[bpm]$ diff -wruN /tmp/urllib2.py urllib2.py --- /tmp/urllib2.py 2004-11-09 18:48:54.263275984 -0600 +++ urllib2.py 2004-11-09 18:45:54.555595704 -0600 @@ -465,7 +465,7 @@ handler_order = 1000 # after all other processing def http_response(self, request, response): - code, msg, hdrs = response.code, response.msg, response.info() + code, msg, hdrs = response.status, response.msg, response.info() if code not in (200, 206): response = self.parent.error( --END-- works for me.
works nice thanks!
That's not the right fix for this problem. Fixing it in python 2.4 isn't likely to get into fedora core. we need to fix it in urlgrabber for python 2.4 - but I think this backward compat break better be noted as a python bug.
If this is an incompatibility between Python 2.3 and Python 2.4, please produce a small test-case that doesn't involve yum and submit it to the Python bug tracker on sourceforge. If this can't be fixed for 2.4.0, it may be fixed for 2.4.1. http://sourceforge.net/tracker/?group_id=5470&atid=105470
me too - yum crashes: # yum -y update Setting up Update Process Setting up Repo: development Traceback (most recent call last): File "/usr/bin/yum", line 8, in ? yummain.main(sys.argv[1:]) File "/usr/share/yum-cli/yummain.py", line 68, in main result, resultmsgs = base.doCommands() File "/usr/share/yum-cli/cli.py", line 381, in doCommands return self.updatePkgs() File "/usr/share/yum-cli/cli.py", line 766, in updatePkgs self.doRepoSetup() File "/usr/share/yum-cli/cli.py", line 70, in doRepoSetup repo.getRepoXML() File "/usr/lib/python2.4/site-packages/yum/repos.py", line 465, in getRepoXML result = self.get(relative=remote, local=local, copy_local=1) File "/usr/lib/python2.4/site-packages/yum/repos.py", line 443, in get checkfunc=checkfunc) File "/usr/lib/python2.4/site-packages/urlgrabber/mirror.py", line 414, in urlgrab return self._mirror_try(func, url, kw) File "/usr/lib/python2.4/site-packages/urlgrabber/mirror.py", line 400, in _mirror_try return func_ref( *(fullurl,), **kwargs ) File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 595, in urlgrab return self._retry(opts, retryfunc, url, filename) File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 527, in _retry return apply(func, (opts,) + args, {}) File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 581, in retryfunc fo = URLGrabberFileObject(url, filename, opts) File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 703, in __init__ self._do_open() File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 747, in _do_open fo, hdr = self._make_request(req, opener) File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 823, in _make_request fo = opener.open(req) File "/usr/lib/python2.4/urllib2.py", line 364, in open response = meth(req, response) File "/usr/lib/python2.4/urllib2.py", line 468, in http_response code, msg, hdrs = response.code, response.msg, response.info() AttributeError: HTTPResponse instance has no attribute 'code'
Seth, sorry to add to this thread. I've had to add yet another patch to a python file to get yum to download and install todays rawhide updates. This is given the first patch to urllib2.pl. This is to urllib.pl. I know that this is not a YUM error, but a python bug. All I care is that I can get it to work. It probally is a very bad HACK. It might cause cancer in lab rats for all I know. It does enable my system to be be bandaged (without to much bleeding) till a good fix is available. [bpm]$ diff -wruN /tmp/urllib.py ./urllib.py --- /tmp/urllib.py 2004-11-11 08:36:35.785508280 -0600 +++ ./urllib.py 2004-11-11 08:27:34.000000000 -0600 @@ -836,6 +836,8 @@ addbase.__init__(self, fp) self.headers = headers self.url = url + self.status = 200 # Status-Code + self.msg = None def info(self): return self.headers
Was this filed upstream yet? If so, what is the bug number?
Thanks, both changes fixed yum for now.
*** Bug 138987 has been marked as a duplicate of this bug. ***
Tested both patches from bryan. This seemed to fix this issue.
This is definitely a bug in Python 2.4. It seems that jhylton applied a patch (SF number 852995) between revisions 1.56 and 1.57 of urllib2.py but never bothered to finish the processor implementation. Give my patch a try and see if it resolves it.
Created attachment 106866 [details] Patch to Python 2.4b2's urllib2.py
well.... Traceback (most recent call last): File "/usr/bin/yum", line 8, in ? yummain.main(sys.argv[1:]) File "/usr/share/yum-cli/yummain.py", line 51, in main base.getOptionsConfig(args) File "/usr/share/yum-cli/cli.py", line 133, in getOptionsConfig self.conf = yumconf(configfile = yumconffile, root=root) File "/usr/lib/python2.4/site-packages/yum/config.py", line 227, in __init__ self._doFileRepo(fn) File "/usr/lib/python2.4/site-packages/yum/config.py", line 299, in _doFileRepo doRepoSection(self, repoconf, section) File "/usr/lib/python2.4/site-packages/yum/config.py", line 313, in doRepoSection mirrorurls = getMirrorList(mirrorlist) File "/usr/lib/python2.4/site-packages/yum/config.py", line 390, in getMirrorList fo = urlresolver.urlopen(url) File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 427, in urlopen return default_grabber.urlopen(url, **kwargs) File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 555, in urlopen return self._retry(opts, retryfunc, url) File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 527, in _retry return apply(func, (opts,) + args, {}) File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 554, in retryfunc return URLGrabberFileObject(url, filename=None, opts=opts) File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 703, in __init__ self._do_open() File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 747, in _do_open fo, hdr = self._make_request(req, opener) File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 823, in _make_request fo = opener.open(req) File "/usr/lib/python2.4/urllib2.py", line 364, in open response = meth(req, response) File "/usr/lib/python2.4/urllib2.py", line 468, in http_response code, msg, hdrs = response.__dict__.get(status, None), response.msg, response.info() NameError: global name 'status' is not defined
Crap, my bad...
Created attachment 106867 [details] FIXED Python 2.4b2 patch
Good. Works fine. Does this obsolete the hack to urllib.pl?
Yes, it does. And could someone please tell the Python people that they need to test not only the success paths, but the failure paths as well? ;)
Humm.. I still have a traceback (but a new one this time): <snip> livna-test: ################################################## 8/8 Resolving Dependencies --> Populating transaction set with selected packages. Please wait. ---> Downloading header for kdepim-devel to pack into transaction set. Traceback (most recent call last): File "/usr/bin/yum", line 8, in ? yummain.main(sys.argv[1:]) File "/usr/share/yum-cli/yummain.py", line 104, in main (result, resultmsgs) = base.buildTransaction() File "/usr/lib/python2.4/site-packages/yum/__init__.py", line 174, in buildTransaction (rescode, restring) = self.resolveDeps() File "/usr/lib/python2.4/site-packages/yum/depsolve.py", line 148, in resolveDeps self.populateTs(test=1) File "/usr/lib/python2.4/site-packages/yum/depsolve.py", line 106, in populateTs self.downloadHeader(po) File "/usr/lib/python2.4/site-packages/yum/__init__.py", line 454, in downloadHeader end=end, checkfunc=checkfunc, copy_local=1) File "/usr/lib/python2.4/site-packages/yum/repos.py", line 443, in get checkfunc=checkfunc) File "/usr/lib/python2.4/site-packages/urlgrabber/mirror.py", line 414, in urlgrab return self._mirror_try(func, url, kw) File "/usr/lib/python2.4/site-packages/urlgrabber/mirror.py", line 400, in _mirror_try return func_ref( *(fullurl,), **kwargs ) File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 595, in urlgrab return self._retry(opts, retryfunc, url, filename) File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 527, in _retry return apply(func, (opts,) + args, {}) File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 581, in retryfunc fo = URLGrabberFileObject(url, filename, opts) File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 703, in __init__ self._do_open() File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 747, in _do_open fo, hdr = self._make_request(req, opener) File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 823, in _make_request fo = opener.open(req) File "/usr/lib/python2.4/urllib2.py", line 364, in open response = meth(req, response) File "/usr/lib/python2.4/urllib2.py", line 469, in http_response code, msg, hdrs = response.__dict__.get('status', None), response.msg, response.info() AttributeError: addinfourl instance has no attribute 'msg'
Ignacio, if you have a correct patch, submit it along with a test case that fails before the patch and succeeds after to the Python patch tracker on sourceforge.net, http://sourceforge.net/tracker/?group_id=5470&atid=305470
After looking deeper into Python 2.4b2 it seems that my patch isn't enough to fix all issues. It seems that Python is very, very broken in its current beta. A downgrade might be a good idea at this point, but of course it's not my call.
*** Bug 139891 has been marked as a duplicate of this bug. ***
but how to apply this patch? what file is the .diff or patch file? please elaborate. don't know about patching, much. thanks
Given the size of the changes, I'd recommend just using vi and changing the lines manually
code, msg, hdrs = response.__dict__.get('status', None), response.__dict__.get('msg',None), response.info() made things "work" for me (before I got the traceback in #20), but I've not looked at the code enough to know whether this is enough to fix the problem properly or not. Suppose response.info() could be undefined too ;)
Doing the patch as suggested fixes things for a yum check-update point of view but yum update produces the following output :- [root@xxxxxxxx ~]# yum update Setting up Update Process Setting up Repo: development repomd.xml 100% |=========================| 1.1 kB 00:00 Reading repository metadata in from local files developmen: ################################################## 3559/3559 Resolving Dependencies --> Populating transaction set with selected packages. Please wait. ---> Downloading header for esound-devel to pack into transaction set. Traceback (most recent call last): File "/usr/bin/yum", line 8, in ? yummain.main(sys.argv[1:]) File "/usr/share/yum-cli/yummain.py", line 104, in main (result, resultmsgs) = base.buildTransaction() File "/usr/lib/python2.4/site-packages/yum/__init__.py", line 174, in buildTransaction (rescode, restring) = self.resolveDeps() File "/usr/lib/python2.4/site-packages/yum/depsolve.py", line 148, in resolveDeps self.populateTs(test=1) File "/usr/lib/python2.4/site-packages/yum/depsolve.py", line 106, in populateTs self.downloadHeader(po) File "/usr/lib/python2.4/site-packages/yum/__init__.py", line 454, in downloadHeader end=end, checkfunc=checkfunc, copy_local=1) File "/usr/lib/python2.4/site-packages/yum/repos.py", line 443, in get checkfunc=checkfunc) File "/usr/lib/python2.4/site-packages/urlgrabber/mirror.py", line 414, in urlgrab return self._mirror_try(func, url, kw) File "/usr/lib/python2.4/site-packages/urlgrabber/mirror.py", line 400, in _mirror_try return func_ref( *(fullurl,), **kwargs ) File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 595, in urlgrab return self._retry(opts, retryfunc, url, filename) File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 527, in _retry return apply(func, (opts,) + args, {}) File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 581, in retryfunc fo = URLGrabberFileObject(url, filename, opts) File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 703, in __init__ self._do_open() File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 747, in _do_open fo, hdr = self._make_request(req, opener) File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 823, in _make_request fo = opener.open(req) File "/usr/lib/python2.4/urllib2.py", line 364, in open response = meth(req, response) File "/usr/lib/python2.4/urllib2.py", line 468, in http_response code, msg, hdrs = response.__dict__.get('status', None), response.msg, response.info() AttributeError: addinfourl instance has no attribute 'msg' Also after the patch up2date in GUI mode nolonger shows any channel information and doesn't get any updates it just reports that the system is up2date when updates are actually available so for now I have removed the patch and am relying on up2date to get any updates.
David: do the patch in Additional Comment #6 as well to make the "AttributeError: addinfourl instance has no attribute 'msg'" error go away
The patches at comment #6 and #17 fixed yum for me! :-)
*** Bug 140835 has been marked as a duplicate of this bug. ***
The common problem between urlgrabber.py and python-2.4 is how to attach a readline object to read a response. urlgrabber has chosen to steal various socket methods and add them to a sub-classed HTTPResponse. python-2.4 has chosen a twistier (and apparently Windoze friendly path instead). Here's comments from the do_open method in urllib2.py: # Pick apart the HTTPResponse object to get the addinfourl # object initialized properly. # Wrap the HTTPResponse object in socket's file object adapter # for Windows. That adapter calls recv(), so delegate recv() # to read(). This weird wrapping allows the returned object to # have readline() and readlines() methods. # XXX It might be better to extract the read buffering code # out of socket._fileobject() and into a base class. r.recv = r.read fp = socket._fileobject(r) resp = addinfourl(fp, r.msg, req.get_full_url()) resp.code = r.status resp.msg = r.reason return resp Note that the returned object has changed in python-2.4, urlgrabber do_open is still returning r, and not returning a addinfourl object. Patch to do so in a moment. Disclaimer: I have no bloody clue whether python or urlgrabber is right or wrong, only what I see when I stare at print foo inserted at various points, and even then I'm not sure what I'm looking at. But the patch WORKSFORME, and applies to urlgrabber, not python.
It's a backward compat issue for python. If they're breaking API for these modules then it shouldn't be happening a in a minor version change. Patching it in yum is the WRONG PLACE.
Created attachment 107469 [details] Patch to fox urlgrabber in yum.
Hmmm ... Either "an object that has readline" or "readline method" is a more accurate term than "readline object". Sorry for the imprecision, I don't need no steenkin' objects ;-) And argue with the python guys, not me. All I see is bustage, urlgrabber is doing one thing and python is doing another. The patch WORKSFORME, and is a far smaller context with less risk than hacking python release candidates. jmho, ymmv. And none of the other hacks here are anywhere near close to solving the problem. Sure you can variables to objects until joy, I tried that for a couple hours too.
And thanks for the help. Next time you might want to look at the patch before screaming WRONG PLACE.
Well done to the folks who made the patches :-)
The new patch for urlgrabber is fixing the issue (on three fedora-devel boxes) and seems better than the two other patches (comment #6 and #17) regarding isolation.
Just another workaround that was mentioned on the ml was to use an ftp repositry instead of http. It doesn't fix the actual problem, but it sounds to me like the easiest temporary workaround for now, and requires only a minor configuration change.
here is my output after applying the patch mentioned here: https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=138535#c33 patch -p1 keepalive.py < grabber.patch missing header for unified diff at line 3 of patch patching file keepalive.py and now yum works again for me thanks a lot
This patch isn't fixing in the right place but I'm juggling too many other things at the moment to delve deeper into the python changes. Adding jbj's hack to our yum package but leaving this open so that we can still get python fixed.
I agree with Seth (comment #32) that the bug is in python, but we have to live with it, so yum has to work around that bug. But the patch Jeff posted and was added in rawhide (comment #33) breaks on python <= 2.2 (e.g. Fedora Core 1). Granted, FC1 is EOLed (and used yum 2.0.x), but for the sake of upstream yum and Fedora Legacy projects, could that patch be revised to work for python 2.x and Seth, could that be applied to upstream yum? Thanks!
the urlgrabber maintainer is working on a fix for this.
You guys keep repeating that this is a Python bug. Has anyone created a small standalone script that reproduces the problem (works in 2.3.x, breaks in 2.4.0), and submitted that bug to the Python bug database at http://sf.net/projects/python ? One report refers to the patch http://python.org/sf/852995 but nobody has added a comment to that bug before now. I've added one, but since there's no testcase I didn't have much to say. At least there is a link from that SF tracker item to this bugzilla item now.
I'm the author of the Python urllib2 patch. Thankyou Jeff: if it weren't for your comment, it seems nobody who helps maintain Python would have learned about this... If this is an Python standard library API change, I'd like to know where you found this urllib2 "API" documented! (the details of the HTTP response API that happens to be used in the implementation of HTTPHandler & its new friend HTTPErrorProcessor, that is: whoever wrote keepalive.py made a guess that these implementation details would not change -- that guess turned out to be incorrect) It's not documented, unless I missed it (always possible...), and is an implementation detail of those classes. If you want to be immune from such changes, you must instantiate your own OpenerDirector and .add_handler() for each of an explicit set of handlers (which is trivial), instead of using the rather annoying build_opener() utility function; you should also not derive from HTTPHandler (from a quick glance at keepalive.py, I don't believe you have any reason to derive from it, and implementation inheritance is well-known to be fragile like this). These changes would prevent your code getting clobbered by base-class or (as in this case) friend class changes. Nevertheless. of course, we certainly intended to avoid gratuitous (or even non-gratuitous!-) breakage of classes reimplementing HTTPHandler, and .do_open() in particular. So, I agree that in that sense this is a Python bug, though I'm not immediately sure what the correct fix is, if any. Will submit a Python patch when I think about it again at a more civilised hour, and post a reference to the Python SF tracker when I do so.
(Hmm, by "I'm the author of the Python urllib2 patch.", I mean the one everyone here is complaining about, http://python.org/sf/852995 , rather than the fixes for yum that have been posted here)
*** Bug 147931 has been marked as a duplicate of this bug. ***
I'm pretty confident this is fixed and closed upstream. latest urlgrabber in yum 2.2.0 for FC3 and CVS for rawhide
I've still got this problem in FC4 final version. Python 2.4.1. Traceback is: Gathering header information file(s) from server(s) Server: Fedora Core 4 - i386 - Base retrygrab() failed for: http://fedora.redhat.com/releases/fedora-core-4/headers/header.info Executing failover method Traceback (most recent call last): File "/usr/bin/yum", line 30, in ? yummain.main(sys.argv[1:]) File "/usr/share/yum/yummain.py", line 233, in main clientStuff.get_package_info_from_servers(serverlist, HeaderInfo) File "/usr/share/yum/clientStuff.py", line 846, in get_package_info_from_servers progress_obj=None) File "/usr/share/yum/clientStuff.py", line 1327, in grab bandwidth, conf.retries, retrycodes, checkfunc) File "/usr/share/yum/urlgrabber.py", line 237, in retrygrab progress_obj, throttle, bandwidth) File "/usr/share/yum/urlgrabber.py", line 314, in urlgrab fo = urllib2.urlopen(url) File "/usr/lib/python2.4/urllib2.py", line 130, in urlopen return _opener.open(url, data) File "/usr/lib/python2.4/urllib2.py", line 364, in open response = meth(req, response) File "/usr/lib/python2.4/urllib2.py", line 468, in http_response code, msg, hdrs = response.code, response.msg, response.info() AttributeError: HTTPResponse instance has no attribute 'code'
What ver of yum are you using? I ask b/c there's no reason for anything to be attempting to get http://fedora.redhat.com/releases/fedora-core-4/headers/header.info I think you've got an old version of yum for some reason. yum --version
bash-3.00$ yum --version 2.0.7 I just tried reinstalling yum-2.3.2-7.noarch.rpm and it failed horribly. This is probably the cause of the problem. I did an upgrade from FC2 rather than a clean install. I'll do a reinstall from scratch.