Bug 619276 - "koji build" no longer works with python 2.7
Summary: "koji build" no longer works with python 2.7
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: koji
Version: rawhide
Hardware: All
OS: Linux
low
urgent
Target Milestone: ---
Assignee: David Cantrell
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 620146 620879 621376 622877 (view as bug list)
Depends On:
Blocks: F14Target Python27
TreeView+ depends on / blocked
 
Reported: 2010-07-29 06:58 UTC by Mamoru TASAKA
Modified: 2013-01-10 06:05 UTC (History)
13 users (show)

Fixed In Version: koji-1.4.0-4.fc14
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2010-08-05 14:24:44 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)

Description Mamoru TASAKA 2010-07-29 06:58:05 UTC
Description of problem:
Well, I just tried scratch build but:
$ koji build --scratch dist-rawhide ./rb_libtorrent-0.14.10-3.2.fc14.src.rpm 
AttributeError: PlgHTTPS instance has no attribute 'getresponse'

With temporarily installed Fedora 13 environ, this still works:
http://koji.fedoraproject.org/koji/taskinfo?taskID=2356563

Also before upgrading python to 2.7 koji build --scratch worked.
So I guess there is something wrong in koji under python 2.7

Version-Release number of selected component (if applicable):
koji-1.4.0-3.fc14.noarch
python-2.7-7.fc14.i686

Comment 1 Mamoru TASAKA 2010-07-30 04:56:03 UTC
More verbose messages from "fedpkg build"

[tasaka1@localhost f14]$ pwd
/home/tasaka1/rpmbuild/fedora-specific/mirage/f14
[tasaka1@localhost f14]$ fedpkg build --nowait
Traceback (most recent call last):
  File "/usr/bin/fedpkg", line 959, in <module>
    args.command(args)
  File "/usr/bin/fedpkg", line 297, in build
    mymodule.init_koji(args.user, kojiconfig)
  File "/usr/lib/python2.7/site-packages/pyfedpkg/__init__.py", line 1102, in init_koji
    defaults['serverca'])
  File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 1628, in ssl_login
    sinfo = self.callMethod('sslLogin', proxyuser)
  File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 1673, in callMethod
    return self._callMethod(name, args, opts)
  File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 1698, in _callMethod
    return proxy.__getattr__(name)(*args)
  File "/usr/lib/python2.7/xmlrpclib.py", line 1224, in __call__
    return self.__send(self.__name, args)
  File "/usr/lib/python2.7/xmlrpclib.py", line 1570, in __request
    verbose=self.__verbose
  File "/usr/lib/python2.7/xmlrpclib.py", line 1264, in request
    return self.single_request(host, handler, request_body, verbose)
  File "/usr/lib/python2.7/xmlrpclib.py", line 1294, in single_request
    response = h.getresponse(buffering=True)
AttributeError: PlgHTTPS instance has no attribute 'getresponse'

Comment 2 Mike McLean 2010-07-30 14:44:49 UTC
this is happening in fedpkg not koji, but it's not fedpkg's fault. Looks like the xmlrpclib in python2.7 needs to be fixed.

Comment 3 Mamoru TASAKA 2010-07-30 18:30:51 UTC
By the way the following workaround works for both
"koji build --scratch" and "fedpkg build"

- put xmlrpclib.py in F-13 python (2.6.x) to somewhere
- PYTHONPATH=<the path where you put xmlrpclib.py> fedpkg build ......

Comment 4 Yanko Kaneti 2010-08-02 06:09:36 UTC
*** Bug 620146 has been marked as a duplicate of this bug. ***

Comment 5 Richard W.M. Jones 2010-08-02 07:50:05 UTC
pygtk2-2.17.0-7.fc15.x86_64 does *NOT* fix this problem for me.
I still get:


$ fedpkg build
Traceback (most recent call last):
  File "/usr/bin/fedpkg", line 959, in <module>
    args.command(args)
  File "/usr/bin/fedpkg", line 297, in build
    mymodule.init_koji(args.user, kojiconfig)
  File "/usr/lib/python2.7/site-packages/pyfedpkg/__init__.py", line 1102, in init_koji
    defaults['serverca'])
  File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 1628, in ssl_login
    sinfo = self.callMethod('sslLogin', proxyuser)
  File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 1673, in callMethod
    return self._callMethod(name, args, opts)
  File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 1698, in _callMethod
    return proxy.__getattr__(name)(*args)
  File "/usr/lib64/python2.7/xmlrpclib.py", line 1224, in __call__
    return self.__send(self.__name, args)
  File "/usr/lib64/python2.7/xmlrpclib.py", line 1570, in __request
    verbose=self.__verbose
  File "/usr/lib64/python2.7/xmlrpclib.py", line 1264, in request
    return self.single_request(host, handler, request_body, verbose)
  File "/usr/lib64/python2.7/xmlrpclib.py", line 1294, in single_request
    response = h.getresponse(buffering=True)
AttributeError: PlgHTTPS instance has no attribute 'getresponse'


$ rpm -qa | grep pygtk2
pygtk2-doc-2.17.0-7.fc15.noarch
pygtk2-devel-2.17.0-7.fc15.x86_64
pygtk2-codegen-2.17.0-7.fc15.x86_64
pygtk2-libglade-2.17.0-7.fc15.x86_64
pygtk2-2.17.0-7.fc15.x86_64

Comment 6 Richard W.M. Jones 2010-08-02 07:51:23 UTC
Ugh, wrong bug - ignore that previous comment.

Comment 7 Matthias Clasen 2010-08-02 21:00:23 UTC
http://lists.fedoraproject.org/pipermail/devel/2010-August/140115.html has a patch to workaround this issue on the koji side

Comment 8 Kevin Fenzi 2010-08-03 17:13:43 UTC
Just an update here... 

Dennis is looking to see if this is planned breakage in the koji api, or if this is a python2.7 xmlrpc issue. 

Please use one of the workarounds until this is fixed?

Comment 9 Dave Malcolm 2010-08-03 20:52:57 UTC
*** Bug 620879 has been marked as a duplicate of this bug. ***

Comment 10 Dave Malcolm 2010-08-03 23:00:06 UTC
I can confirm that the patch fixes it for me.

I'm still digging through the stack analysing the second hunk of the patch.

As for the first hunk...

Python's socket.py:class _socketobject:makefile has had this signature:
    def makefile(self, mode='r', bufsize=-1):
for at least 9 years [1]

koji/ssl/XMLRPCServerProxy.py contains a SSLConnection class which emulates the above API, but with this signature:
    def makefile(self, mode, bufsize):

Looks like upstream Python commit r68532 [2] to httplib [3] adds an invocation:
   self.fp = sock.makefile('rb')
that only supplies a "mode" argument, relying on the default arguments.

Hence Koji's implementation needs to supply default arguments (which the first hunk of the patch mentioned in comment #7 does add) to work with Python 2.7's httplib

[1] http://svn.python.org/view?view=rev&revision=16823
[2] http://svn.python.org/view?view=rev&revision=68532
[3] http://svn.python.org/view/python/trunk/Lib/httplib.py?r1=68532&r2=68531&pathrev=68532

Comment 11 Dave Malcolm 2010-08-03 23:25:12 UTC
Python's xmlrpclib has changed somewhat in Python 2.7

xmlrpclib Transport.single_request calls make_connection.

By default this makes a httplib.HTTPSConnection instance.

Eventually single_request makes a call on the object returned from make_connection to process the result.

koji's PlgSSL_Transport overrides xmlrpclib.Transport's make_connection, returning PlgHTTPS, a subclass of httplib.HTTP

Python's xmlrpclib has changed from calling getreply() to getresponse() in r73638 [1] [2]

httplib.HTTP (and thus PlgHTTPS) does not have a getresponse method.  It does have a getreply method (which delegates to an underlying connection instance's getresponse, with some wrapper logic).

So I believe we're seeing a long-standing bug in Koji, which the change in Python 2.7's xmlrpclib has shaken out.

The second hunk of the patch in comment #7 changes it to return PlgHTTPSConnection, a subclass of httplib.HTTPConnection.  This looks like the correct fix (though I'm not as expert in these modules as I wish to be)

[1] http://svn.python.org/view?view=rev&revision=73638
[2] http://svn.python.org/view/python/trunk/Lib/xmlrpclib.py?r1=73638&r2=73637&pathrev=73638

Comment 12 Dave Malcolm 2010-08-03 23:34:35 UTC
Building with patch into f-14:
  http://koji.fedoraproject.org/koji/taskinfo?taskID=2377894

and into f15:
  http://koji.fedoraproject.org/koji/taskinfo?taskID=2377903

Comment 13 Fedora Update System 2010-08-03 23:38:17 UTC
koji-1.4.0-4.fc14 has been submitted as an update for Fedora 14.
http://admin.fedoraproject.org/updates/koji-1.4.0-4.fc14

Comment 14 Mike McLean 2010-08-04 13:35:32 UTC
dmalcolm - are these patches safe for earlier pythons? (I still care about python 2.3 working for rhel-4 clients)

Comment 15 Fedora Update System 2010-08-05 14:24:38 UTC
koji-1.4.0-4.fc14 has been pushed to the Fedora 14 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 16 Jesse Keating 2010-08-06 19:28:02 UTC
*** Bug 621376 has been marked as a duplicate of this bug. ***

Comment 17 Michael Schwendt 2010-08-10 16:04:28 UTC
mikem,

the XMLRPCServerProxy.py patch will break with Python 2.3 (it broke also with Plague and Python 2.4).

Comment 18 Mike McLean 2010-08-10 19:53:17 UTC
Thanks Michael. I guess I'll have to adjust it before applying upstream.

Comment 19 Daniel Walsh 2010-08-11 13:05:28 UTC
*** Bug 622877 has been marked as a duplicate of this bug. ***

Comment 20 Mike McLean 2010-12-08 21:37:21 UTC
Fixed upstream now
https://fedorahosted.org/koji/changeset/7876bc06fe9151fb521409c2e8f6a3535e1eacfe

The patch from comment 7 breaks koji for earlier pythons, so the upstream change is a little different.

Comment 21 Fedora Update System 2011-07-28 21:25:12 UTC
goobook-1.3-8.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/goobook-1.3-8.fc15


Note You need to log in before you can comment on or make changes to this bug.