Bug 619276 - "koji build" no longer works with python 2.7
"koji build" no longer works with python 2.7
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: koji (Show other bugs)
rawhide
All Linux
low Severity urgent
: ---
: ---
Assigned To: David Cantrell
Fedora Extras Quality Assurance
:
: 620146 620879 621376 622877 (view as bug list)
Depends On:
Blocks: F14Target Python27
  Show dependency treegraph
 
Reported: 2010-07-29 02:58 EDT by Mamoru TASAKA
Modified: 2013-01-10 01:05 EST (History)
13 users (show)

See Also:
Fixed In Version: koji-1.4.0-4.fc14
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2010-08-05 10:24:44 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)

  None (edit)
Description Mamoru TASAKA 2010-07-29 02:58:05 EDT
Description of problem:
Well, I just tried scratch build but:
$ koji build --scratch dist-rawhide ./rb_libtorrent-0.14.10-3.2.fc14.src.rpm 
AttributeError: PlgHTTPS instance has no attribute 'getresponse'

With temporarily installed Fedora 13 environ, this still works:
http://koji.fedoraproject.org/koji/taskinfo?taskID=2356563

Also before upgrading python to 2.7 koji build --scratch worked.
So I guess there is something wrong in koji under python 2.7

Version-Release number of selected component (if applicable):
koji-1.4.0-3.fc14.noarch
python-2.7-7.fc14.i686
Comment 1 Mamoru TASAKA 2010-07-30 00:56:03 EDT
More verbose messages from "fedpkg build"

[tasaka1@localhost f14]$ pwd
/home/tasaka1/rpmbuild/fedora-specific/mirage/f14
[tasaka1@localhost f14]$ fedpkg build --nowait
Traceback (most recent call last):
  File "/usr/bin/fedpkg", line 959, in <module>
    args.command(args)
  File "/usr/bin/fedpkg", line 297, in build
    mymodule.init_koji(args.user, kojiconfig)
  File "/usr/lib/python2.7/site-packages/pyfedpkg/__init__.py", line 1102, in init_koji
    defaults['serverca'])
  File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 1628, in ssl_login
    sinfo = self.callMethod('sslLogin', proxyuser)
  File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 1673, in callMethod
    return self._callMethod(name, args, opts)
  File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 1698, in _callMethod
    return proxy.__getattr__(name)(*args)
  File "/usr/lib/python2.7/xmlrpclib.py", line 1224, in __call__
    return self.__send(self.__name, args)
  File "/usr/lib/python2.7/xmlrpclib.py", line 1570, in __request
    verbose=self.__verbose
  File "/usr/lib/python2.7/xmlrpclib.py", line 1264, in request
    return self.single_request(host, handler, request_body, verbose)
  File "/usr/lib/python2.7/xmlrpclib.py", line 1294, in single_request
    response = h.getresponse(buffering=True)
AttributeError: PlgHTTPS instance has no attribute 'getresponse'
Comment 2 Mike McLean 2010-07-30 10:44:49 EDT
this is happening in fedpkg not koji, but it's not fedpkg's fault. Looks like the xmlrpclib in python2.7 needs to be fixed.
Comment 3 Mamoru TASAKA 2010-07-30 14:30:51 EDT
By the way the following workaround works for both
"koji build --scratch" and "fedpkg build"

- put xmlrpclib.py in F-13 python (2.6.x) to somewhere
- PYTHONPATH=<the path where you put xmlrpclib.py> fedpkg build ......
Comment 4 Yanko Kaneti 2010-08-02 02:09:36 EDT
*** Bug 620146 has been marked as a duplicate of this bug. ***
Comment 5 Richard W.M. Jones 2010-08-02 03:50:05 EDT
pygtk2-2.17.0-7.fc15.x86_64 does *NOT* fix this problem for me.
I still get:


$ fedpkg build
Traceback (most recent call last):
  File "/usr/bin/fedpkg", line 959, in <module>
    args.command(args)
  File "/usr/bin/fedpkg", line 297, in build
    mymodule.init_koji(args.user, kojiconfig)
  File "/usr/lib/python2.7/site-packages/pyfedpkg/__init__.py", line 1102, in init_koji
    defaults['serverca'])
  File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 1628, in ssl_login
    sinfo = self.callMethod('sslLogin', proxyuser)
  File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 1673, in callMethod
    return self._callMethod(name, args, opts)
  File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 1698, in _callMethod
    return proxy.__getattr__(name)(*args)
  File "/usr/lib64/python2.7/xmlrpclib.py", line 1224, in __call__
    return self.__send(self.__name, args)
  File "/usr/lib64/python2.7/xmlrpclib.py", line 1570, in __request
    verbose=self.__verbose
  File "/usr/lib64/python2.7/xmlrpclib.py", line 1264, in request
    return self.single_request(host, handler, request_body, verbose)
  File "/usr/lib64/python2.7/xmlrpclib.py", line 1294, in single_request
    response = h.getresponse(buffering=True)
AttributeError: PlgHTTPS instance has no attribute 'getresponse'


$ rpm -qa | grep pygtk2
pygtk2-doc-2.17.0-7.fc15.noarch
pygtk2-devel-2.17.0-7.fc15.x86_64
pygtk2-codegen-2.17.0-7.fc15.x86_64
pygtk2-libglade-2.17.0-7.fc15.x86_64
pygtk2-2.17.0-7.fc15.x86_64
Comment 6 Richard W.M. Jones 2010-08-02 03:51:23 EDT
Ugh, wrong bug - ignore that previous comment.
Comment 7 Matthias Clasen 2010-08-02 17:00:23 EDT
http://lists.fedoraproject.org/pipermail/devel/2010-August/140115.html has a patch to workaround this issue on the koji side
Comment 8 Kevin Fenzi 2010-08-03 13:13:43 EDT
Just an update here... 

Dennis is looking to see if this is planned breakage in the koji api, or if this is a python2.7 xmlrpc issue. 

Please use one of the workarounds until this is fixed?
Comment 9 Dave Malcolm 2010-08-03 16:52:57 EDT
*** Bug 620879 has been marked as a duplicate of this bug. ***
Comment 10 Dave Malcolm 2010-08-03 19:00:06 EDT
I can confirm that the patch fixes it for me.

I'm still digging through the stack analysing the second hunk of the patch.

As for the first hunk...

Python's socket.py:class _socketobject:makefile has had this signature:
    def makefile(self, mode='r', bufsize=-1):
for at least 9 years [1]

koji/ssl/XMLRPCServerProxy.py contains a SSLConnection class which emulates the above API, but with this signature:
    def makefile(self, mode, bufsize):

Looks like upstream Python commit r68532 [2] to httplib [3] adds an invocation:
   self.fp = sock.makefile('rb')
that only supplies a "mode" argument, relying on the default arguments.

Hence Koji's implementation needs to supply default arguments (which the first hunk of the patch mentioned in comment #7 does add) to work with Python 2.7's httplib

[1] http://svn.python.org/view?view=rev&revision=16823
[2] http://svn.python.org/view?view=rev&revision=68532
[3] http://svn.python.org/view/python/trunk/Lib/httplib.py?r1=68532&r2=68531&pathrev=68532
Comment 11 Dave Malcolm 2010-08-03 19:25:12 EDT
Python's xmlrpclib has changed somewhat in Python 2.7

xmlrpclib Transport.single_request calls make_connection.

By default this makes a httplib.HTTPSConnection instance.

Eventually single_request makes a call on the object returned from make_connection to process the result.

koji's PlgSSL_Transport overrides xmlrpclib.Transport's make_connection, returning PlgHTTPS, a subclass of httplib.HTTP

Python's xmlrpclib has changed from calling getreply() to getresponse() in r73638 [1] [2]

httplib.HTTP (and thus PlgHTTPS) does not have a getresponse method.  It does have a getreply method (which delegates to an underlying connection instance's getresponse, with some wrapper logic).

So I believe we're seeing a long-standing bug in Koji, which the change in Python 2.7's xmlrpclib has shaken out.

The second hunk of the patch in comment #7 changes it to return PlgHTTPSConnection, a subclass of httplib.HTTPConnection.  This looks like the correct fix (though I'm not as expert in these modules as I wish to be)

[1] http://svn.python.org/view?view=rev&revision=73638
[2] http://svn.python.org/view/python/trunk/Lib/xmlrpclib.py?r1=73638&r2=73637&pathrev=73638
Comment 12 Dave Malcolm 2010-08-03 19:34:35 EDT
Building with patch into f-14:
  http://koji.fedoraproject.org/koji/taskinfo?taskID=2377894

and into f15:
  http://koji.fedoraproject.org/koji/taskinfo?taskID=2377903
Comment 13 Fedora Update System 2010-08-03 19:38:17 EDT
koji-1.4.0-4.fc14 has been submitted as an update for Fedora 14.
http://admin.fedoraproject.org/updates/koji-1.4.0-4.fc14
Comment 14 Mike McLean 2010-08-04 09:35:32 EDT
dmalcolm - are these patches safe for earlier pythons? (I still care about python 2.3 working for rhel-4 clients)
Comment 15 Fedora Update System 2010-08-05 10:24:38 EDT
koji-1.4.0-4.fc14 has been pushed to the Fedora 14 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 16 Jesse Keating 2010-08-06 15:28:02 EDT
*** Bug 621376 has been marked as a duplicate of this bug. ***
Comment 17 Michael Schwendt 2010-08-10 12:04:28 EDT
mikem,

the XMLRPCServerProxy.py patch will break with Python 2.3 (it broke also with Plague and Python 2.4).
Comment 18 Mike McLean 2010-08-10 15:53:17 EDT
Thanks Michael. I guess I'll have to adjust it before applying upstream.
Comment 19 Daniel Walsh 2010-08-11 09:05:28 EDT
*** Bug 622877 has been marked as a duplicate of this bug. ***
Comment 20 Mike McLean 2010-12-08 16:37:21 EST
Fixed upstream now
https://fedorahosted.org/koji/changeset/7876bc06fe9151fb521409c2e8f6a3535e1eacfe

The patch from comment 7 breaks koji for earlier pythons, so the upstream change is a little different.
Comment 21 Fedora Update System 2011-07-28 17:25:12 EDT
goobook-1.3-8.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/goobook-1.3-8.fc15

Note You need to log in before you can comment on or make changes to this bug.