Bug 702766

Summary: [F15 regression] traceback when sending over SSL
Product: [Fedora] Fedora Reporter: Stefan Becker <chemobejk>
Component: m2cryptoAssignee: Miloslav Trmač <mitr>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 15CC: gholms, mitr
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: m2crypto-0.21.1-4.fc15 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-05-19 05:09:24 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Reproducer none

Description Stefan Becker 2011-05-06 22:02:12 UTC
Description of problem:

After upgrading from F14 to F15 I now get a traceback when using the Python application "osc" (openSUSE Build Service Commander)

$ osc commit
...
Sending    pidgin-sipe-1.11.2.tar.gz
Transmitting file data .Traceback (most recent call last):
  File "/usr/bin/osc", line 21, in <module>
    r = babysitter.run(osccli)
  File "/usr/lib/python2.7/site-packages/osc/babysitter.py", line 51, in run
    return prg.main()
  File "/usr/lib/python2.7/site-packages/osc/cmdln.py", line 310, in main
    return self.cmd(args)
  File "/usr/lib/python2.7/site-packages/osc/cmdln.py", line 333, in cmd
    retval = self.onecmd(argv)
  File "/usr/lib/python2.7/site-packages/osc/cmdln.py", line 449, in onecmd
    return self._dispatch_cmd(handler, argv)
  File "/usr/lib/python2.7/site-packages/osc/cmdln.py", line 1179, in _dispatch_cmd
    return handler(argv[0], opts, *args)
  File "/usr/lib/python2.7/site-packages/osc/commandline.py", line 3465, in do_commit
    prj.commit(packages, msg=msg, files=files)
  File "/usr/lib/python2.7/site-packages/osc/core.py", line 762, in commit
    p.commit(msg, validators_dir=validators_dir, verbose_validation=verbose_validation)
  File "/usr/lib/python2.7/site-packages/osc/core.py", line 1272, in commit
    self.put_source_file(filename)
  File "/usr/lib/python2.7/site-packages/osc/core.py", line 1134, in put_source_file
    http_PUT(u, file = os.path.join(self.dir, n))
  File "/usr/lib/python2.7/site-packages/osc/core.py", line 2914, in http_PUT
    def http_PUT(*args, **kwargs):    return http_request('PUT', *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/osc/core.py", line 2900, in http_request
    fd = urllib2.urlopen(req, data=data)
  File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
  File "/usr/lib/python2.7/urllib2.py", line 392, in open
    response = self._open(req, data)
  File "/usr/lib/python2.7/urllib2.py", line 410, in _open
    '_open', req)
  File "/usr/lib/python2.7/urllib2.py", line 370, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/site-packages/osc/oscssl.py", line 208, in https_open
    h.request(req.get_method(), selector, req.data, headers)
  File "/usr/lib/python2.7/httplib.py", line 941, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python2.7/httplib.py", line 975, in _send_request
    self.endheaders(body)
  File "/usr/lib/python2.7/httplib.py", line 937, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python2.7/httplib.py", line 801, in _send_output
    self.send(message_body)
  File "/usr/lib/python2.7/httplib.py", line 773, in send
    self.sock.sendall(data)
  File "/usr/lib/python2.7/site-packages/M2Crypto/SSL/Connection.py", line 222, in write
    return self._write_bio(data)
  File "/usr/lib/python2.7/site-packages/M2Crypto/SSL/Connection.py", line 205, in _write_bio
    return m2.ssl_write(self.ssl, data, self._timeout)
TypeError: 'buffer' does not have the buffer interface

Version-Release number of selected component (if applicable):
m2crypto-0.21.1-3.fc15.i686
python-2.7.1-7.fc15.i686
osc-0.131.1-39.1.i686


This error message seems to be related to bytes/string differences in newer Pythons. I'm not a real Python expert, but I hacked the line 205 in Connection.py from

       return m2.ssl_write(self.ssl, data, self._timeout)

to

       return m2.ssl_write(self.ssl, bytes(data), self._timeout)

I'm guessing also other Python applications sending data over HTTPS will be affected by this problem.

Comment 1 Miloslav Trmač 2011-05-06 22:42:26 UTC
Thanks for your report.

Can you point me to source code of osc?

This may be a stand-alone reproducer, but without seeing osc it's just a guess:

>>> import M2Crypto
>>> c = M2Crypto.m2.ssl_ctx_new(M2Crypto.m2.tlsv1_method())
>>> s = M2Crypto.m2.ssl_new(c)
>>> buf = buffer('abc')
>>> M2Crypto.m2.ssl_write(s, buf)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'buffer' does not have the buffer interface

A few notes to self:

* This is probably related to https://bugzilla.osafoundation.org/show_bug.cgi?id=13005 , where we diverge from upstream to provide timeout support (the change was necessary because socket._fileobject uses the memoryview interface).

* If so, osc will have to adapt in the longer term (at least for Python 3).

* Not being able to get a memoryview of a buffer is rather surprising anyway.

Comment 2 Stefan Becker 2011-05-06 23:00:55 UTC
For my F15 build I downloaded the F14 SRPM from here:

  <http://download.opensuse.org/repositories/openSUSE:/Tools/Fedora_14/src/osc-0.131http://gitorious.org/opensuse/osc.1-39.1.src.rpm>

This is probably the source repository:

  <http://gitorious.org/opensuse/osc>

Comment 3 Miloslav Trmač 2011-05-10 13:12:41 UTC
Created attachment 498001 [details]
Reproducer

Comment 4 Miloslav Trmač 2011-05-10 20:18:58 UTC
Thanks, m2crypto-0.21.1-4 should fix this (it fixes the reproducer from comment #3, at least).

Comment 5 Fedora Update System 2011-05-10 20:28:47 UTC
m2crypto-0.21.1-4.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/m2crypto-0.21.1-4.fc15

Comment 6 Stefan Becker 2011-05-11 05:29:30 UTC
I installed the latest version from koji and "osc commit" went through fine. Thanks.

Comment 7 Fedora Update System 2011-05-11 05:50:17 UTC
Package m2crypto-0.21.1-4.fc15:
* should fix your issue,
* was pushed to the Fedora 15 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing m2crypto-0.21.1-4.fc15'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/m2crypto-0.21.1-4.fc15
then log in and leave karma (feedback).

Comment 8 Fedora Update System 2011-05-19 05:09:19 UTC
m2crypto-0.21.1-4.fc15 has been pushed to the Fedora 15 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 9 Fedora Update System 2011-05-20 16:20:40 UTC
python26-m2crypto-0.21.1-5.el5 has been submitted as an update for Fedora EPEL 5.
https://admin.fedoraproject.org/updates/python26-m2crypto-0.21.1-5.el5

Comment 10 Fedora Update System 2011-06-15 14:59:13 UTC
python26-m2crypto-0.21.1-5.el5 has been pushed to the Fedora EPEL 5 stable repository.  If problems still persist, please make note of it in this bug report.