Bug 1241561

Summary: TypeError: cannot concatenate 'str' and 'tuple' objects
Product: Red Hat Enterprise Linux 7 Reporter: Alexander Todorov <atodorov>
Component: python-requestsAssignee: Matej Stuchlik <mstuchli>
Status: CLOSED NEXTRELEASE QA Contact: BaseOS QE - Apps <qe-baseos-apps>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 7.1CC: crobinso, dzickus, jberan, mstuchli, wwoods
Target Milestone: rc   
Target Release: 7.1   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-07-10 11:26:35 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Alexander Todorov 2015-07-09 13:42:31 UTC
Description of problem:

I'm receiving this message from a script using python-bugzilla instead of logging in correctly or receiving a login error. The actual error is hidden under some try-except blocks but I've managed to narrow it down to:

  File "./bugzilla_plugin.py", line 79, in get_metrics
    bz.login(user, password)
  File "/usr/lib/python2.7/site-packages/bugzilla/base.py", line 646, in login
    ret = self._login(self.user, self.password)
  File "/usr/lib/python2.7/site-packages/bugzilla/base.py", line 615, in _login
    return self._proxy.User.login({'login': user, 'password': password})
  File "/usr/lib64/python2.7/xmlrpclib.py", line 1224, in __call__
    return self.__send(self.__name, args)
  File "/usr/lib/python2.7/site-packages/bugzilla/base.py", line 168, in _ServerProxy__request
    ret = ServerProxy._ServerProxy__request(self, methodname, params)
  File "/usr/lib64/python2.7/xmlrpclib.py", line 1578, in __request
    verbose=self.__verbose
  File "/usr/lib/python2.7/site-packages/bugzilla/base.py", line 265, in request
    return self._request_helper(url, request_body)
  File "/usr/lib/python2.7/site-packages/bugzilla/base.py", line 223, in _request_helper
    url, data=request_body, **self.request_defaults)
  File "/usr/lib/python2.7/site-packages/requests/api.py", line 87, in post
    return request('post', url, data=data, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/api.py", line 44, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 288, in request
    resp = self.send(prep, stream=stream, timeout=timeout, verify=verify, cert=cert, proxies=proxies)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 383, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 219, in send
    r = self.build_response(request, resp)
  File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 96, in build_response
    response.encoding = get_encoding_from_headers(response.headers)
  File "/usr/lib/python2.7/site-packages/requests/utils.py", line 285, in get_encoding_from_headers
    content_type, params = cgi.parse_header(content_type)
  File "/usr/lib64/python2.7/cgi.py", line 310, in parse_header
    parts = _parseparam(';' + line)
TypeError: cannot concatenate 'str' and 'tuple' objects


Here line is a tuple with value ('content-type', 'text/xml')

This is returned previously to a call of cgi.parse_header() but it could have something to do with different response from the server, I have no idea at this time. 




Version-Release number of selected component (if applicable):
python-bugzilla-1.2.1-1.el7.noarch

How reproducible:
Always

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 Alexander Todorov 2015-07-09 13:51:26 UTC
You can also reproduce with:

$ bugzilla --bugzilla https://bugzilla.redhat.com --user atodorov login
Bugzilla Password: xxx



I've provided false password expecting a login failed error.

Comment 2 Alexander Todorov 2015-07-09 14:42:44 UTC
On line 93 of requests/adapters.py there is:

response.headers = CaseInsensitiveDict(getattr(resp, 'headers', {}))

where resp['headers'] is of type urllib3.HTTPHeaderDict which stores both the header name and value as a tuple. 

python-urllib3 changed recently and I've updated on Jun 26th as it seems. Looking into this now to figure out if it changed behavior somehow.

Comment 3 Alexander Todorov 2015-07-09 15:37:18 UTC
I've found the issue. 

urllib3 1.10.2 introduces the HTTPHeaderDict class which is not a simple dict anymore but instead stores the header name and value as a tuple inside the dict value.  Somewhere along the line initialization of CaseInsensitiveDict (also a dict) brings in those tuples.


We can either patch python-request to account for that (I've found one place and it works for me) or better provide a backward compatible update to urllib3 if I manage to find out what that needs to be.

Comment 4 Alexander Todorov 2015-07-09 15:49:32 UTC
Reported upstream for comments:
https://github.com/shazow/urllib3/issues/670

This patch to requests/adapters.py works for me for now.

--- adapters.py.orig	2015-07-09 18:48:24.172045495 +0300
+++ adapters.py	2015-07-09 18:48:38.869488926 +0300
@@ -89,8 +89,18 @@
         # Fallback to None if there's no status_code, for whatever reason.
         response.status_code = getattr(resp, 'status', None)
 
+        # urllib3 1.10.2 and later changes the headers attribute from
+        # dict to HTTPHeaderDict which stores values as tuples
+        # see rhbz #1241561
+        headers = {}
+        for key, value in getattr(resp, 'headers', {}).items():
+            val = value
+            if type(value) == tuple:
+                val = ', '.join(value[1:])
+            headers[key] = value
+
         # Make headers case-insensitive.
-        response.headers = CaseInsensitiveDict(getattr(resp, 'headers', {}))
+        response.headers = CaseInsensitiveDict(headers)
 
         # Set encoding.
         response.encoding = get_encoding_from_headers(response.headers)

Comment 5 Alexander Todorov 2015-07-09 16:00:42 UTC
python-requests-1.1.0-8.el7.noarch

https://github.com/shazow/urllib3/pull/633

Comment 7 Matej Stuchlik 2015-07-10 08:26:00 UTC
(In reply to Alexander Todorov from comment #5)
> python-requests-1.1.0-8.el7.noarch
> 
> https://github.com/shazow/urllib3/pull/633

This doesn't seem right, 7.1.z/7.2.0 should include python-requests-2.6.0. Could you please try updating, and let me know if the issue remains?

Comment 10 Alexander Todorov 2015-07-10 11:26:35 UTC
python-requests-2.6.0-1.el7_1.noarch fixes the issue (I've installed from a Brew build) so closing this one.