Description of problem: [root@Linux Wani]# fedora-easy-karma Getting list of installed packages... Getting list of packages in updates-testing... Traceback (most recent call last): File "/usr/bin/fedora-easy-karma", line 510, in <module> FedoraEasyKarma() File "/usr/bin/fedora-easy-karma", line 310, in __init__ testing_updates = bc.query(release=release, status="testing", limit=1000)["updates"] File "/usr/lib/python2.7/site-packages/fedora/client/bodhi.py", line 147, in query return self.send_request('list', req_params=params, auth=auth) File "/usr/lib/python2.7/site-packages/fedora/client/baseclient.py", line 344, in send_request auth_params=auth_params, retries=retries) File "/usr/lib/python2.7/site-packages/fedora/client/proxyclient.py", line 394, in send_request if 'exc' in data: TypeError: argument of type 'NoneType' is not iterable [root@Linux Wani]# Version-Release number of selected component (if applicable): fedora-easy-karma.noarch 0:0-0.16.20110825git36efb338.fc17
This happens because of a bug in python-fedora.
Is the error coming because I am behind a proxy server? I've tried setting the http_proxy variable but to no avail!(In reply to comment #1) > This happens because of a bug in python-fedora.
What version of python-fedora do you have installed? Can you try it a few times to see if it's an intermittent issue?
My python-fedora package version is : python-fedora-0.3.32.3-1.fc17.noarch. I've tried it several times now and I still get the same error!(In reply to comment #3) > What version of python-fedora do you have installed? Can you try it a few > times to see if it's an intermittent issue?
Can you try running fedora-easy-karma with the --debug argument and paste the output?
[root@Linux Wani]# fedora-easy-karma --debug DEBUG: fedora_cert_error - timedelta: 0:00:00.000064 DEBUG: fas_username NameError - timedelta: 0:00:00.000063 Getting list of installed packages... DEBUG: starting yum query - timedelta: 0:00:00.022471 Getting list of packages in updates-testing... DEBUG: starting bodhi query - timedelta: 0:00:00.883169 Traceback (most recent call last): File "/usr/bin/fedora-easy-karma", line 510, in <module> FedoraEasyKarma() File "/usr/bin/fedora-easy-karma", line 310, in __init__ testing_updates = bc.query(release=release, status="testing", limit=1000)["updates"] File "/usr/lib/python2.7/site-packages/fedora/client/bodhi.py", line 147, in query return self.send_request('list', req_params=params, auth=auth) File "/usr/lib/python2.7/site-packages/fedora/client/baseclient.py", line 344, in send_request auth_params=auth_params, retries=retries) File "/usr/lib/python2.7/site-packages/fedora/client/proxyclient.py", line 394, in send_request if 'exc' in data: TypeError: argument of type 'NoneType' is not iterable [root@Linux Wani]#
Ralph, how does python-requests handle proxies? Is it with the http_proxy environment variable?
According to this, http://docs.python-requests.org/en/latest/user/advanced/#proxies http_proxy is honored. Is your proxy using authentication?
What's the format of your http_proxy environment variable?
Okay -- and I found more thing I need to ask you about: Do you have https_proxy set? The servers that fedora-easy-karma is trying to access are https, not http. For those, python-requests is looking for the value of the https_proxy environment variable.
My proxy server doesn't require authentication. Even if I set the https_proxy env variable, I get the same error. [Wani@Linux ~]$ export https_proxy="10.4.3.204:8080" [Wani@Linux ~]$ export http_proxy="10.4.3.204:8080" [Wani@Linux ~]$ fedora-easy-karma Getting list of installed packages... Getting list of packages in updates-testing... Traceback (most recent call last): File "/usr/bin/fedora-easy-karma", line 510, in <module> FedoraEasyKarma() File "/usr/bin/fedora-easy-karma", line 310, in __init__ testing_updates = bc.query(release=release, status="testing", limit=1000)["updates"] File "/usr/lib/python2.7/site-packages/fedora/client/bodhi.py", line 147, in query return self.send_request('list', req_params=params, auth=auth) File "/usr/lib/python2.7/site-packages/fedora/client/baseclient.py", line 344, in send_request auth_params=auth_params, retries=retries) File "/usr/lib/python2.7/site-packages/fedora/client/proxyclient.py", line 394, in send_request if 'exc' in data: TypeError: argument of type 'NoneType' is not iterable [Wani@Linux ~]$ (In reply to comment #10) > Okay -- and I found more thing I need to ask you about: Do you have > https_proxy set? > > The servers that fedora-easy-karma is trying to access are https, not http. > For those, python-requests is looking for the value of the https_proxy > environment variable.
(In reply to comment #11) > My proxy server doesn't require authentication. Even if I set the > https_proxy env variable, I get the same error. > [Wani@Linux ~]$ export https_proxy="10.4.3.204:8080" Probably it needs to be HTTPS_PROXY in all caps to work (see for example the curl man page).
[Wani@Linux ~]$ export http_proxy="http://10.4.3.204:8080" [Wani@Linux ~]$ export https_proxy="https://10.4.3.204:8080" [Wani@Linux ~]$ export HTTP_PROXY="http://10.4.3.204:8080" [Wani@Linux ~]$ export HTTPS_PROXY="https://10.4.3.204:8080" [Wani@Linux ~]$ fedora-easy-karma Getting list of installed packages... Getting list of packages in updates-testing... Traceback (most recent call last): File "/usr/bin/fedora-easy-karma", line 510, in <module> FedoraEasyKarma() File "/usr/bin/fedora-easy-karma", line 310, in __init__ testing_updates = bc.query(release=release, status="testing", limit=1000)["updates"] File "/usr/lib/python2.7/site-packages/fedora/client/bodhi.py", line 147, in query return self.send_request('list', req_params=params, auth=auth) File "/usr/lib/python2.7/site-packages/fedora/client/baseclient.py", line 344, in send_request auth_params=auth_params, retries=retries) File "/usr/lib/python2.7/site-packages/fedora/client/proxyclient.py", line 351, in send_request verify=not self.insecure, File "/usr/lib/python2.7/site-packages/requests/api.py", line 98, in post return request('post', url, data=data, **kwargs) File "/usr/lib/python2.7/site-packages/requests/safe_mode.py", line 39, in wrapped return function(method, url, **kwargs) File "/usr/lib/python2.7/site-packages/requests/api.py", line 51, in request return session.request(method=method, url=url, **kwargs) File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 241, in request r.send(prefetch=prefetch) File "/usr/lib/python2.7/site-packages/requests/models.py", line 639, in send raise SSLError(e) requests.exceptions.SSLError: [Errno 1] _ssl.c:504: error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol [Wani@Linux ~]$ (In reply to comment #12) > (In reply to comment #11) > > My proxy server doesn't require authentication. Even if I set the > > https_proxy env variable, I get the same error. > > [Wani@Linux ~]$ export https_proxy="10.4.3.204:8080" > > Probably it needs to be HTTPS_PROXY in all caps to work (see for example the > curl man page).
Bug is in python-requests. I setup a squid proxy. Set https_proxy=192.168.3.13:3128 Verified that the proxy was working with curl (pycurl [using old version of python-fedora] also worked): curl https://admin.fedoraproject.org/accounts/ Then tried this test case in a python shell: import requests from getpass import getpass username = 'toshio' password = getpass() headers = {'Accept': 'application/json', 'User agent': 'testing client'} params = {'user_name': username, 'password': password, 'login': 'Login'} response = requests.post(url, data=params, headers=headers, verify=True) response.json => None response.text => html of the web page If I instead do: response = requests.post(url+'?tg_format=json', data=complete_params, headers=headers, verify=True) response.json => expected values So the bug seems to be when python-requests uses an https proxy set via the https_proxy environment variable, it does not send the value of the headers parameter on to the server. One enhancement for python-fedora: We should change the code to throw an error about being unable to decode json in this situation. I don't see a way for response.json to do that in the current version (request-1.1.0 documents: "In case the JSON decoding fails, r.json raises an exception." so that might work out of the box with an upgraded python-requests [although, we need to know if what exception is thrown so we can catch and rethrow it properly]) In the current version we are probably best off checking if response.json is None: raise ServerError('msg about being unable to decode json')
forgot one piece of the test case: url = 'https://admin.fedoraproject.org/accounts/home'
In the bundled urllib3 poolmanager.py:ProxyManager() def _set_proxy_headers(self, headers=None): headers = headers or {} # Same headers are curl passes for --proxy1.0 headers['Accept'] = '*/*' headers['Proxy-Connection'] = 'Keep-Alive' return headers so... it's overwriting our Accept header when using a proxy.
And... here's the fix: https://github.com/shazow/urllib3/pull/93 That needs to be applied to python-urllib3 (the 1.5 release doesn't have it) and if we haven't unbundled python-urllib3 from python-requests, it needs to be applied to the bundled python-urllib3 as well. @threebean, let me know if you need any further help.
Cool. I created updates for the unbundling yesterday in python-requests-1.1.0-3 https://bugzilla.redhat.com/show_bug.cgi?id=904623 Patch to python-urllib3 on its way! Thanks for figuring this out, Toshio.
python-urllib3-1.5-4.fc18 has been submitted as an update for Fedora 18. https://admin.fedoraproject.org/updates/python-urllib3-1.5-4.fc18
python-urllib3-1.5-4.fc17 has been submitted as an update for Fedora 17. https://admin.fedoraproject.org/updates/python-urllib3-1.5-4.fc17
python-urllib3-1.5-4.el6 has been submitted as an update for Fedora EPEL 6. https://admin.fedoraproject.org/updates/python-urllib3-1.5-4.el6
*** Bug 921163 has been marked as a duplicate of this bug. ***
Closing. Should be fixed with python-urllib3-1.5-5 and python-requests-1.1.0-3 https://admin.fedoraproject.org/updates/python-urllib3 https://admin.fedoraproject.org/updates/python-requests
python-urllib3-1.5-6.fc18 has been submitted as an update for Fedora 18. https://admin.fedoraproject.org/updates/python-urllib3-1.5-6.fc18
python-urllib3-1.5-6.fc19 has been submitted as an update for Fedora 19. https://admin.fedoraproject.org/updates/python-urllib3-1.5-6.fc19
python-urllib3-1.5-6.fc17 has been submitted as an update for Fedora 17. https://admin.fedoraproject.org/updates/python-urllib3-1.5-6.fc17
python-urllib3-1.5-6.fc19 has been pushed to the Fedora 19 stable repository. If problems still persist, please make note of it in this bug report.
python-urllib3-1.5-6.fc18 has been pushed to the Fedora 18 stable repository. If problems still persist, please make note of it in this bug report.
python-urllib3-1.5-6.fc17 has been pushed to the Fedora 17 stable repository. If problems still persist, please make note of it in this bug report.
python-urllib3-1.5-7.el6 has been submitted as an update for Fedora EPEL 6. https://admin.fedoraproject.org/updates/python-urllib3-1.5-7.el6
Package python-urllib3-1.5-7.el6: * should fix your issue, * was pushed to the Fedora EPEL 6 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=epel-testing python-urllib3-1.5-7.el6' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-EPEL-2013-11982/python-urllib3-1.5-7.el6 then log in and leave karma (feedback).
python-urllib3-1.5-7.el6 has been pushed to the Fedora EPEL 6 stable repository. If problems still persist, please make note of it in this bug report.