Bug 1785696
| Summary: | Changes to how urllib3 was imported caused Timeout stacktrace | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Jesus M. Rodriguez <jesusr> | ||||
| Component: | python-requests | Assignee: | Charalampos Stratakis <cstratak> | ||||
| Status: | CLOSED ERRATA | QA Contact: | RHEL CS Apps Subsystem QE <rhel-cs-apps-subsystem-qe> | ||||
| Severity: | unspecified | Docs Contact: | |||||
| Priority: | high | ||||||
| Version: | 7.8 | CC: | balleman-rh, cstratak, jhouska, jreznik, lbalhar, matt, peter+fedora, pviktori, python-maint, wchadwic | ||||
| Target Milestone: | rc | Keywords: | ZStream | ||||
| Target Release: | --- | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | No Doc Update | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | |||||||
| : | 1811105 1811106 1811107 1811108 (view as bug list) | Environment: | |||||
| Last Closed: | 2020-09-29 19:39:41 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | |||||||
| Bug Blocks: | 1802126, 1811105, 1811106, 1811107, 1811108 | ||||||
| Attachments: |
|
||||||
The bug that caused our investigation was during the testing of the Ansible Operator: https://bugzilla.redhat.com/show_bug.cgi?id=1783829 We tracked it down to this specific change: http://pkgs.devel.redhat.com/cgit/rpms/python-requests/commit/?h=rhel-7.7&id=c2b7d2c6fae9ea48268a2931980b7274c1d8e538 The above change was done to address azure bug: https://bugzilla.redhat.com/show_bug.cgi?id=1750839 In order to fix our bug so we could ship OpenShift 4.3, we had to pin the python-requests release to 2.6.0-7 which fixed our problems. http://pkgs.devel.redhat.com/cgit/containers/openshift-enterprise-apb-base/tree/Dockerfile?h=rhaos-4.3-rhel-7#n8 (In reply to Miro Hrončok from comment #4) > any chance this is related to bz1785607? Anything's possible but I don't get an AttributeError. I think I see the same error on CentOS 7.7:
# rpm -q python-requests
python-requests-2.6.0-8.el7_7.noarch
# COMPOSE_HTTP_TIMEOUT=240 docker-compose pull
Traceback (most recent call last):
File "/bin/docker-compose", line 11, in <module>
sys.exit(main())
File "/usr/lib/python2.7/site-packages/compose/cli/main.py", line 71, in main
command()
File "/usr/lib/python2.7/site-packages/compose/cli/main.py", line 124, in perform_command
project = project_from_options('.', options)
File "/usr/lib/python2.7/site-packages/compose/cli/command.py", line 41, in project_from_options
compatibility=options.get('--compatibility'),
File "/usr/lib/python2.7/site-packages/compose/cli/command.py", line 126, in get_project
project_name, config_data, client, environment.get('DOCKER_DEFAULT_PLATFORM')
File "/usr/lib/python2.7/site-packages/compose/project.py", line 100, in from_config
service_networks = get_networks(service_dict, networks)
File "/usr/lib/python2.7/site-packages/compose/network.py", line 320, in get_networks
networks[network.true_name] = netdef
File "/usr/lib/python2.7/site-packages/compose/network.py", line 125, in true_name
self._set_legacy_flag()
File "/usr/lib/python2.7/site-packages/compose/network.py", line 146, in _set_legacy_flag
data = self.inspect(legacy=True)
File "/usr/lib/python2.7/site-packages/compose/network.py", line 106, in inspect
return self.client.inspect_network(self.legacy_full_name)
File "/usr/lib/python2.7/site-packages/docker/utils/decorators.py", line 19, in wrapped
return f(self, resource_id, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/docker/api/network.py", line 211, in inspect_network
res = self._get(url, params=params)
File "/usr/lib/python2.7/site-packages/docker/utils/decorators.py", line 46, in inner
return f(self, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/docker/api/client.py", line 198, in _get
return self.get(url, **self._set_request_timeout(kwargs))
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 498, in get
return self.request('GET', url, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 486, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 598, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 370, in send
timeout=timeout
File "/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py", line 582, in urlopen
timeout_obj = self._get_timeout(timeout)
File "/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py", line 309, in _get_timeout
return Timeout.from_float(timeout)
File "/usr/lib/python2.7/site-packages/requests/packages/urllib3/util/timeout.py", line 154, in from_float
return Timeout(read=timeout, connect=timeout)
File "/usr/lib/python2.7/site-packages/requests/packages/urllib3/util/timeout.py", line 97, in __init__
self._connect = self._validate_timeout(connect, 'connect')
File "/usr/lib/python2.7/site-packages/requests/packages/urllib3/util/timeout.py", line 127, in _validate_timeout
"int or float." % (name, value))
ValueError: Timeout value connect was Timeout(connect=240, read=240, total=None), but it must be an int or float.
The command works once I downgrade to python-requests-2.6.0-7.el7_7.noarch.
Relevant info to this docker issue:
https://github.com/docker/docker-py/issues/1645
https://github.com/psf/requests/issues/4160
Peter, could you please test the build from comment#6. Sorry! We linked to a private build instead. We'll send the RPM to you privately. This has broken some in-house code, pinning back to 2.6.0-7 fixes things for now. VERIFIED: all modules from comment 16 was successfully loaded. version: python-requests-2.6.0-10.el7.noarch Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (python-requests bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:3886 |
Created attachment 1646861 [details] log from operator pod showing full stacktrace Description of problem: A recent change to python-requests caused previously working code to fail with ValueError: Timeout value connect was Timeout(connect=None, read=None, total=None), but it must be an int, float or None. Version-Release number of selected component (if applicable): 2.6.0-8 How reproducible: Deploy a version of the ansible-service-broker operator that has python-requests-2.6.0-8 installed in its base image. Actual results: {"level":"error","ts":1576464180.9534247,"logger":"runner","msg":"Traceback (most recent call last): File \"/usr/lib/python2.7/site-packages/ansible_runner/__main__.py\", line 329, in main res = run(**run_options) File \"/usr/lib/python2.7/site-packages/ansible_runner/interface.py\", line 162, in run r.run() File \"/usr/lib/python2.7/site-packages/ansible_runner/runner.py\", line 93, in run self.status_callback('starting') File \"/usr/lib/python2.7/site-packages/ansible_runner/runner.py\", line 84, in status_callback ansible_runner.plugins[plugin].status_handler(self.config, status_data) File \"/usr/lib/python2.7/site-packages/ansible_runner_http/events.py\", line 38, in status_handler urlpath=plugin_config['runner_path']) File \"/usr/lib/python2.7/site-packages/ansible_runner_http/events.py\", line 18, in send_request return session.post(url_actual, headers=headers, json=(data)) File \"/usr/lib/python2.7/site-packages/requests/sessions.py\", line 529, in post return self.request('POST', url, data=data, json=json, **kwargs) File \"/usr/lib/python2.7/site-packages/requests/sessions.py\", line 486, in request resp = self.send(prep, **send_kwargs) File \"/usr/lib/python2.7/site-packages/requests/sessions.py\", line 598, in send r = adapter.send(request, **kwargs) File \"/usr/lib/python2.7/site-packages/requests/adapters.py\", line 370, in send timeout=timeout File \"/usr/lib/python2.7/site-packages/urllib3/connectionpool.py\", line 587, in urlopen timeout_obj = self._get_timeout(timeout) File \"/usr/lib/python2.7/site-packages/urllib3/connectionpool.py\", line 302, in _get_timeout return Timeout.from_float(timeout) File \"/usr/lib/python2.7/site-packages/urllib3/util/timeout.py\", line 154, in from_float return Timeout(read=timeout, connect=timeout) File \"/usr/lib/python2.7/site-packages/urllib3/util/timeout.py\", line 94, in __init__ self._connect = self._validate_timeout(connect, 'connect') File \"/usr/lib/python2.7/site-packages/urllib3/util/timeout.py\", line 127, in _validate_timeout \"int, float or None.\" % (name, value)) ValueError: Timeout value connect was Timeout(connect=None, read=None, total=None), but it must be an int, float or None. Expected results: No traceback Additional info: