Bug 949186
| Summary: | The pycurl downloader times out on active downloads, even during very good transfer rates | ||
|---|---|---|---|
| Product: | [Retired] Pulp | Reporter: | Randy Barlow <rbarlow> |
| Component: | z_other | Assignee: | Randy Barlow <rbarlow> |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Preethi Thomas <pthomas> |
| Severity: | high | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 2.1 Beta | ||
| Target Milestone: | --- | ||
| Target Release: | 2.4.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2014-08-09 06:54:53 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
This is very likely due to line 174 in our curl.py: 174 easy_handle.setopt(pycurl.TIMEOUT, DEFAULT_REQUEST_TIMEOUT) The DEFAULT_REQUEST_TIMEOUT is set to 300 (seconds). According to the curl documentation, pycurl.TIMEOUT is the maximum time an operation is allowed to take[0]. I believe it could be considered a bug to set this option to any value, and we should stick to their default (0), which means no timeout. There are options for low speed limits and options that will cause a timeout if no bytes have been received in a given amount of time. I think it would be reasonable to set those values in lieu of this particular value. [0] http://curl.haxx.se/libcurl/c/curl_easy_setopt.html#CURLOPTTIMEOUT This is fixed by https://github.com/pulp/pulp/pull/427 Merged. This was fixed in a prior release of Pulp, but we never put it through the QE process. Moving to ON_QA. verified [root@hp-dl380pgen8-01 ~]# rpm -qa pulp-server pulp-server-2.4.0-0.19.beta.fc20.noarch [root@hp-dl380pgen8-01 ~]# [root@hp-dl380pgen8-01 ~]# pulp-admin iso repo create --repo-id cdn-iso --feed https://cdn.redhat.com/content/dist/rhel/server/6/6.5/x86_64/iso/ --feed-ca-cert CDN/cdn.redhat.com-chain.crt --feed-cert CDN/1391032266_4771.crt --feed-key CDN/1391032266_4771.key Successfully created repository [cdn-iso] [root@hp-dl380pgen8-01 ~]# [root@hp-dl380pgen8-01 ~]# [root@hp-dl380pgen8-01 ~]# [root@hp-dl380pgen8-01 ~]# pulp-admin iso repo sync run --repo-id cdn-iso +----------------------------------------------------------------------+ Synchronizing Repository [cdn-iso] +----------------------------------------------------------------------+ This command may be exited via ctrl+c without affecting the request. The Pulp Manifest was downloaded successfully. Downloading 3 ISOs... [==================================================] 100% ISOs: 3/3 Data: 10.7 GB/10.7 GB Avg: 3.8 MB/s Successfully downloaded 3 ISOs. Task Succeeded The repository was successfully published. Task Succeeded [root@hp-dl380pgen8-01 ~]# pulp-admin iso repo list +----------------------------------------------------------------------+ ISO Repositories +----------------------------------------------------------------------+ Id: iso Display Name: iso Description: None Content Unit Counts: Iso: 1 Id: cdn-iso Display Name: cdn-iso Description: None Content Unit Counts: Iso: 3 [root@hp-dl380pgen8-01 ~]# This has been fixed in Pulp 2.4.0-1. |
When testing the 2.1.1 branch for the ability to synchronize ISOs from the CDN to the proxy, I realized that the download fails exactly five minutes into the download, and that it is repeatable. I inserted some logging, and discovered that the pycurl downloader is reporting that the download is timing out: 2013-04-06 16:55:02,192 pulp_rpm.plugins.importers.iso_importer.sync:ERROR: report.error_report: {'response_code': 200, 'error_message': 'Operation timed out after 300053 milliseconds with 69531012 out of 3679453184 bytes received', 'error_code': 28} 2013-04-06 16:55:02,198 pulp_rpm.plugins.importers.iso_importer.sync:ERROR: report.error_report: {'response_code': 200, 'error_message': 'Operation timed out after 300054 milliseconds with 69531012 out of 3720347648 bytes received', 'error_code': 28} 2013-04-06 16:55:02,200 pulp_rpm.plugins.importers.iso_importer.sync:ERROR: report.error_report: {'response_code': 200, 'error_message': 'Operation timed out after 300067 milliseconds with 69531012 out of 3589636096 bytes received', 'error_code': 28} 2013-04-06 16:55:02,206 pulp.server.dispatch.task:ERROR: Importer indicated a failed response Traceback (most recent call last): File "/home/rbarlow/devel/pulp/platform/src/pulp/server/dispatch/task.py", line 138, in _run result = call(*args, **kwargs) File "/home/rbarlow/devel/pulp/platform/src/pulp/server/managers/repo/sync.py", line 118, in sync raise PulpExecutionException(_('Importer indicated a failed response')) PulpExecutionException: Importer indicated a failed response 2013-04-06 16:55:02,206 pulp.server.dispatch.task:INFO: FAILURE: Task 0a022e11-600b-4e78-af2c-cf12a47fd70a: CallRequest: RepoSyncManager.sync(u'cdn_proxy', sync_config_override=None)