Bug 729496
| Summary: | Adding timeout in sync ends with error in log | ||
|---|---|---|---|
| Product: | [Retired] Pulp | Reporter: | Preethi Thomas <pthomas> |
| Component: | user-experience | Assignee: | Jason Connor <jconnor> |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Preethi Thomas <pthomas> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | unspecified | CC: | cevich, jslagle, mmccune, skarmark |
| Target Milestone: | --- | Keywords: | Triaged |
| Target Release: | Sprint 28 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2012-02-24 20:16:07 UTC | Type: | --- |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
To work around the 'maximumrecursion depth exceeded' error, use the mongo shell to clear out the task_snapshots collection:
$ mongo pulp_database
MongoDB shell version: 1.8.2
connecting to: pulp_database
> db.task_snapshots.drop()
true
>
Stumbled across this bug. How about something like this as a fix:
--- src/pulp/server/tasking/taskqueue/queue.py.original 2011-09-01 16:20:32.000000000 -0400
+++ src/pulp/server/tasking/taskqueue/queue.py.modified 2011-09-01 16:28:07.000000000 -0400
@@ -180,7 +180,11 @@
# queue, but the task thread has not had a chance to execute yet
if None in (task.timeout_delta, task.start_time):
continue
- if now - task.start_time < task.timeout_delta:
+ if isinstance(task.timeout_delta, isodate.Duration):
+ timedelta_tasktimeout = task.timeout_delta.todatetime(start=task.start_time)
+ else:
+ timedelta_tasktimeout = task.timeout_delta
+ if now - task.start_time < timedelta_tasktimeout:
continue
task.timeout()
i.e. make sure everything is in a common timedelta format
Alternatively, validate and throw an exception when setting task.timeout_delta if it's longer than 365 days - realistically, will a task ever need to timeout longer than that? i.e. completely avoid the 1y != 365d problem.
Added a timeout parsing wrapper to the webservices that disallows timeout intervals to be specified with months and years. committed in fd38843e56ecd250b46a1ea5a09f2a9eb022b04a build: 0.231 fails_qa
[root@preethi ~]# rpm -q pulp
pulp-0.0.232-1.fc15.noarch
failing since the help needs to updated
[root@preethi ~]# pulp-admin -u admin -p admin repo sync --id=f15 --help
Usage: pulp-admin <options> repo sync <options>
Options:
-h, --help show this help message and exit
--id=ID repository id (required)
--timeout=TIMEOUT repository sync timeout specified in iso8601 duration
format (P[n]Y[n]M[n]DT[n]H[n]M[n]S)
build: 0.238 fails_qa
help needs to updated
[root@preethi ~]# rpm -q pulp
pulp-0.0.238-1.fc15.noarch
failing since the help needs to updated
[root@preethi ~]# pulp-admin repo sync --id=f14 --help
Usage: pulp-admin <options> repo sync <options>
Options:
-h, --help show this help message and exit
--id=ID repository id (required)
--timeout=TIMEOUT repository sync timeout specified in iso8601 duration
format (P[n]Y[n]M[n]DT[n]H[n]M[n]S)
this has been changed by the sync scheduling stuff. build: 0.244 verified
[root@preethi ~]# rpm -q pulp
pulp-0.0.248-1.fc15.noarch
[root@preethi ~]# pulp-admin repo sync --id=f14 --help
Usage: pulp-admin <options> repo sync <options>
--timeout=TIMEOUT repository sync timeout specified in iso8601 duration
format (P[n]Y[n]M[n]DT[n]H[n]M[n]S)
Pulp v1.0 is released Closed Current Release. Pulp v1.0 is released. |
Description of problem: Adding timeout in sync ends with error in log from the pulp.log 2011-08-09 16:16:52,151 8132:140569892341504: pulp.server.webservices.controllers.repositories:INFO: repositories:609 sync timeout passed : P2Y3M4DT5H6M 2011-08-09 16:16:52,540 8132:140568155903744: pulp.server.api.repo_sync:INFO: repo_sync:222 Sync invoked for repo <f14> 2011-08-09 16:16:52,596 8132:140569671800576: pulp:CRITICAL: queue:128 Exception in FIFO Queue Dispatch Thread Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/pulp/server/tasking/taskqueue/queue.py", line 124, in _dispatch self._timeout_tasks() File "/usr/lib/python2.7/site-packages/pulp/server/tasking/taskqueue/queue.py", line 183, in _timeout_tasks if now - task.start_time < task.timeout_delta: TypeError: can't compare datetime.timedelta to Duration After restarting pulp-server got ISE using pulp cli commands Also saw this in ssl_log [Tue Aug 09 18:25:57 2011] [error] [client 127.0.0.1] File "/usr/lib/python2.7 /site-packages/isodate/duration.py", line 95, in __getattr__ [Tue Aug 09 18:25:57 2011] [error] [client 127.0.0.1] return getattr(self.td elta, name) [Tue Aug 09 18:25:57 2011] [error] [client 127.0.0.1] File "/usr/lib/python2.7 /site-packages/isodate/duration.py", line 95, in __getattr__ [Tue Aug 09 18:25:57 2011] [error] [client 127.0.0.1] return getattr(self.td elta, name) [Tue Aug 09 18:25:57 2011] [error] [client 127.0.0.1] File "/usr/lib/python2.7 /site-packages/isodate/duration.py", line 95, in __getattr__ [Tue Aug 09 18:25:57 2011] [error] [client 127.0.0.1] return getattr(self.td elta, name) [Tue Aug 09 18:25:57 2011] [error] [client 127.0.0.1] RuntimeError: maximum recu rsion depth exceeded Version-Release number of selected component (if applicable): [root@preethi ~]# rpm -q pulp pulp-0.0.223-1.fc14.noarch [root@preethi ~]#