Bug 765853 - clone fails pickling task objects
Summary: clone fails pickling task objects
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Pulp
Classification: Retired
Component: z_other
Version: unspecified
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: Sprint 30
Assignee: Jason Connor
QA Contact: Preethi Thomas
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-12-09 15:01 UTC by Pradeep Kilambi
Modified: 2014-03-31 01:39 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2012-02-24 20:10:44 UTC


Attachments (Terms of Use)

Description Pradeep Kilambi 2011-12-09 15:01:58 UTC
Description of problem:
2011-12-01 11:24:01,550 11843:140513840776960: pulp.server.tasking.task:ERROR: task:298 Snapshot of Task e033f06e-1c38-11e1-8e00-08002719b328: _clone(ACME_Corporation-Development-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_6_Server_RPMs_61_x86_64, feed=parent, clone_name=Red Hat Enterprise Linux 6 Server RPMs 61 x86_64, relative_path=ACME_Corporation/Development/content/dist/rhel/server/6/6.1/x86_64/os, synchronizer=<pulp.server.api.synchronizers.YumSynchronizer object at 0x7fcbc81e7490>, filters=[], id=ACME_Corporation-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_6_Server_RPMs_61_x86_64, groupid=['product:69', 'env:2', 'org:1']) failed error pickling field: kwargs
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/pulp/server/tasking/task.py", line 295, in snapshot
    data[field] = pickle.dumps(getattr(self, field))
  File "/usr/lib64/python2.6/pickle.py", line 1366, in dumps
    Pickler(file, protocol).dump(obj)
  File "/usr/lib64/python2.6/pickle.py", line 224, in dump
    self.save(obj)
  File "/usr/lib64/python2.6/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib64/python2.6/pickle.py", line 649, in save_dict
    self._batch_setitems(obj.iteritems())
  File "/usr/lib64/python2.6/pickle.py", line 661, in _batch_setitems
    for k, v in items:
RuntimeError: dictionary changed size during iteration

dictionary changed size during iteration
2011-12-01 11:24:01,707 11843:140513840776960: pulp:CRITICAL: queue:130 Exception in FIFO Queue Dispatch Thread
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/pulp/server/tasking/taskqueue/queue.py", line 124, in _dispatch
    self.run(task)
  File "/usr/lib/python2.6/site-packages/pulp/server/tasking/taskqueue/queue.py", line 301, in run
    self.__storage.store_running(task)
  File "/usr/lib/python2.6/site-packages/pulp/server/tasking/taskqueue/storage.py", line 232, in store_running
    self._snapshot_task(task)
  File "/usr/lib/python2.6/site-packages/pulp/server/tasking/taskqueue/storage.py", line 223, in _snapshot_task
    snapshot = task.snapshot()
  File "/usr/lib/python2.6/site-packages/pulp/server/api/repo_clone_task.py", line 65, in snapshot
    snapshot = super(RepoCloneTask, self).snapshot()
  File "/usr/lib/python2.6/site-packages/pulp/server/tasking/task.py", line 295, in snapshot
    data[field] = pickle.dumps(getattr(self, field))
  File "/usr/lib64/python2.6/pickle.py", line 1366, in dumps
    Pickler(file, protocol).dump(obj)
  File "/usr/lib64/python2.6/pickle.py", line 224, in dump
    self.save(obj)
  File "/usr/lib64/python2.6/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib64/python2.6/pickle.py", line 649, in save_dict
    self._batch_setitems(obj.iteritems())
  File "/usr/lib64/python2.6/pickle.py", line 661, in _batch_setitems
    for k, v in items:
SnapshotFailure: Snapshot of Task e033f06e-1c38-11e1-8e00-08002719b328: _clone(ACME_Corporation-Development-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_6_Server_RPMs_61_x86_64, feed=parent, clone_name=Red Hat Enterprise Linux 6 Server RPMs 61 x86_64, relative_path=ACME_Corporation/Development/content/dist/rhel/server/6/6.1/x86_64/os, synchronizer=<pulp.server.api.synchronizers.YumSynchronizer object at 0x7fcbc81e7490>, filters=[], id=ACME_Corporation-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_6_Server_RPMs_61_x86_64, groupid=['product:69', 'env:2', 'org:1']) failed error pickling field: kwargs
dictionary changed size during iteration

Comment 1 Jason Connor 2011-12-09 19:47:11 UTC
This was a race condition between the task being run and the setting of the progress call backs. It affected both repo clone and repo sync.

Fix pushed in: c0555b9c8499e5a5eae93b4cd28d4fb4919b316e

Comment 2 Jeff Ortel 2011-12-12 14:14:43 UTC
build: 0.254-5

Comment 3 Preethi Thomas 2012-01-03 15:23:24 UTC
[root@katello-test ~]# rpm -q pulp
pulp-0.0.255-1.el6.noarch

moving to verified

[root@katello-test ~]# pulp-admin -u admin -p admin  repo sync --id=pulp -F
Sync for repository pulp started
Sync: Finished
20/20 new items downloaded
0/20 existing items processed

Item Details: 
RPMs: 20/20

[root@katello-test ~]# pulp-admin repo clone
Usage: pulp-admin <options> repo clone <options>

pulp-admin: error: Option --id is required; please see --help
[root@katello-test ~]# pulp-admin repo clone --id=pulp --clone-id=pulp-clone
Usage: pulp-admin <options> repo clone <options>

pulp-admin: error: no such option: --clone-id
[root@katello-test ~]# pulp-admin repo clone --id=pulp --clone_id=pulp-clone
error:  error: operation failed: sslv3 alert certificate expired
[root@katello-test ~]# pulp-admin -u admin -p admin repo clone --id=pulp --clone_id=pulp-clone --feed=origin -F
Repository [pulp] is being cloned as [pulp-clone]
Clone: Finished
Item Details: 
RPMs: 20/20

[root@katello-test ~]#

Comment 4 Preethi Thomas 2012-02-24 20:10:44 UTC
Pulp v1.0 is released
Closed Current Release.


Note You need to log in before you can comment on or make changes to this bug.