Bug 745458 - [RFE] repo clone error if the pulp server is restarted while cloning
Summary: [RFE] repo clone error if the pulp server is restarted while cloning
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Pulp
Classification: Retired
Component: user-experience
Version: 1.0.0
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: Sprint 30
Assignee: Sayli Karmarkar
QA Contact: Preethi Thomas
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-10-12 12:41 UTC by Preethi Thomas
Modified: 2015-03-23 01:11 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Enhancement
Doc Text:
Clone Of:
Environment:
Last Closed: 2012-02-24 20:16:23 UTC


Attachments (Terms of Use)

Description Preethi Thomas 2011-10-12 12:41:49 UTC
Description of problem:
[root@preethi ~]# rpm -q pulp
pulp-0.0.237-4.fc15.noarch
[root@preethi ~]# 

As per the release notes 

Ability to snapshot and restart a running clone in case of server restart was implemented

but after restart you can see this error in the pulp log

PulpException: A Repo with id ---  exists. Choose a different id

Steps
 -Clone a repo
 - While cloning is done restart pulp-server
 - do pulp-admin command (example (pulp-admin repo list) to trigger the cloning

See the error in pulp.log or do a pulp-admin repo status on the clone_id

From pulp.log

2011-10-12 08:45:27,227 20715:140292036466432: pulp.server.tasking.task:ERROR: task:391 Task failed: Task f7a1033a-f4cf-11e0-a280-002564a85a58: _clone(f15-clone, feed=parent, progress_callback=<bound method RepoCloneTask.progress_callback of <pulp.server.api.repo_clone_task.RepoCloneTask object at 0x7f98444a9d90>>, clone_name=f15-clone, id=f15, synchronizer=<pulp.server.api.synchronizers.YumSynchronizer object at 0x7f98444b2250>, filters=[], relative_path=None, groupid=None)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/pulp/server/tasking/task.py", line 340, in run
    result = self.callable(*self.args, **self.kwargs)
  File "/usr/lib/python2.7/site-packages/pulp/server/api/repo_sync.py", line 92, in _clone
    raise PulpException("A Repo with id %s exists. Choose a different id." % clone_id)
PulpException: A Repo with id f15-clone exists. Choose a different id.

Comment 1 Sayli Karmarkar 2011-12-04 23:23:31 UTC
Fixed. 

Started cloning :

$ sudo pulp-admin repo clone --id f15_i386 --clone_id abc -F


Restarted pulp server from other tab: 

$ sudo service pulp-server restart
Stopping httpd:                                            [  OK  ]
Stopping Qpid AMQP daemon:                                 [  OK  ]
Stopping mongod:                                           [  OK  ]
Starting mongod:                                           [  OK  ]
Starting Qpid AMQP daemon:                                 [  OK  ]
Starting httpd: httpd: Could not reliably determine the server's fully qualified domain name, using ::1 for ServerName
                                                           [  OK  ]

After restart is complete:

$ sudo pulp-admin repo status --id abc
+------------------------------------------+
               Status for abc
+------------------------------------------+
Repository: abc
Number of Packages: 0
Last Sync: never
Currently cloning: 66% done (1836 of 2943 packages cloned)

and

tail -f /var/log/pulp/pulp.log

2011-12-04 15:17:23,800 8887:140554766088192: pulp.server.async:INFO: async:147 Loaded Task from database: Task 16bf4fc2-1ece-11e1-a1a3-1c659d8e492e: _clone(abc, feed=parent, progress_callback=<bound method RepoCloneTask.progress_callback of <pulp.server.api.repo_clone_task.RepoCloneTask object at 0x7fd550023fd0>>, clone_name=abc, id=f15_i386, synchronizer=<pulp.server.api.synchronizers.YumSynchronizer object at 0x7fd55003af50>, filters=[], relative_path=abc, groupid=None)
2011-12-04 15:17:23,810 8887:140554362967808: gofer.messaging.broker:INFO: broker:103 {localhost:5672} connected to AMQP
2011-12-04 15:17:24,514 8887:140554247583488: pulp.server.api.repo_sync:INFO: repo_sync:256 Sync invoked for repo <abc>
2011-12-04 15:17:24,518 8887:140554766088192: pulp.server.content.types.parser:INFO: parser:141 Loading type descriptors []
2011-12-04 15:17:24,518 8887:140554766088192: pulp.server.content.types.parser:INFO: parser:143 Parsing type descriptors
2011-12-04 15:17:24,523 8887:140554766088192: pulp.server.content.types.parser:INFO: parser:146 Validating type descriptor syntactic integrity
2011-12-04 15:17:24,523 8887:140554766088192: pulp.server.content.types.parser:INFO: parser:149 Validating type descriptor semantic integrity
2011-12-04 15:17:24,524 8887:140554766088192: pulp.server.content.types.database:INFO: database:83 Updating the database with types []
2011-12-04 15:17:24,523 8887:140554247583488: pulp.server.api.repo_sync:INFO: repo_sync:270 Sync of abc starting, skip_dict = {}
2011-12-04 15:17:24,525 8887:140554766088192: pulp.server.content.loader:WARNING: loader:383 Profilers load called, but not implemented
2011-12-04 15:17:24,527 8887:140554247583488: pulp.server.api.synchronizers:INFO: synchronizers:1091 sync of /var/lib/pulp//repos/fedora/linux/releases/15/Fedora/i386/os for repo abc
2011-12-04 15:17:24,575 8887:140554247583488: pulp.server.api.synchronizers:INFO: synchronizers:789 Found 2943 packages in /var/lib/pulp//repos/fedora/linux/releases/15/Fedora/i386/os
2011-12-04 15:17:24,653 8887:140554247583488: pulp.server.api.synchronizers:INFO: synchronizers:801 Found 0 delta rpm packages in /var/lib/pulp//repos/fedora/linux/releases/15/Fedora/i386/os
2011-12-04 15:17:27,732 8887:140554247583488: pulp.server.api.synchronizers:INFO: synchronizers:882 Working on 0/2943
2011-12-04 15:17:44,975 8887:140554247583488: pulp.server.api.synchronizers:INFO: synchronizers:882 Working on 500/2943
2011-12-04 15:18:00,478 8887:140554247583488: pulp.server.api.synchronizers:INFO: synchronizers:882 Working on 1000/2943


Finally repo status shows,

$ sudo pulp-admin repo status --id abc
+------------------------------------------+
               Status for abc
+------------------------------------------+
Repository: abc
Number of Packages: 0
Last Sync: never
Currently cloning: 100% done (2943 of 2943 packages cloned)

Comment 2 Sayli Karmarkar 2011-12-04 23:34:50 UTC
commit 76691c794e32b8bdba6cdcf254811d271f3d8a97
Author: Sayli Karmarkar <skarmark>
Date:   Sun Dec 4 15:32:44 2011 -0800

    745458 - Separated asynchronous operations in cloning process from synchronous operations such that clone task can be reloaded from db from a saved state in case p

Comment 3 Jeff Ortel 2011-12-12 14:14:45 UTC
build: 0.254-5

Comment 4 Preethi Thomas 2011-12-21 18:09:19 UTC
verified
[root@preethi ~]# rpm -q pulp
pulp-0.0.255-1.fc15.noarch
[root@preethi ~]# 

[root@preethi ~]# sudo pulp-admin repo status --id abc

+------------------------------------------+
               Status for abc
+------------------------------------------+
Repository: abc
Number of Packages: 0
Last Sync: never
Currently cloning: 29% done (1075 of 2940 packages cloned)

[root@preethi ~]# 
[root@preethi ~]# sudo pulp-admin repo status --id abc
+------------------------------------------+
               Status for abc
+------------------------------------------+
Repository: abc
Number of Packages: 0
Last Sync: never
Currently cloning: 100% done (2940 of 2940 packages cloned)

[root@preethi ~]# sudo pulp-admin repo status --id abc
+------------------------------------------+
               Status for abc
+------------------------------------------+
Repository: abc
Number of Packages: 0
Last Sync: never
Currently cloning: 100% done (2940 of 2940 packages cloned)

[root@preethi ~]# sudo pulp-admin repo status --id abc
+------------------------------------------+
               Status for abc
+------------------------------------------+
Repository: abc
Number of Packages: 2935
Last Sync: 2011-12-21 13:17:47-05:00

[root@preethi ~]#

Comment 5 Preethi Thomas 2012-02-24 20:16:23 UTC
Pulp v1.0 is released
Closed Current Release.

Comment 6 Preethi Thomas 2012-02-24 20:17:46 UTC
Pulp v1.0 is released.


Note You need to log in before you can comment on or make changes to this bug.