I'm running RHUI on a default Pulp installation (more than one task queue thread). When I schedule more than one sync to occur, they randomly fail. So far I've not seen the issue when running a single sync at a time. This appeared while synccing 4 repos (RHUI 1.2, RPMs and SRPMS, i386 and x86_64). Two of the 4 failed. 2011-04-12 15:03:45,158 [INFO][Dummy-2] sync() @ repositories.py:557 - sync timeout passed : None 2011-04-12 15:03:45,178 [INFO][Thread-202] _sync() @ repo.py:1453 - Sync of rhui-1.2-5Server-x86_64 starting, skip_dict = {} 2011-04-12 15:03:45,203 [ERROR][Thread-186] failed() @ task.py:303 - Task id:96ef297a-6537-11e0-92c9-00508d977dff, method_name:_sync: Traceback (most recent call last): File "/home/jdob/code/pulp/src/pulp/server/tasking/task.py", line 251, in run result = self.callable(*self.args, **self.kwargs) File "/home/jdob/code/pulp/src/pulp/server/api/repo.py", line 1461, in _sync synchronizer) File "/home/jdob/code/pulp/src/pulp/server/api/repo_sync.py", line 134, in sync repo_dir = synchronizer.sync(repo, repo_source, skip_dict, progress_callback) File "/home/jdob/code/pulp/src/pulp/server/api/repo_sync.py", line 429, in sync report = self.yum_repo_grinder.fetchYumRepo(store_path, callback=progress_callback) File "/home/jdob/code/grinder/src/grinder/RepoFetch.py", line 355, in fetchYumRepo self.yumFetch.getRepoData() File "/home/jdob/code/grinder/src/grinder/RepoFetch.py", line 140, in getRepoData for ftype in self.getRepoXmlFileTypes(): File "/home/jdob/code/grinder/src/grinder/RepoFetch.py", line 126, in getRepoXmlFileTypes return self.repo.repoXML.fileTypes() File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1413, in <lambda> repoXML = property(fget=lambda self: self._getRepoXML(), File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1409, in _getRepoXML raise Errors.RepoError, msg RepoError: Cannot retrieve repository metadata (repomd.xml) for repository: . Please verify its path and try again 2011-04-12 15:03:45,213 [INFO][Dummy-5] sync() @ repositories.py:557 - sync timeout passed : None 2011-04-12 15:03:45,247 [INFO][Thread-218] _sync() @ repo.py:1453 - Sync of rhui-srpms-1.2-5Server-i386 starting, skip_dict = {} 2011-04-12 15:03:45,266 [INFO][Dummy-2] sync() @ repositories.py:557 - sync timeout passed : None 2011-04-12 15:03:45,277 [INFO][Thread-234] _sync() @ repo.py:1453 - Sync of rhui-srpms-1.2-5Server-x86_64 starting, skip_dict = {} 2011-04-12 15:03:45,373 [ERROR][Thread-218] failed() @ task.py:303 - Task id:9700fdb5-6537-11e0-933e-00508d977dff, method_name:_sync: Traceback (most recent call last): File "/home/jdob/code/pulp/src/pulp/server/tasking/task.py", line 251, in run result = self.callable(*self.args, **self.kwargs) File "/home/jdob/code/pulp/src/pulp/server/api/repo.py", line 1461, in _sync synchronizer) File "/home/jdob/code/pulp/src/pulp/server/api/repo_sync.py", line 134, in sync repo_dir = synchronizer.sync(repo, repo_source, skip_dict, progress_callback) File "/home/jdob/code/pulp/src/pulp/server/api/repo_sync.py", line 429, in sync report = self.yum_repo_grinder.fetchYumRepo(store_path, callback=progress_callback) File "/home/jdob/code/grinder/src/grinder/RepoFetch.py", line 355, in fetchYumRepo self.yumFetch.getRepoData() File "/home/jdob/code/grinder/src/grinder/RepoFetch.py", line 140, in getRepoData for ftype in self.getRepoXmlFileTypes(): File "/home/jdob/code/grinder/src/grinder/RepoFetch.py", line 126, in getRepoXmlFileTypes return self.repo.repoXML.fileTypes() File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1413, in <lambda> repoXML = property(fget=lambda self: self._getRepoXML(), File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1409, in _getRepoXML raise Errors.RepoError, msg RepoError: Cannot retrieve repository metadata (repomd.xml) for repository: . Please verify its path and try again
Note: this is from the RHUI branch. If this was fixed in master, we need to make sure the fix has made it to the RHUI branch as well.
Pushed associated commits to rhui branch on 4/13/11. There is a specific unit tests to test syncing 6 repos (containing pulp qe builds). unit{rhui}$ nosetests test_repo_sync.py:TestRepoSync.test_sync_multiple_repos . ---------------------------------------------------------------------- Ran 1 test in 21.536s OK
build: 0.168
Select one or more repositories to schedule to be synchronized before its scheduled time. The sync will happen as soon as possible depending on other tasks that may be executing in the RHUI. Only repositories that are not currently synchronizing are displayed. Last Result Next Sync Repository ------------------------------------------ - 1 : scheduled 05-16-2011 22:38 Red Hat Enterprise Linux Server (RPMs) (5Server-x86_64) - 2 : scheduled 05-16-2011 22:6 Red Hat Enterprise Linux Server 6 Optional Releases (SRPMS) (6Server-i386) - 3 : scheduled 05-16-2011 22:6 Red Hat Enterprise Linux Server 6 Optional Releases (SRPMS) (6Server-x86_64) - 4 : scheduled 05-16-2011 22:6 Red Hat Enterprise Linux Server 6 Optional Updates (RPMs) (6Server-i386) - 5 : scheduled 05-16-2011 22:6 Red Hat Enterprise Linux Server 6 Optional Updates (RPMs) (6Server-x86_64) x 6 : scheduled 05-17-2011 1:20 Red Hat Enterprise Linux Server 6 Optional Updates (SRPMS) (6Server-i386) x 7 : scheduled 05-17-2011 1:20 Red Hat Enterprise Linux Server 6 Optional Updates (SRPMS) (6Server-x86_64) - 8 : scheduled 05-16-2011 22:38 Red Hat Enterprise Linux Server 6 Releases (RPMs) (6Server-x86_64) - 9 : scheduled 05-16-2011 22:7 Red Hat Update Infrastructure 1.2 (RPMs) (5Server-i386) - 10: scheduled 05-16-2011 22:38 Red Hat Update Infrastructure 1.2 (RPMs) (5Server-x86_64) x 11: scheduled 05-17-2011 1:20 Red Hat Update Infrastructure 1.2 (SRPMS) (5Server-i386) x 12: scheduled 05-17-2011 1:20 Red Hat Update Infrastructure 1.2 (SRPMS) (5Server-x86_64) Enter value from (1-12) to toggle selection, 'c' to confirm selections, or '?' for more commands: c The following repositories will be scheduled for synchronization: Red Hat Enterprise Linux Server 6 Optional Updates (SRPMS) (6Server-i386) Red Hat Enterprise Linux Server 6 Optional Updates (SRPMS) (6Server-x86_64) Red Hat Update Infrastructure 1.2 (SRPMS) (5Server-i386) Red Hat Update Infrastructure 1.2 (SRPMS) (5Server-x86_64) Proceed? (y/n) y Scheduling sync for Red Hat Enterprise Linux Server 6 Optional Updates (SRPMS) (6Server-i386)... ... successfully scheduled for the next available timeslot. Scheduling sync for Red Hat Enterprise Linux Server 6 Optional Updates (SRPMS) (6Server-x86_64)... ... successfully scheduled for the next available timeslot. Scheduling sync for Red Hat Update Infrastructure 1.2 (SRPMS) (5Server-i386)... ... successfully scheduled for the next available timeslot. Scheduling sync for Red Hat Update Infrastructure 1.2 (SRPMS) (5Server-x86_64)... ... successfully scheduled for the next available timeslot. ------------------------------------------------------------------------------ rhui (sync) => dr Tried syncing multiple repos and it worked fine. [root@pulp-server ~]# rpm -qav | grep -ie pulp -ie rhui pulp-0.0.171-1.el6.noarch rh-rhui-tools-2.0.22-1.el6.noarch pulp-common-0.0.171-1.el6.noarch pulp-client-0.0.171-1.el6.noarch
Closing with Community Release 15 pulp-0.0.223-4.