Bug 695826

Summary:	Error running multiple concurrent syncs
Product:	[Retired] Pulp	Reporter:	Jay Dobies <jason.dobies>
Component:	z_other	Assignee:	John Matthews <jmatthew>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Preethi Thomas <pthomas>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	unspecified	CC:	jmatthew, kbidarka
Target Milestone:	---	Keywords:	Triaged
Target Release:	Sprint 22
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2011-08-16 13:59:38 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	641987, 688298

Description Jay Dobies 2011-04-12 19:03:10 UTC

I'm running RHUI on a default Pulp installation (more than one task queue thread). When I schedule more than one sync to occur, they randomly fail. So far I've not seen the issue when running a single sync at a time.

This appeared while synccing 4 repos (RHUI 1.2, RPMs and SRPMS, i386 and x86_64). Two of the 4 failed.

2011-04-12 15:03:45,158 [INFO][Dummy-2] sync() @ repositories.py:557 - sync timeout passed : None
2011-04-12 15:03:45,178 [INFO][Thread-202] _sync() @ repo.py:1453 - Sync of rhui-1.2-5Server-x86_64 starting, skip_dict = {}
2011-04-12 15:03:45,203 [ERROR][Thread-186] failed() @ task.py:303 - Task id:96ef297a-6537-11e0-92c9-00508d977dff, method_name:_sync:
Traceback (most recent call last):
  File "/home/jdob/code/pulp/src/pulp/server/tasking/task.py", line 251, in run
    result = self.callable(*self.args, **self.kwargs)
  File "/home/jdob/code/pulp/src/pulp/server/api/repo.py", line 1461, in _sync
    synchronizer)
  File "/home/jdob/code/pulp/src/pulp/server/api/repo_sync.py", line 134, in sync
    repo_dir = synchronizer.sync(repo, repo_source, skip_dict, progress_callback)
  File "/home/jdob/code/pulp/src/pulp/server/api/repo_sync.py", line 429, in sync
    report = self.yum_repo_grinder.fetchYumRepo(store_path, callback=progress_callback)
  File "/home/jdob/code/grinder/src/grinder/RepoFetch.py", line 355, in fetchYumRepo
    self.yumFetch.getRepoData()
  File "/home/jdob/code/grinder/src/grinder/RepoFetch.py", line 140, in getRepoData
    for ftype in self.getRepoXmlFileTypes():
  File "/home/jdob/code/grinder/src/grinder/RepoFetch.py", line 126, in getRepoXmlFileTypes
    return self.repo.repoXML.fileTypes()
  File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1413, in <lambda>
    repoXML = property(fget=lambda self: self._getRepoXML(),
  File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1409, in _getRepoXML
    raise Errors.RepoError, msg
RepoError: Cannot retrieve repository metadata (repomd.xml) for repository: . Please verify its path and try again

2011-04-12 15:03:45,213 [INFO][Dummy-5] sync() @ repositories.py:557 - sync timeout passed : None
2011-04-12 15:03:45,247 [INFO][Thread-218] _sync() @ repo.py:1453 - Sync of rhui-srpms-1.2-5Server-i386 starting, skip_dict = {}
2011-04-12 15:03:45,266 [INFO][Dummy-2] sync() @ repositories.py:557 - sync timeout passed : None
2011-04-12 15:03:45,277 [INFO][Thread-234] _sync() @ repo.py:1453 - Sync of rhui-srpms-1.2-5Server-x86_64 starting, skip_dict = {}
2011-04-12 15:03:45,373 [ERROR][Thread-218] failed() @ task.py:303 - Task id:9700fdb5-6537-11e0-933e-00508d977dff, method_name:_sync:
Traceback (most recent call last):
  File "/home/jdob/code/pulp/src/pulp/server/tasking/task.py", line 251, in run
    result = self.callable(*self.args, **self.kwargs)
  File "/home/jdob/code/pulp/src/pulp/server/api/repo.py", line 1461, in _sync
    synchronizer)
  File "/home/jdob/code/pulp/src/pulp/server/api/repo_sync.py", line 134, in sync
    repo_dir = synchronizer.sync(repo, repo_source, skip_dict, progress_callback)
  File "/home/jdob/code/pulp/src/pulp/server/api/repo_sync.py", line 429, in sync
    report = self.yum_repo_grinder.fetchYumRepo(store_path, callback=progress_callback)
  File "/home/jdob/code/grinder/src/grinder/RepoFetch.py", line 355, in fetchYumRepo
    self.yumFetch.getRepoData()
  File "/home/jdob/code/grinder/src/grinder/RepoFetch.py", line 140, in getRepoData
    for ftype in self.getRepoXmlFileTypes():
  File "/home/jdob/code/grinder/src/grinder/RepoFetch.py", line 126, in getRepoXmlFileTypes
    return self.repo.repoXML.fileTypes()
  File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1413, in <lambda>
    repoXML = property(fget=lambda self: self._getRepoXML(),
  File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1409, in _getRepoXML
    raise Errors.RepoError, msg
RepoError: Cannot retrieve repository metadata (repomd.xml) for repository: . Please verify its path and try again

Comment 1 Jay Dobies 2011-04-12 19:04:35 UTC

Note: this is from the RHUI branch. If this was fixed in master, we need to make sure the fix has made it to the RHUI branch as well.

Comment 2 John Matthews 2011-04-13 19:44:05 UTC

Pushed associated commits to rhui branch on 4/13/11.  

There is a specific unit tests to test syncing 6 repos (containing pulp qe builds).


unit{rhui}$ nosetests test_repo_sync.py:TestRepoSync.test_sync_multiple_repos
.
----------------------------------------------------------------------
Ran 1 test in 21.536s

OK

Comment 3 Jeff Ortel 2011-04-19 18:49:17 UTC

build: 0.168

Comment 4 Kedar Bidarkar 2011-05-16 12:02:39 UTC

Select one or more repositories to schedule to be synchronized before its scheduled time.
The sync will happen as soon as possible depending on other tasks that may be executing
in the RHUI. Only repositories that are not currently synchronizing are displayed.

         Last Result   Next Sync         Repository
         ------------------------------------------
  -  1 : scheduled  05-16-2011 22:38  Red Hat Enterprise Linux Server (RPMs) (5Server-x86_64)
  -  2 : scheduled  05-16-2011 22:6   Red Hat Enterprise Linux Server 6 Optional Releases (SRPMS) (6Server-i386)
  -  3 : scheduled  05-16-2011 22:6   Red Hat Enterprise Linux Server 6 Optional Releases (SRPMS) (6Server-x86_64)
  -  4 : scheduled  05-16-2011 22:6   Red Hat Enterprise Linux Server 6 Optional Updates (RPMs) (6Server-i386)
  -  5 : scheduled  05-16-2011 22:6   Red Hat Enterprise Linux Server 6 Optional Updates (RPMs) (6Server-x86_64)
  x  6 : scheduled  05-17-2011 1:20   Red Hat Enterprise Linux Server 6 Optional Updates (SRPMS) (6Server-i386)
  x  7 : scheduled  05-17-2011 1:20   Red Hat Enterprise Linux Server 6 Optional Updates (SRPMS) (6Server-x86_64)
  -  8 : scheduled  05-16-2011 22:38  Red Hat Enterprise Linux Server 6 Releases (RPMs) (6Server-x86_64)
  -  9 : scheduled  05-16-2011 22:7   Red Hat Update Infrastructure 1.2 (RPMs) (5Server-i386)
  -  10: scheduled  05-16-2011 22:38  Red Hat Update Infrastructure 1.2 (RPMs) (5Server-x86_64)
  x  11: scheduled  05-17-2011 1:20   Red Hat Update Infrastructure 1.2 (SRPMS) (5Server-i386)
  x  12: scheduled  05-17-2011 1:20   Red Hat Update Infrastructure 1.2 (SRPMS) (5Server-x86_64)
Enter value from (1-12) to toggle selection, 'c' to confirm selections, or '?' for more commands: c


The following repositories will be scheduled for synchronization:
  Red Hat Enterprise Linux Server 6 Optional Updates (SRPMS) (6Server-i386)
  Red Hat Enterprise Linux Server 6 Optional Updates (SRPMS) (6Server-x86_64)
  Red Hat Update Infrastructure 1.2 (SRPMS) (5Server-i386)
  Red Hat Update Infrastructure 1.2 (SRPMS) (5Server-x86_64)
Proceed? (y/n) y


Scheduling sync for Red Hat Enterprise Linux Server 6 Optional Updates (SRPMS) (6Server-i386)...
... successfully scheduled for the next available timeslot.
Scheduling sync for Red Hat Enterprise Linux Server 6 Optional Updates (SRPMS) (6Server-x86_64)...
... successfully scheduled for the next available timeslot.
Scheduling sync for Red Hat Update Infrastructure 1.2 (SRPMS) (5Server-i386)...
... successfully scheduled for the next available timeslot.
Scheduling sync for Red Hat Update Infrastructure 1.2 (SRPMS) (5Server-x86_64)...
... successfully scheduled for the next available timeslot.

------------------------------------------------------------------------------
rhui (sync) => dr




Tried syncing multiple repos and it worked fine.


[root@pulp-server ~]# rpm -qav | grep -ie pulp -ie rhui
pulp-0.0.171-1.el6.noarch
rh-rhui-tools-2.0.22-1.el6.noarch
pulp-common-0.0.171-1.el6.noarch
pulp-client-0.0.171-1.el6.noarch

Comment 5 Preethi Thomas 2011-08-16 13:59:38 UTC

Closing with Community Release 15

pulp-0.0.223-4.