Bug 912766

relevant tracebacks:


Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/pulp/server/tasking/task.py", line 418, in run
    result = self.callable(*self.args, **self.kwargs)
  File "/usr/lib/python2.6/site-packages/pulp/server/api/repo_sync.py", line 283, in _sync
    progress_callback, synchronizer, max_speed, threads)
  File "/usr/lib/python2.6/site-packages/pulp/server/api/repo_sync.py", line 379, in fetch_content
    synchronizer.add_distribution_from_dir(repo_dir, repo_id, skip_dict)
  File "/usr/lib/python2.6/site-packages/pulp/server/api/synchronizers.py", line 347, in add_distribution_from_dir
    self._process_repo_images(dir, repo)
  File "/usr/lib/python2.6/site-packages/pulp/server/api/synchronizers.py", line 484, in _process_repo_images
    os.mkdir(distro_path)
OSError: [Errno 17] File exists: '/var/lib/pulp/published/ks'


upon trying to re-sync:



Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/pulp/server/tasking/task.py", line 418, in run
    result = self.callable(*self.args, **self.kwargs)
  File "/usr/lib/python2.6/site-packages/pulp/server/api/repo_sync.py", line 283, in _sync
    progress_callback, synchronizer, max_speed, threads)
  File "/usr/lib/python2.6/site-packages/pulp/server/api/repo_sync.py", line 372, in fetch_content
    progress_callback, max_speed, threads)
  File "/usr/lib/python2.6/site-packages/pulp/server/api/synchronizers.py", line 164, in sync
    max_speed=max_speed, threads=threads)
  File "/usr/lib/python2.6/site-packages/pulp/server/api/synchronizers.py", line 811, in remote
    verify_options=verify_options)
  File "/usr/lib/python2.6/site-packages/grinder/RepoFetch.py", line 204, in fetchYumRepo
    self.setupDistroInfo()
  File "/usr/lib/python2.6/site-packages/grinder/RepoFetch.py", line 154, in setupDistroInfo
    distro_items = info.prepareTrees(self.repoFetch)
  File "/usr/lib/python2.6/site-packages/grinder/DistroInfo.py", line 42, in prepareTrees
    return self.__prepareTrees(tree_manifest)
  File "/usr/lib/python2.6/site-packages/grinder/DistroInfo.py", line 126, in __prepareTrees
    os.makedirs(info['pkgpath'])
  File "/usr/lib64/python2.6/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 17] File exists: '/var/lib/pulp/distributions/ks-Red Hat Enterprise Linux-Server-6.4-x86_64'

Discussion with prad:

jsherrill | anyone have a moment for a pulpv1 sync issue?                                                                      │
jsherrill | bugzilla is here https://bugzilla.redhat.com/show_bug.cgi?id=887027,  but basically when we sync multiple repos at │
          | the same time, i think we might be hitting a race condition with regard to distributions                           │
jsherrill | jdob: prad-  ^                                                                                                     │
    prad- | jsherrill, looking                                                                                                 │
jsherrill | prad-: thanks!                                                                                                     │
jsherrill | prad-: it was filed a katello bug, but i've put the relevant pulp tracebacks at the bottom                         │
    prad- | hmm looks like a race condition to me where, multiple threads are trying to create that dir/symlink                │
jsherrill | right, that's kinda what i was thinking                                                                            │
    prad- | we might need some locking around it looks like                                                                    │
jsherrill | prad-: would that be a large/difficult effort to do?                                                               │
    prad- | jsherrill, shouldnt be, as we do something similar in other areas, so we should be able to reuse some of it        │

Created attachment 699503 [details]
pulp.log with errors (bziped for size)

I believe we are no longer supporting Pulp v1.