Bug 1017353

+++ This bug was initially created as a clone of Bug #976858 +++

Description of problem:


There seems to be a race condition when syncing multiple repos that all share content.  This probably isn't reproducible a lot, but omachiel saw it. 

 

Version-Release number of selected component (if applicable):
2.1.2


How reproducible:
Rarely


Steps to Reproduce:
1.  Create multiple repos that all point to the same upstream  repo 
2.  Sync them all at once


Do this enough times with new repos and one may fail



Additional info:

Traceback observed:


2013-06-21 08:55:40,881 pulp.plugins.conduits.mixins:ERROR: Exception from server requesting unit filename for relative path [.//netpbm-devel/10.35.58/10.el5/i386/d19e7c48d06b706bc8413779f5393833755a82c4/netpbm-devel-10.35.58-10.el5.i386.rpm]
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/pulp/plugins/conduits/mixins.py", line 432, in init_unit
    path = content_query_manager.request_content_unit_file_path(type_id, relative_path)
  File "/usr/lib/python2.6/site-packages/pulp/server/managers/content/query.py", line 283, in request_content_unit_file_path
    os.makedirs(unit_dir)
  File "/usr/lib64/python2.6/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 17] File exists: '/var/lib/pulp/content/rpm/.//netpbm-devel/10.35.58/10.el5/i386/d19e7c48d06b706bc8413779f5393833755a82c4'

--- Additional comment from Michael Hrivnak on 2013-06-21 13:43:47 EDT ---

Can you reproduce this on a clean install or at least a different system? I'm wondering if something so seemingly-unlikely could be the result of an anomaly on that system.

--- Additional comment from Justin Sherrill on 2013-06-21 13:49:44 EDT ---

Adding omaciel to the cc list, maybe he can try reproduce.  I'm guessing its not easily reproducible.

--- Additional comment from Jason Connor on 2013-06-28 15:29:40 EDT ---



--- Additional comment from Og Maciel on 2013-06-28 15:31:09 EDT ---

Reproduced it today while syncing RHEL 5 and RHEL 6 RPMs for both i386 and x86_64

--- Additional comment from Michael Hrivnak on 2013-06-28 15:33:03 EDT ---

This is likely already fixed in the 2.2.0 beta release. Will re-visit this once 2.2.0 stable is released.

--- Additional comment from Randy Barlow on 2013-09-05 15:33:51 EDT ---

Marking as low priority, as there is a high likelyhood that this issue is resolved by the new importer.

--- Additional comment from Sayli Karmarkar on 2013-09-06 19:53:58 EDT ---

Created 3 repos with the same feed and different relative-urls and synced them at the same time with more than 3000 packages. All syncs completed without errors or race condition. Seems like this is fixed in the new importer. Moving to on_qa.

--- Additional comment from Preethi Thomas on 2013-09-19 17:26:28 EDT ---

moving to verified
[root@ibm-x3655-04 ~]# rpm -qa pulp-server
pulp-server-2.3.0-0.14.alpha.el6.noarch
[root@ibm-x3655-04 ~]# 


I created & synced 4 repos all rhel 6-2, 6-3, 6-4 and 6-3 i386 at the same time and all of the syncs finished without error

[root@ibm-x3655-04 ~]# pulp-admin repo list
+----------------------------------------------------------------------+
                              Repositories
+----------------------------------------------------------------------+

Id:                  foo
Display Name:        foo
Description:         None
Content Unit Counts: 

Id:                  rhel6-2
Display Name:        rhel6-2
Description:         None
Content Unit Counts: 
  Distribution:           1
  Erratum:                1048
  Package Category:       10
  Package Group:          202
  Rpm:                    7281
  Yum Repo Metadata File: 1

Id:                  rhel6-386
Display Name:        rhel6-386
Description:         None
Content Unit Counts: 
  Distribution:           1
  Erratum:                1415
  Package Category:       10
  Package Group:          202
  Rpm:                    6834
  Yum Repo Metadata File: 1

Id:                  rhel6-3
Display Name:        rhel6-3
Description:         None
Content Unit Counts: 
  Distribution:           1
  Erratum:                1476
  Package Category:       10
  Package Group:          202
  Rpm:                    8921
  Yum Repo Metadata File: 1

Id:                  rhel6-4
Display Name:        rhel6-4
Description:         None
Content Unit Counts: 
  Distribution:           1
  Erratum:                1935
  Package Category:       10
  Package Group:          202
  Rpm:                    10923
  Yum Repo Metadata File: 1


[root@ibm-x3655-04 ~]#