Bug 976858 - possible race condition in syncs with shared content
possible race condition in syncs with shared content
Status: CLOSED CURRENTRELEASE
Product: Pulp
Classification: Community
Component: rpm-support (Show other bugs)
2.1.1
Unspecified Unspecified
low Severity medium
: ---
: 2.3.0
Assigned To: Sayli Karmarkar
Preethi Thomas
: Triaged
Depends On:
Blocks: 950743 1017353
  Show dependency treegraph
 
Reported: 2013-06-21 12:32 EDT by Justin Sherrill
Modified: 2015-03-22 21:11 EDT (History)
7 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1017353 (view as bug list)
Environment:
Last Closed: 2013-12-09 09:31:05 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
traceback (11.35 KB, text/plain)
2013-06-28 15:29 EDT, Jason Connor
no flags Details

  None (edit)
Description Justin Sherrill 2013-06-21 12:32:04 EDT
Description of problem:


There seems to be a race condition when syncing multiple repos that all share content.  This probably isn't reproducible a lot, but omachiel saw it. 

 

Version-Release number of selected component (if applicable):
2.1.2


How reproducible:
Rarely


Steps to Reproduce:
1.  Create multiple repos that all point to the same upstream  repo 
2.  Sync them all at once


Do this enough times with new repos and one may fail



Additional info:

Traceback observed:


2013-06-21 08:55:40,881 pulp.plugins.conduits.mixins:ERROR: Exception from server requesting unit filename for relative path [.//netpbm-devel/10.35.58/10.el5/i386/d19e7c48d06b706bc8413779f5393833755a82c4/netpbm-devel-10.35.58-10.el5.i386.rpm]
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/pulp/plugins/conduits/mixins.py", line 432, in init_unit
    path = content_query_manager.request_content_unit_file_path(type_id, relative_path)
  File "/usr/lib/python2.6/site-packages/pulp/server/managers/content/query.py", line 283, in request_content_unit_file_path
    os.makedirs(unit_dir)
  File "/usr/lib64/python2.6/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 17] File exists: '/var/lib/pulp/content/rpm/.//netpbm-devel/10.35.58/10.el5/i386/d19e7c48d06b706bc8413779f5393833755a82c4'
Comment 1 Michael Hrivnak 2013-06-21 13:43:47 EDT
Can you reproduce this on a clean install or at least a different system? I'm wondering if something so seemingly-unlikely could be the result of an anomaly on that system.
Comment 2 Justin Sherrill 2013-06-21 13:49:44 EDT
Adding omaciel to the cc list, maybe he can try reproduce.  I'm guessing its not easily reproducible.
Comment 3 Jason Connor 2013-06-28 15:29:40 EDT
Created attachment 766709 [details]
traceback
Comment 4 Og Maciel 2013-06-28 15:31:09 EDT
Reproduced it today while syncing RHEL 5 and RHEL 6 RPMs for both i386 and x86_64
Comment 5 Michael Hrivnak 2013-06-28 15:33:03 EDT
This is likely already fixed in the 2.2.0 beta release. Will re-visit this once 2.2.0 stable is released.
Comment 6 Randy Barlow 2013-09-05 15:33:51 EDT
Marking as low priority, as there is a high likelyhood that this issue is resolved by the new importer.
Comment 7 Sayli Karmarkar 2013-09-06 19:53:58 EDT
Created 3 repos with the same feed and different relative-urls and synced them at the same time with more than 3000 packages. All syncs completed without errors or race condition. Seems like this is fixed in the new importer. Moving to on_qa.
Comment 8 Preethi Thomas 2013-09-19 17:26:28 EDT
moving to verified
[root@ibm-x3655-04 ~]# rpm -qa pulp-server
pulp-server-2.3.0-0.14.alpha.el6.noarch
[root@ibm-x3655-04 ~]# 


I created & synced 4 repos all rhel 6-2, 6-3, 6-4 and 6-3 i386 at the same time and all of the syncs finished without error

[root@ibm-x3655-04 ~]# pulp-admin repo list
+----------------------------------------------------------------------+
                              Repositories
+----------------------------------------------------------------------+

Id:                  foo
Display Name:        foo
Description:         None
Content Unit Counts: 

Id:                  rhel6-2
Display Name:        rhel6-2
Description:         None
Content Unit Counts: 
  Distribution:           1
  Erratum:                1048
  Package Category:       10
  Package Group:          202
  Rpm:                    7281
  Yum Repo Metadata File: 1

Id:                  rhel6-386
Display Name:        rhel6-386
Description:         None
Content Unit Counts: 
  Distribution:           1
  Erratum:                1415
  Package Category:       10
  Package Group:          202
  Rpm:                    6834
  Yum Repo Metadata File: 1

Id:                  rhel6-3
Display Name:        rhel6-3
Description:         None
Content Unit Counts: 
  Distribution:           1
  Erratum:                1476
  Package Category:       10
  Package Group:          202
  Rpm:                    8921
  Yum Repo Metadata File: 1

Id:                  rhel6-4
Display Name:        rhel6-4
Description:         None
Content Unit Counts: 
  Distribution:           1
  Erratum:                1935
  Package Category:       10
  Package Group:          202
  Rpm:                    10923
  Yum Repo Metadata File: 1


[root@ibm-x3655-04 ~]#
Comment 9 Preethi Thomas 2013-12-09 09:31:05 EST
Pulp 2.3 released.

Note You need to log in before you can comment on or make changes to this bug.