Bug 1006629 - simultaneous sync causes IO/filesystem error
Summary: simultaneous sync causes IO/filesystem error
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Pulp
Classification: Retired
Component: rpm-support   
(Show other bugs)
Version: 2.2
Hardware: Unspecified
OS: Unspecified
urgent
high
Target Milestone: ---
: 2.2.0
Assignee: Jason Connor
QA Contact: pulp-qe-list
URL:
Whiteboard:
Keywords: Triaged
Depends On:
Blocks: 950743
TreeView+ depends on / blocked
 
Reported: 2013-09-11 00:51 UTC by Mike McCune
Modified: 2014-03-31 01:40 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-09-11 23:06:16 UTC
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

Description Mike McCune 2013-09-11 00:51:56 UTC
I was running 3-4 syncs on a series of repos and for newly queued syncs while a longer one was running I received the error pasted below.


steps to reproduce:

1) via katello sync RHEL 6.4 x86_64 from Red Hat's CDN

2) setup a series of custom repositories for testing with feeds at:

http://mmccune.fedorapeople.org/repos/demo/repo-1/
http://mmccune.fedorapeople.org/repos/demo/repo-2/
http://mmccune.fedorapeople.org/repos/demo/repo-3/

3) initiate a sync of RHEL 6.4, once that is running attempt to sync any of the above repositories

4) you get:

2013-09-10 20:45:04,296 urllib3.connectionpool:INFO: Starting new HTTP connection (1): mmccune.fedorapeople.org
2013-09-10 20:45:04,429 nectar.downloaders.threaded:ERROR: Download of http://mmccune.fedorapeople.org/repos/demo/repo-2/treeinfo failed with code 404: Not Found
2013-09-10 20:45:04,468 pulp_rpm.plugins.importers.yum.sync:ERROR: sync failed
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 135, in run
    self.get_errata(metadata_files)
  File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 415, in get_errata
    errata_file_handle = metadata_files.get_metadata_file_handle(updateinfo.METADATA_FILE_NAME)
  File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/repomd/metadata.py", line 248, in get_metadata_file_handle
    file_handle = gzip.open(file_path, 'r')
  File "/usr/lib64/python2.6/gzip.py", line 33, in open
    return GzipFile(filename, mode, compresslevel)
  File "/usr/lib64/python2.6/gzip.py", line 79, in __init__
    fileobj = self.myfileobj = __builtin__.open(filename, mode or 'rb')
IOError: [Errno 2] No such file or directory: u'/var/lib/pulp/working/repos/Zombo_Inc-Finance_Tracker-web-app-x86_64/importers/yum_importer/tmpJKvnJn/67d40f6282e5858c723bad931dc17a462195a6aefcbd304de26e18615f82afdd-updateinfo.xml.gz'

Once the larger RHEL repo finishes I'll see if I can sync the demo repos by themselves.

Comment 1 Mike McCune 2013-09-11 04:30:02 UTC
if you repeatedly re-sync the repos after the error state they will eventually sync correctly

Comment 2 Ivan Necas 2013-09-11 12:57:46 UTC
I'm running into similar issue as well, seeing:

2013-09-11 07:53:00,750 nectar.downloaders.threaded:ERROR: Request timed out.
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/nectar/downloaders/threaded.py", line 225, in _fetch
    response = session.get(request.url, headers=headers)
  File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 310, in get
    return self.request('GET', url, **kwargs)
  File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 279, in request
    resp = self.send(prep, stream=stream, timeout=timeout, verify=verify, cert=cert, proxies=proxies)
  File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 374, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python2.6/site-packages/requests/adapters.py", line 217, in send
    raise Timeout('Request timed out.')
Timeout: Request timed out.


And

2013-09-11 07:54:17,499 pulp_rpm.plugins.importers.yum.parse.treeinfo:ERROR: some distro file downloads failed

Comment 3 Mike McCune 2013-09-11 15:09:04 UTC
Using:

pulp-nodes-common-2.2.0-1.el6.noarch
pulp-nodes-parent-2.2.0-1.el6.noarch
pulp-puppet-plugins-2.2.0-1.el6.noarch
pulp-rpm-plugins-2.2.0-1.el6.noarch
pulp-selinux-2.2.0-1.el6.noarch
pulp-server-2.2.0-1.el6.noarch

Comment 4 Jason Connor 2013-09-11 23:06:16 UTC
This bug is really just a failure to include the latest nectar (1.1.1) as a dependency for Pulp 2.2


Note You need to log in before you can comment on or make changes to this bug.