Bug 1006629 - simultaneous sync causes IO/filesystem error
simultaneous sync causes IO/filesystem error
Status: CLOSED CURRENTRELEASE
Product: Pulp
Classification: Community
Component: rpm-support (Show other bugs)
2.2
Unspecified Unspecified
urgent Severity high
: ---
: 2.2.0
Assigned To: Jason Connor
pulp-qe-list
: Triaged
Depends On:
Blocks: 950743
  Show dependency treegraph
 
Reported: 2013-09-10 20:51 EDT by Mike McCune
Modified: 2014-03-30 21:40 EDT (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-09-11 19:06:16 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Mike McCune 2013-09-10 20:51:56 EDT
I was running 3-4 syncs on a series of repos and for newly queued syncs while a longer one was running I received the error pasted below.


steps to reproduce:

1) via katello sync RHEL 6.4 x86_64 from Red Hat's CDN

2) setup a series of custom repositories for testing with feeds at:

http://mmccune.fedorapeople.org/repos/demo/repo-1/
http://mmccune.fedorapeople.org/repos/demo/repo-2/
http://mmccune.fedorapeople.org/repos/demo/repo-3/

3) initiate a sync of RHEL 6.4, once that is running attempt to sync any of the above repositories

4) you get:

2013-09-10 20:45:04,296 urllib3.connectionpool:INFO: Starting new HTTP connection (1): mmccune.fedorapeople.org
2013-09-10 20:45:04,429 nectar.downloaders.threaded:ERROR: Download of http://mmccune.fedorapeople.org/repos/demo/repo-2/treeinfo failed with code 404: Not Found
2013-09-10 20:45:04,468 pulp_rpm.plugins.importers.yum.sync:ERROR: sync failed
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 135, in run
    self.get_errata(metadata_files)
  File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 415, in get_errata
    errata_file_handle = metadata_files.get_metadata_file_handle(updateinfo.METADATA_FILE_NAME)
  File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/repomd/metadata.py", line 248, in get_metadata_file_handle
    file_handle = gzip.open(file_path, 'r')
  File "/usr/lib64/python2.6/gzip.py", line 33, in open
    return GzipFile(filename, mode, compresslevel)
  File "/usr/lib64/python2.6/gzip.py", line 79, in __init__
    fileobj = self.myfileobj = __builtin__.open(filename, mode or 'rb')
IOError: [Errno 2] No such file or directory: u'/var/lib/pulp/working/repos/Zombo_Inc-Finance_Tracker-web-app-x86_64/importers/yum_importer/tmpJKvnJn/67d40f6282e5858c723bad931dc17a462195a6aefcbd304de26e18615f82afdd-updateinfo.xml.gz'

Once the larger RHEL repo finishes I'll see if I can sync the demo repos by themselves.
Comment 1 Mike McCune 2013-09-11 00:30:02 EDT
if you repeatedly re-sync the repos after the error state they will eventually sync correctly
Comment 2 Ivan Necas 2013-09-11 08:57:46 EDT
I'm running into similar issue as well, seeing:

2013-09-11 07:53:00,750 nectar.downloaders.threaded:ERROR: Request timed out.
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/nectar/downloaders/threaded.py", line 225, in _fetch
    response = session.get(request.url, headers=headers)
  File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 310, in get
    return self.request('GET', url, **kwargs)
  File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 279, in request
    resp = self.send(prep, stream=stream, timeout=timeout, verify=verify, cert=cert, proxies=proxies)
  File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 374, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python2.6/site-packages/requests/adapters.py", line 217, in send
    raise Timeout('Request timed out.')
Timeout: Request timed out.


And

2013-09-11 07:54:17,499 pulp_rpm.plugins.importers.yum.parse.treeinfo:ERROR: some distro file downloads failed
Comment 3 Mike McCune 2013-09-11 11:09:04 EDT
Using:

pulp-nodes-common-2.2.0-1.el6.noarch
pulp-nodes-parent-2.2.0-1.el6.noarch
pulp-puppet-plugins-2.2.0-1.el6.noarch
pulp-rpm-plugins-2.2.0-1.el6.noarch
pulp-selinux-2.2.0-1.el6.noarch
pulp-server-2.2.0-1.el6.noarch
Comment 4 Jason Connor 2013-09-11 19:06:16 EDT
This bug is really just a failure to include the latest nectar (1.1.1) as a dependency for Pulp 2.2

Note You need to log in before you can comment on or make changes to this bug.