Bug 988919

Summary:	attempting to sync a repo with an unknown sqlite metadata file results in a KeyError on 'local_path'
Product:	[Retired] Pulp	Reporter:	Jeremy Cline <jcline>
Component:	rpm-support	Assignee:	Michael Hrivnak <mhrivnak>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Preethi Thomas <pthomas>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	Master	CC:	nobody
Target Milestone:	---	Keywords:	Triaged
Target Release:	2.2.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:
Clones:	994164 (view as bug list)		Environment:
Last Closed:	2013-09-10 15:45:46 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	994164
Bug Blocks:

Description Jeremy Cline 2013-07-26 17:34:48 UTC

Description of problem: Attempting to sync a repository with an unknown sqlite metadata file fails. An example of a repository that causes this can be found here: http://mirror.cc.vt.edu/pub/fedora/linux/updates/19/x86_64/


Version-Release number of selected component (if applicable): master


How reproducible: Always


Steps to Reproduce:
1. pulp-admin rpm repo create --repo-id=fedora-updates --feed=http://mirror.cc.vt.edu/pub/fedora/linux/updates/19/x86_64/
2. pulp-admin rpm repo sync run --repo-id=fedora-updates

Actual results:
+----------------------------------------------------------------------+
               Synchronizing Repository [fedora-updates]
+----------------------------------------------------------------------+        
                                                                                                                                                              
This command may be exited by pressing ctrl+c without affecting the actual                                                                                    
operation on the server.

Downloading metadata...
[\]
... failed

'local_path'


Expected results: A successful sync


Additional info: 
2013-07-26 11:26:27,183 pulp_rpm.plugins.importers.yum.sync:ERROR: sync failed
Traceback (most recent call last):
  File "/home/jcline/devel/pulp_rpm/plugins/pulp_rpm/plugins/importers/yum/sync.py", line 109, in run
    metadata_files = self.get_metadata()
  File "/home/jcline/devel/pulp_rpm/plugins/pulp_rpm/plugins/importers/yum/sync.py", line 204, in get_metadata
    self.import_unknown_metadata_files(metadata_files)
  File "/home/jcline/devel/pulp_rpm/plugins/pulp_rpm/plugins/importers/yum/sync.py", line 227, in import_unknown_metadata_files
    relative_path = os.path.join(model.relative_dir, os.path.basename(file_info['local_path']))
KeyError: 'local_path'
2013-07-26 11:26:27,202 pulp.server.dispatch.task:ERROR: Importer indicated a failed response
Traceback (most recent call last):
  File "/home/jcline/devel/pulp/platform/src/pulp/server/dispatch/task.py", line 138, in _run
    result = call(*args, **kwargs)
  File "/home/jcline/devel/pulp/platform/src/pulp/server/managers/repo/sync.py", line 117, in sync
    raise PulpExecutionException(_('Importer indicated a failed response'))
PulpExecutionException: Importer indicated a failed response
2013-07-26 11:26:27,203 pulp.server.dispatch.task:INFO: FAILURE: Task a7bb990e-a242-4c43-806b-46192aed17c6: CallRequest: RepoSyncManager.sync(u'fedora-updates', sync_config_override=None)

Comment 1 Michael Hrivnak 2013-07-26 21:30:21 UTC

https://github.com/pulp/pulp_rpm/pull/287

Comment 2 Michael Hrivnak 2013-08-02 14:24:42 UTC

build: 2.2.0-0.23.beta

Comment 3 Preethi Thomas 2013-08-07 17:25:48 UTC

verified
[root@cloud-qe-1-vm-1 ~]# rpm -qa pulp-server
pulp-server-2.2.0-0.24.beta.el6.noarch
[root@cloud-qe-1-vm-1 ~]# 



[root@cloud-qe-1-vm-1 ~]#  pulp-admin rpm repo update --repo-id=fedora-updates --feed=http://mirror.cc.vt.edu/pub/fedora/linux/updates/19/x86_64/ --max-downloads 2
Repository [fedora-updates] successfully updated

[root@cloud-qe-1-vm-1 ~]# pulp-admin rpm repo sync run  --repo-id=fedora-updates
+----------------------------------------------------------------------+
               Synchronizing Repository [fedora-updates]
+----------------------------------------------------------------------+

This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.

Downloading metadata...
[|]
... completed

Downloading repository content...
[==================================================] 100%
RPMs:       10693/10693 items
Delta RPMs: 9972/9972 items

... completed

Individual package errors encountered during sync:

Package:
http://mirror.cc.vt.edu/pub/fedora/linux/updates/19/x86_64/gfal2-plugin-gridftp-
2.3.0-0.fc19.x86_64.rpm
Error:   


Package:
http://mirror.cc.vt.edu/pub/fedora/linux/updates/19/x86_64/sugar-maze-24-1.fc19.
noarch.rpm
Error:   


Package:
http://mirror.cc.vt.edu/pub/fedora/linux/updates/19/x86_64/ghc-bytestring-devel-
0.9.2.1-11.1.fc19.x86_64.rpm
Error:   


Package:
http://mirror.cc.vt.edu/pub/fedora/linux/updates/19/x86_64/globus-gram-job-manag
er-callout-error-doc-2.1-7.fc19.noarch.rpm
Error:   


Package:
http://mirror.cc.vt.edu/pub/fedora/linux/updates/19/x86_64/texlive-parrun-doc-sv
n15878.0-0.1.fc19.noarch.rpm
Error:   



Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed

Importing errata...
[-]
... completed

Importing package groups/categories...
[|]
... completed

Publishing packages...
[==================================================] 100%
Packages: 20654/20654 items
... completed

Publishing distributions...
[==================================================] 100%
Distributions: 0/0 items
... completed

Generating metadata
[|]
... completed

Publishing repository over HTTPS
[-]
... completed
[root@cloud-qe-1-vm-1 ~]# pulp-admin rpm repo list
+----------------------------------------------------------------------+
                            RPM Repositories
+----------------------------------------------------------------------+

Id:                  magic
Display Name:        magic
Description:         None
Content Unit Counts: 
  Distribution:     1
  Package Category: 14
  Package Group:    210
  Rpm:              6381

Id:                  fedora-updates
Display Name:        fedora-updates
Description:         None
Content Unit Counts: 
  Drpm:                   9971
  Erratum:                1365
  Package Category:       12
  Package Group:          159
  Rpm:                    10683
  Yum Repo Metadata File: 2


[root@cloud-qe-1-vm-1 ~]#

Comment 5 Preethi Thomas 2013-09-10 15:45:46 UTC

2.2 released
http://repos.fedorapeople.org/repos/pulp/pulp/stable/2.2/