Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1168602

Summary:	pulp-rpm-plugins RPM package is missing the pulp_distribution.xsd file, causing distribution sync to fail
Product:	[Retired] Pulp	Reporter:	Stuart Sears <stuart.sears>
Component:	rpm-support	Assignee:	Barnaby Court <bcourt>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Irina Gulina <igulina>
Severity:	high	Docs Contact:
Priority:	high
Version:	2.5	CC:	bmbouter, cduryee, igulina
Target Milestone:	---	Keywords:	Triaged
Target Release:	2.5.1
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2014-12-22 18:05:39 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Stuart Sears 2014-11-27 11:44:17 UTC

Description of problem:
Missing XSD file for distribution parsing


Version-Release number of selected component (if applicable):
pulp-rpm-plugins-2.5.0-1.el6.noarch

How reproducible:
any repository that provides a pulp_distribution.xml file will fail to synchronise without this file

Steps to Reproduce:
1. add a CDN repository for a kickstart tree, for example using
https://cdn.redhat.com/content/dist/rhel/server/6/6.3/x86_64/kickstart

2. attempt to synchronise
pulp-admin rpm repo sync run --repo-id ks-rhel-x86_64-server-6-6.3

3. Errors ensue:
+----------------------------------------------------------------------+
         Synchronizing Repository [ks-rhel-x86_64-server-6-6.3]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.


Downloading metadata...
[|]
... completed

Downloading repository content...
[==================================================] 100%
RPMs:       0/0 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items

Errors encountered during distribution sync:


Task Failed

Importer indicated a failed response


Actual results:
Error seen above and the following traceback in logs
Nov 27 10:03:54 dlonindcms12 pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): cdn.redhat.com
Nov 27 10:03:55 dlonindcms12 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: sync failed
Nov 27 10:03:55 dlonindcms12 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: Traceback (most recent call last):
Nov 27 10:03:55 dlonindcms12 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR:   File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 126, in run
Nov 27 10:03:55 dlonindcms12 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR:     self.nectar_config, self.distribution_report, self.set_progress)
Nov 27 10:03:55 dlonindcms12 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR:   File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/parse/treeinfo.py", line 67, in sync
Nov 27 10:03:55 dlonindcms12 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR:     dist_files = process_distribution(feed, tmp_dir, nectar_config, model, report)
Nov 27 10:03:55 dlonindcms12 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR:   File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/parse/treeinfo.py", line 205, in process_distribution
Nov 27 10:03:55 dlonindcms12 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR:     schema_doc = ET.parse(xsd)
Nov 27 10:03:55 dlonindcms12 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR:   File "lxml.etree.pyx", line 2692, in lxml.etree.parse (src/lxml/lxml.etree.c:49594)
Nov 27 10:03:55 dlonindcms12 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR:   File "parser.pxi", line 1500, in lxml.etree._parseDocument (src/lxml/lxml.etree.c:71364)
Nov 27 10:03:55 dlonindcms12 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR:   File "parser.pxi", line 1529, in lxml.etree._parseDocumentFromURL (src/lxml/lxml.etree.c:71647)
Nov 27 10:03:55 dlonindcms12 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR:   File "parser.pxi", line 1429, in lxml.etree._parseDocFromFile (src/lxml/lxml.etree.c:70742)
Nov 27 10:03:55 dlonindcms12 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR:   File "parser.pxi", line 975, in lxml.etree._BaseParser._parseDocFromFile (src/lxml/lxml.etree.c:67740)
Nov 27 10:03:55 dlonindcms12 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR:   File "parser.pxi", line 539, in lxml.etree._ParserContext._handleParseResultDoc (src/lxml/lxml.etree.c:63824)
Nov 27 10:03:55 dlonindcms12 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR:   File "parser.pxi", line 625, in lxml.etree._handleParseResult (src/lxml/lxml.etree.c:64745)
Nov 27 10:03:55 dlonindcms12 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR:   File "parser.pxi", line 563, in lxml.etree._raiseParseError (src/lxml/lxml.etree.c:64060)
Nov 27 10:03:55 dlonindcms12 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: IOError: Error reading file '/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/parse/pulp_distribution.xsd': failed to load external entity "/usr/lib
/python2.6/site-packages/pulp_rpm/plugins/importers/yum/parse/pulp_distribution.xsd"

Expected results:
distributions available from the remote repository should be synchronised

Additional info:
fetching the pulp_distribution.xsd file from github:
https://github.com/pulp/pulp_rpm/blob/master/plugins/pulp_rpm/plugins/importers/yum/parse/pulp_distribution.xsd

and placing it in /usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/parse/

allows the synchronisation to complete successfully

I think this is a packaging bug

Comment 1 Brian Bouterse 2014-12-01 17:25:03 UTC

The pulp_distribution.xsd should not be in the Python are of the filesystem. Potentially in /usr/share/ because it's a data file.

Check for other files to see if we are not treating those correctly also. Fix this file's location correctly, but if there are others that need fixing file a separate RFE.

Comment 2 Barnaby Court 2014-12-01 20:22:50 UTC

https://github.com/pulp/pulp_rpm/pull/604

Comment 3 Chris Duryee 2014-12-05 00:50:29 UTC

fixed in 2.5.1-0.1.beta

Comment 4 Irina Gulina 2014-12-10 16:55:33 UTC

Verified. 

>>rpm -qa | grep pulp-server
pulp-server-2.5.1-0.1.beta.el6.noarch

>> pulp-admin -u admin -p admin rpm repo create --repo-id kik-repo --feed='https://cdn.redhat.com/content/dist/rhel/server/6/6.3/x86_64/kickstart/' --feed-cert=x.cert --feed-key=y.key  --feed-ca-cert='z.pem'
Successfully created repository [kik-repo]

>> pulp-admin rpm repo sync run --repo-id kik-repo
+----------------------------------------------------------------------+
                  Synchronizing Repository [kik-repo]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.


Downloading metadata...
[|]
... completed

Downloading repository content...
[==================================================] 100%
RPMs:       3585/3585 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 65/65 items
... completed

Importing errata...
[-]
... completed

Importing package groups/categories...
[/]
... completed


Task Succeeded



Initializing repo metadata
[-]
... completed

Publishing Distribution files
[-]
... completed

Publishing RPMs
[==================================================] 100%
3585 of 3585 items
... completed

Publishing Delta RPMs
... skipped

Publishing Errata
[-]
... completed

Publishing Comps file
[==================================================] 100%
212 of 212 items
... completed

Publishing Metadata.
[-]
... completed

Closing repo metadata
[-]
... completed

Generating sqlite files
... skipped

Publishing files to web
[|]
... completed

Writing Listings File
[-]
... completed


Task Succeeded

>> pulp-admin rpm repo list
+----------------------------------------------------------------------+
                            RPM Repositories
+----------------------------------------------------------------------+

Id:                  kik-repo
Display Name:        kik-repo
Description:         None
Content Unit Counts: 
  Distribution:           1
  Package Category:       10
  Package Group:          202
  Rpm:                    3585
  Yum Repo Metadata File: 1

Comment 5 Randy Barlow 2014-12-22 18:05:39 UTC

Fixed in 2.5.1-1.