Bug 980181

Summary:	RFE - Publish listing files in exported repositories
Product:	[Retired] Pulp	Reporter:	Mike McCune <mmccune>
Component:	user-experience	Assignee:	Jason Connor <jconnor>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Preethi Thomas <pthomas>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	2.1.1	CC:	jortel, mhrivnak, mmccune, skarmark, tomckay
Target Milestone:	---	Keywords:	Triaged
Target Release:	2.3.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2013-12-09 14:31:24 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	950743

Description Mike McCune 2013-07-01 15:32:11 UTC

Red Hat's CDN uses special 'listing' files to indicate, at a product level, what repositories are available underneath, examples include:

RHEL6:

http://cdn.rcm-qa.redhat.com/content/dist/rhel/server/6/listing

RHEL6.4, arches:

http://cdn.rcm-qa.redhat.com/content/dist/rhel/server/6/6.4/listing

Requirements:

1) Published repository directories hosted by Pulp need equivalent listing files
2) Exported (ISO or otherwise) repositories need to preserve the listing files so when copied and synced into another Pulp server these listing files can be used

In order to make Pulp repositories look and act like Red Hat CDN hosted repositories for purposes of export we need to either:

 (A) Preserve the listing files hosted at Red Hat's CDN.  This may be difficult because when syncing a repository pulp only knows the URL of the repo itself, not the higher level directory structure
 (B) Publish new listing files that are just a reflection of the directories contained within the published repositories.  This seems more doable since Pulp has access to the file system and could possibly create listing files based on the published structure

Comment 1 Mike McCune 2013-07-01 15:33:05 UTC

related to:

https://bugzilla.redhat.com/show_bug.cgi?id=867602

but happy to fill out more details here if needed.

Comment 2 Mike McCune 2013-07-10 15:04:07 UTC

*** Bug 867602 has been marked as a duplicate of this bug. ***

Comment 3 Michael Hrivnak 2013-07-10 16:16:31 UTC

The export workflow will not be impacted by this RFE, as Pulp won't make any effort to preserve listing files from an upstream source. Instead, we'll follow solution B and generate listing files at publish time for every yum repo.

When a yum repository is published, pulp will re-generate the "listing" files in each parent directory up to /pulp/repos/  For example:

1) a repo gets published at /pulp/repos/a/b/c/
2) change working directory to the parent directory
3) create or overwrite a file named "listing" which contains the names of each directory in the current working directory, one per line
4) if the current working directory is not /pulp/repos/, GOTO 2

Please confirm that this will be sufficient.

Comment 4 Mike McCune 2013-07-10 19:55:38 UTC

ACK to the above

Comment 5 Jason Connor 2013-07-30 14:21:36 UTC

listing file generation added to repo publish 
PR: https://github.com/pulp/pulp_rpm/pull/288

Comment 6 Michael Hrivnak 2013-08-02 15:13:41 UTC

This ended up in PR https://github.com/pulp/pulp_rpm/pull/293

Comment 7 Jeff Ortel 2013-08-02 15:16:41 UTC

build: 2.2.0-0.23.beta.

Comment 8 Jeff Ortel 2013-08-05 15:17:27 UTC

Correction: not included in 2.2.

Comment 9 Jeff Ortel 2013-09-12 22:35:47 UTC

build: 2.3.0-0.10.alpha

Comment 10 Preethi Thomas 2013-10-09 14:42:03 UTC

verified
[root@pulp-v2-server ~]# rpm -qa |grep pulp-server
pulp-server-2.3.0-0.17.beta.el6.noarch

after publish

listing file exist with repo info

https://pulp-v2-server.usersys.redhat.com/pulp/repos/

Comment 11 Preethi Thomas 2013-12-09 14:31:24 UTC

Pulp 2.3 released.