Bug 738834

Summary: Add support for multiple repositories.
Product: [Other] Topic Tool Reporter: Stephen Gordon <sgordon>
Component: cli-Topic_ToolAssignee: Stephen Gordon <sgordon>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 0.0.xCC: topic-tool-list
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: topic-tool-0.0.8-0 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-09-29 04:26:50 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Stephen Gordon 2011-09-15 23:33:14 UTC 
Description of problem:

This is mainly to support topic export from multiple sources, including skynet which now supports exporting of topics over http :D.

topic.conf syntax:

TOPIC_INDEX0=
TOPIC_REPO0=
TOPIC_INDEX1=
TOPIC_INDEX1=

For compatibility TOPIC_INDEX/TOPIC_REPO without a numeric suffix should be assumed to be TOPIC_INDEX0/TOPIC_REPO0.
Comment 1 Stephen Gordon 2011-09-28 23:54:44 UTC 
Also needed to change from using URLConnection to HttpURLConnection for export logic to ensure that we can follow redirects where required.
Comment 2 Stephen Gordon 2011-09-29 03:01:56 UTC 
Topic export now looks for:

1) TOPIC_INDEX / TOPIC_INDEX0 - At least one repository must be set. Where both keys are used TOPIC_INDEX is taken and TOPIC_INDEX0 ignores.
2) TOPIC_INDEXn keys increasing from TOPIC_INDEX1 (inclusive) onwards. Once it finds a value of n that is not set it stops looking. As a result, for example, with the configuration:

TOPIC_INDEX=http://repo0.com/
TOPIC_INDEX1=http://repo1.com/
TOPIC_INDEX2=http://repo2.com/
TOPIC_INDEX4=http://repo4.com/

The TOPIC_INDEX4 value is ignored.

3) Where a given URL has been used for more than one TOPIC_INDEXn value then an error is returned and no further exporting occurs.

There is still a potential for error when a topic is used which clashes, by name and path, with a topic from another repo. This needs to be looked at further though as any fix needs to also take into account the fact that including the same topic multiple times (from a single repo) is an existing valid use case. As a result basic checks such as displaying a warning or error if the given file exists already are not suitable. A marker needs to be kept somewhere during the export operation as to which repo each topic file was exported from so that such an error is only displayed where the repo is actually different.
Comment 3 Stephen Gordon 2011-09-29 03:22:41 UTC 
Moving to MODIFIED in spite of the issue at the end of the above comment, while it's important at this stage the multiple repository support is being used for export of topics from Skynet and the normal topic repo which have substantially differing naming schemes. Not an issue at this time.