Bug 1048175

Summary:	[RFE]Fetch new sources remotely
Product:	[Fedora] Fedora	Reporter:	Christopher Meng <i>
Component:	fedpkg	Assignee:	Dennis Gilmore <dennis>
Status:	CLOSED NOTABUG	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	rawhide	CC:	dennis, dgilmore, rjones, skottler
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2014-02-13 16:33:34 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Christopher Meng 2014-01-03 10:22:07 UTC

Sometimes packagers have bad internet connection and fail to upload big sources to the remote Fedora server. For example if let me to maintain packages have 5MB sources, in most of cases I can't perform that.

But as most of sources can be verified and downloaded directly from URL written in SPEC, fedpkg should also support this function:

1. fedpkg remote-new-sources URL (Or even has option to specify the sourcename if we allow rename action)

2. fedpkg will send submitted URL to the server, server verifies and downloads the sources, then feedbacks the checksum

3. fedpkg writes relevant info to .gitignore and sources.

This will bring benefits of verifying sources consistency, and will save a lot of time for packagers.

I'm not sure if remote Fedora server has ability and permissions to download sources or not. If we allow this, I can help implement this feature.

Thanks.

Comment 1 Sam Kottler 2014-02-08 15:28:19 UTC

> I'm not sure if remote Fedora server has ability and permissions to download
> sources or not. If we allow this, I can help implement this feature.

AFAIK the lookaside cache when combined with fedpkg would need to be made intelligent enough to understand that it was being handed a remote URL; we can probably do that using protocol detection on the argument. It should do the following:

1. Look at the argument and figure out if the input is local or remote.
2. If it's local then just proceed with the path we use right now.
3. If it's remote then validate the URL tell the lookaside cache to fetch the source and add it to the sources.

The main issue is going to be adding support for the remote source inputs to the lookaside.

Thoughts?

Comment 2 Christopher Meng 2014-02-10 07:35:22 UTC

(In reply to Sam Kottler from comment #1)
> 1. Look at the argument and figure out if the input is local or remote.

Yes, check if SourceX tag contains http:// or ftp://. Otherwise this is a snapshot package or modified sources based package(legal/other issues).

> 2. If it's local then just proceed with the path we use right now.

Yes.

> 3. If it's remote then validate the URL tell the lookaside cache to fetch
> the source and add it to the sources.

Yes.

> The main issue is going to be adding support for the remote source inputs to
> the lookaside.

That's the place I'm not familiar, is there any infra doc of the lookaside mechanism?

Comment 3 Dennis Gilmore 2014-02-13 16:33:34 UTC

Having to upload from a local source is a design decision in how everything works. the packager is supposed to verify the contents of the tarballs, and needs to pass along the hash of the source tarball. There is security implications in just fetching from random locations on the internet.

This is not a use case releng or infrastructure is willing to entertain.

Comment 4 David Tardon 2014-04-16 14:16:13 UTC

*** Bug 1002630 has been marked as a duplicate of this bug. ***