Bug 1048175 - [RFE]Fetch new sources remotely
Summary: [RFE]Fetch new sources remotely
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Fedora
Classification: Fedora
Component: fedpkg
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Dennis Gilmore
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 1002630 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-01-03 10:22 UTC by Christopher Meng
Modified: 2014-04-16 14:16 UTC (History)
4 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2014-02-13 16:33:34 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Christopher Meng 2014-01-03 10:22:07 UTC
Sometimes packagers have bad internet connection and fail to upload big sources to the remote Fedora server. For example if let me to maintain packages have 5MB sources, in most of cases I can't perform that.

But as most of sources can be verified and downloaded directly from URL written in SPEC, fedpkg should also support this function:

1. fedpkg remote-new-sources URL (Or even has option to specify the sourcename if we allow rename action)

2. fedpkg will send submitted URL to the server, server verifies and downloads the sources, then feedbacks the checksum

3. fedpkg writes relevant info to .gitignore and sources.

This will bring benefits of verifying sources consistency, and will save a lot of time for packagers.

I'm not sure if remote Fedora server has ability and permissions to download sources or not. If we allow this, I can help implement this feature.

Thanks.

Comment 1 Sam Kottler 2014-02-08 15:28:19 UTC
> I'm not sure if remote Fedora server has ability and permissions to download
> sources or not. If we allow this, I can help implement this feature.

AFAIK the lookaside cache when combined with fedpkg would need to be made intelligent enough to understand that it was being handed a remote URL; we can probably do that using protocol detection on the argument. It should do the following:

1. Look at the argument and figure out if the input is local or remote.
2. If it's local then just proceed with the path we use right now.
3. If it's remote then validate the URL tell the lookaside cache to fetch the source and add it to the sources.

The main issue is going to be adding support for the remote source inputs to the lookaside.

Thoughts?

Comment 2 Christopher Meng 2014-02-10 07:35:22 UTC
(In reply to Sam Kottler from comment #1)
> 1. Look at the argument and figure out if the input is local or remote.

Yes, check if SourceX tag contains http:// or ftp://. Otherwise this is a snapshot package or modified sources based package(legal/other issues).

> 2. If it's local then just proceed with the path we use right now.

Yes.

> 3. If it's remote then validate the URL tell the lookaside cache to fetch
> the source and add it to the sources.

Yes.

> The main issue is going to be adding support for the remote source inputs to
> the lookaside.

That's the place I'm not familiar, is there any infra doc of the lookaside mechanism?

Comment 3 Dennis Gilmore 2014-02-13 16:33:34 UTC
Having to upload from a local source is a design decision in how everything works. the packager is supposed to verify the contents of the tarballs, and needs to pass along the hash of the source tarball. There is security implications in just fetching from random locations on the internet.

This is not a use case releng or infrastructure is willing to entertain.

Comment 4 David Tardon 2014-04-16 14:16:13 UTC
*** Bug 1002630 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.