Bug 178790

Summary: Adding lftp command to get files from HTTP/FTP
Product: [Fedora] Fedora Documentation Reporter: Hugo Cisneiros <hugo>
Component: docs-requestsAssignee: Paul W. Frields <stickster>
Status: CLOSED CURRENTRELEASE QA Contact: Karsten Wade <kwade>
Severity: medium Docs Contact:
Priority: medium    
Version: develCC: kwade, stickster
Target Milestone: ---Keywords: FutureFeature
Target Release: ---   
Hardware: All   
OS: Linux   
URL: http://fedora.redhat.com/docs/mirror/sn-planning-and-setup.html
Whiteboard:
Fixed In Version: 1.0 Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2006-02-07 00:10:27 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Hugo Cisneiros 2006-01-24 13:50:22 UTC
I used the information in this fedora mirror tutorial (thanks!), but I
encountered some difficultes because I don't have rsync internet access here,
and wget is limited in some way to do real mirroring like rsync.

Then I found some new way to do the mirroring: using lftp command!

lftp is powerful enough to work as the rsync command. It retrieves, checks the
files within a local directory, updates it, deletes old files from the local
disk, etc. It does recursive mirroring, and the main feature I was needing: it
supports the downloads on HTTP/FTP throught authenticated proxy (squid). So it
is a great tool for getting data in a non-rsync server.

I think it is interesting to include this command in this tutorial. I used the
following command to make an updates mirror for example:

lftp -e "set mirror:exclude-regex $REGEXEXCLUDE && open $DIRREMOTO && mirror
--delete --verbose && quit"

- REGEXECLUDE is a regular expression that excludes files/directories from the
mirroring. Perfect for excluding some archs or fedora releases as the tutorial
explained. For example, I used the expression "debug\/|repodata\/" to exclude
both debug/ and repodata/ directories in the updates to create my own with
repodata command.

- DIRREMOTO is the remote directory. For example, I used:
http://mirror.linux.duke.edu/pub/fedora/linux/core/updates/4/i386

- It retrieves and checks in the current directory (pwd).

So it worked perfectly, and I would recommend it. If you need some help on more
usage, consult man lftp or I can help too :)

Thanks.

Comment 1 Paul W. Frields 2006-01-24 16:06:13 UTC
Excellent Hugo, thanks for the tip.  I will get this added into the
mirror-tutorial shortly.

Comment 2 eric 2009-07-07 04:09:29 UTC
Ticket moved to allow products to be removed from BZ.