Bug 142816

Summary: RFE: jigdo package and distribution of jigdo templates of iso images
Product: [Fedora] Fedora Reporter: Charles R. Anderson <cra>
Component: distributionAssignee: Bill Nottingham <notting>
Status: CLOSED RAWHIDE QA Contact:
Severity: medium Docs Contact:
Priority: medium    
Version: rawhideCC: dgunchev, nigel, rvokal, yousifnet
Target Milestone: ---Keywords: FutureFeature, Reopened
Target Release: ---   
Hardware: All   
OS: Linux   
URL: http://bugzilla.fedora.us/show_bug.cgi?id=1252
Fixed In Version: Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2008-04-03 23:46:38 EDT Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
Bug Depends On:    
Bug Blocks: 235705    

Description Charles R. Anderson 2004-12-14 09:08:14 EST
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.3)

Description of problem:
Jigsaw Download, or short jigdo, is an intelligent tool that can be
used on the pieces of any chopped-up big file to create a special
"template" file which makes reassembly of the file very easy for users
who only have the pieces.

What makes jigdo special is that there are no restrictions on what
offsets/sizes the individual pieces have in the original big image.
This makes the program very well suited for distributing CD/DVD images
(or large zip/tar archives) because you can put the files on the CD on
an FTP server - when jigdo is presented the files along with the
template you generated, it is able to recreate the CD image.

There is no worry about the generated isos not being "official"
because jigdo uses MD5 to guarantee that the generated file is the
same as the original file used to create the template.

There would be many benefits to Red Hat servers, mirror servers, and
end users if template files would be provided of all the iso images.  

1. If desired, Red Hat could choose to only distribute release trees
and jigdo templates instead of full CD and DVD iso images.  This would
immediately save at least 2/3 the needed space for a release on the
Red Hat and mirror servers, with the potential to save even more if
hardlinking across architectures is used (no more need to have mostly
duplicate SRPMS iso images for i386 and x86_64).  Anyone could easily
recreate the iso images with jigdo, including mirrors if they wish to
provide them for download.

2. Even if iso images are still provided, mirrors could more quickly
and more efficiently sync up by using rsync against the tree.  During
a release sync, mirrors could link RC trees or the rawhide tree to the
release tree followed by an rsync from the master server, which could
save bandwidth and time by only transferring changed files.  Rsyncing
iso images do not gain this advantage, and would be unnessary since
jigdo could recreate the identical official iso images locally.

3. Different kinds of iso images could be provided, without ballooning
up the space requirements on the master server and mirrors, and
without overloading the already stressed internet pipes during mirror
syncing and after release.  For example, jigdo templates could be
provided for binary and source CD iso images, binary and source DVD
iso images, and perhaps even a combined source+binary DVD image for
those who wish to burn to larger, more expensive dual-layer DVD media.

4. Media conversion.  One only needs a set of packages (or release
tree) plus a jigdo template file in order to recreate a big image.  In
this way, someone who already has the CD's can make a DVD or vice
versa without redownloading the mostly redundant information just to
get a DVD image.

5. An intelligent mirror/download tool could be developed (there may
already be one) that could pick bits and pieces of the distro from
different mirror servers (each RPM file download for example).  In
this way, jigdo gains many of the advantages of a "swarm" protocol,
like BitTorrent, but it uses standard FTP/HTTP, which is not
firewalled in many networks like BitTorrent often is.

The URL points to the fedora.us work in progress package of jigdo.
Comment 1 Rahul Sundaram 2005-08-13 19:36:11 EDT
It would be a good idea for someone to package it in Fedora Extras repository.
If you are interested kindly look into 

Comment 2 Yousif Al Saif 2007-10-02 14:09:20 EDT
Redhat should seriously consider using jigdo for file distribution. jigdo was
proven successful with Debian. It's a win-win situation for the Fedora community
and the bug reported thoroughly explained the reasons. Jigdo is already in the
Extras repo. This bug should be reopened with Fedora 8 as a target release
Comment 3 Charles R. Anderson 2007-10-02 14:26:19 EDT
I think this is too late for Fedora 8, but I'll reopen and target for Fedora 9.
Comment 4 Jonathan Steffan 2008-01-16 13:27:59 EST

Also, we are working on pyjigdo, python based jigdo.
Comment 5 Ben Williams 2008-01-16 13:40:10 EST
Fedora Unity went to distributing our re-spins(origional media +updates) with
jigdo  as of F7.  We are very happy with Jigdo as a very low bandwith solution
for distributing the respins.

Comment 6 Charles R. Anderson 2008-04-03 23:46:38 EDT
Rel-Eng is doing this now, right?  Closing...