Bug 585006

In

http://lists.fedoraproject.org/pipermail/test/2010-April/090318.html

Jesse Keating recommended filing a bug under either mkisofs (cdrkit) or implantisomd5 (isomd5sum).  The latter could also be the cause of this bug.

The sizes of the Fedora 11 netinst and live images are all correct.  The sizes of the Fedora 12 netinst and live images are all larger than the ISO header indicates, with the exception of Fedora-12-ppc-netinst.iso which is the correct size.

I found what appear to be Fedora 12 Alpha images at

http://193.190.67.15/packages/fedora/linux/releases/test/12-Alpha/

with valid signed checksum files (though I didn't fully download any of the ISOs to verify that their checksum matches).  The sizes of these are also larger than their ISO header indicates, with the exception of Fedora-12-Alpha-ppc-netinst.iso which is the correct size.

what option did you use to compose iso?

You'll have to ask whoever created these ISOs (Jesse Keating?).

The same problem exists with both live images from the 2010-04-13 Nouveau test day:

http://jlaska.fedorapeople.org/2010-04-13-nouveau-testday/gfx_test_week_20100413_i686.iso
http://adamwill.fedorapeople.org/gfx_test_week_20100412/gfx_test_week_20100412_x86-64.iso

so you could ask either of them also.

Thank you. I will ask.

There are 3 different tools involved.  Anaconda itself makes the boot.iso (which we symlink to netinst-iso).  Pungi makes the CD and DVD install isos, and livecd-tools makes the Live image.

/usr/bin/mkisofs -v -U -J -R -T -m repoview -m boot.iso -b isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table -V "Fedora 13-Beta i386 DVD" -o /srv/pungi/13-Beta.RC5/13-Beta/Fedora/i386/iso/Fedora-13-Beta-i386-DVD.iso /srv/pungi/13-Beta.RC5/13-Beta/Fedora/i386/os

is an example of the mkisofs call used by pungi.

I was able to reproduce this bug on both Fedora 12 x86_64 and Rawhide x86_64, using livecd-tools-031-1.fc12.1.x86_64 (same version in both OSes) and the following command, mentioned in the livecd-creator man page (using the proper path for the .ks file):

livecd-creator --config=/usr/share/doc/livecd-tools-031/livecd-fedora-minimal.ks

In both cases, the actual size of the output file was exactly the same (283115520 = 2048*138240 bytes), but the Volume size according to the ISO header was different (138203 for livecd-fedora-minimal-201004250806.iso in F12, and 138198 for livecd-fedora-minimal-201004250830.iso in Rawhide).

I didn't look at pungi, since all the CD and DVD install ISOs I've checked have the correct size.

Changing the component to livecd-tools.  The file /usr/bin/livecd-creator is a short Python script, but it imports other modules, so I think the maintainer of livecd-tools would probably be the right person to localize where the bug occurs (also I don't know Python).  Hopefully it will explain why anaconda is also making wrong-sized netinst images.  There's probably a common cause since the problem started for both around the same time (after 11 Final, and before/during 12 Alpha).

This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

This bug appears to have been reported against 'rawhide' during the Fedora 14 development cycle.
Changing version to '14'.

More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Still broken in x86_64 Rawhide.

livecd-tools-033-3.fc15.x86_64
genisoimage-1.1.10-2.fc14.x86_64

This is probably a genisoimage bug, but I still don't know how to reproduce it directly.  Here is the mkisofs call made by livecd-creator when it generated the faulty image (output from ps -A e | grep mkisofs leaving out all the environment variables).

/usr/bin/mkisofs -J -r -hide-rr-moved -hide-joliet-trans-tbl -V fedora-minim-x86_64-201008201243 -o /var/tmp/imgcreate-SoUdiU/out/livecd-fedora-minimal-201008201243.iso -b isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot -boot-info-table -boot-load-size 4 /var/tmp/imgcreate-SoUdiU/iso-bHUArI

(In reply to comment #8)
> There are 3 different tools involved.  Anaconda itself makes the boot.iso
> (which we symlink to netinst-iso).  Pungi makes the CD and DVD install isos,
> and livecd-tools makes the Live image.
> 
> /usr/bin/mkisofs -v -U -J -R -T -m repoview -m boot.iso -b
> isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot -boot-load-size 4
> -boot-info-table -V "Fedora 13-Beta i386 DVD" -o
> /srv/pungi/13-Beta.RC5/13-Beta/Fedora/i386/iso/Fedora-13-Beta-i386-DVD.iso
> /srv/pungi/13-Beta.RC5/13-Beta/Fedora/i386/os
> 
> is an example of the mkisofs call used by pungi.

A bit late (6 months!), but I just noticed that this means there are two different tools (anaconda for netinst and livecd-tools for live images) that are generating the wrong-sized images, and from what I've already learned, the problem started with the F12 Alpha images (F11 images were ok) and ppc was never affected. I know it's probably a genisoimage bug, but the problem has been figuring out exactly which options trigger it. What kind of mkisofs call does anaconda make? Is the call different for Intel vs. PPC? I know that's true for livecd-tools. Also, do you know which version of which platform were used in generating each of the F11 and F12 Alpha ISOs? I'd like to try to narrow down the version of genisoimage where this broke.

I'm going to reassign this to cdrkit (genisoimage).

isoinfo -d -i boot.iso
Logical block size is: 2048
Volume size is: 106154

should be 
Logical block size is: 2048
Volume size is: 106496

How to compose boot.iso?

It is a rather involved process using buildinstall from anaconda.

The relevant mkisofs call (for x86) is:

mkisofs -v -o $TOPDESTPATH/images/$BOOTISO $BIOSARGS $EFIARGS -R -J -V '$CDLABEL' -T -graft-points isolinux=$TOPDESTPATH/isolinux images=$TOPDESTPATH/images $EFIGRAFT

I'm going to try and get the values of those args from a log file, or a run with extra verbosity.

I wanted to check if the bug still existed with the latest version of genisoimage, but have been unable to create a Rawhide ISO using livecd-creator for a while now - there's a mkisofs error near the end. I checked

http://alt.fedoraproject.org/pub/alt/stage/14.security-respin/Fedora-14-i686-Live-Security/Fedora-14-i686-Live-Security.iso

which is dated November 2, and that still has the problem. (Note that it's enough to just download the first 50K or so with wget to see the exact size and to get the ISO header, then run "isoinfo -d -i Fedora-14-i686-Live-Security.iso" to see the ISO header size.)

This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

I'm trying to reproduce this bug, so I'm creating miscellaneous Fedora images, but all images I've created have correct ISO head size.

I've even tried to mount incorrect image (http://alt.fedoraproject.org/pub/alt/stage/14.security-respin/Fedora-14-i686-Live-Security/Fedora-14-i686-Live-Security.iso), create a new image from that one (the same data), but the new image has correct ISO head size.

Jesse, can you tell me what version of mkisofs/genisoimage/cdrkit was used to create images with incorrect ISO head size?

That's a good question Jan.  Andre, which current isos exhibit this problem?  With that info I should be able to figure out what versions of the tools were used to create them.

The problem exists with all live and netinst ISOs (except for PPC, which was never affected) from F12 Alpha up to the present. All F11 images were fine. (See comment 2 and comment 3).

For example, both live images from https://fedoraproject.org/wiki/Test_Day:2011-02-03_GNOME3_Alpha have the problem - easily checked by starting the download with wget, Ctrl-C'ing after downloading the first 50K or so, then comparing the actual ISO size (shown by wget) with the ISO header size (shown by "isoinfo -d -i file.iso").

Ok, then those are just made with the tools that are currently in rawhide.

(In reply to comment #19)
> I'm trying to reproduce this bug, so I'm creating miscellaneous Fedora images,
> but all images I've created have correct ISO head size.

Greetings Jan!  Any luck reproducing this failure?  I believe Andre has encountered this issue once again on the F-15-Alpha-TC1 media (currently available at http://serverbeach1.fedoraproject.org/pub/alt/stage/).

(In reply to comment #24)

> Greetings Jan!  Any luck reproducing this failure?  I believe Andre has
> encountered this issue once again on the F-15-Alpha-TC1 media (currently
> available at http://serverbeach1.fedoraproject.org/pub/alt/stage/).

Just the Live and netinst images, as usual. (See comment 21.) The install DVD and the former install CDs are always fine.

The fact that PPC was never affected (when that was a primary arch) may be important. I don't know Python, but noticed that in the python-imgcreate package, in the file live.py, there are separate classes x86LiveImageCreator and ppcLiveImageCreator which use different options for mkisofs.

Finally, I've reproduced the failure today. I've tracked live.py from python-imgcreate package (which is a subpackage of livecd-tools) and found this important call:

  subprocess.call(["/usr/bin/isohybrid", iso])

just after the mkisofs call.

Then I tested file size and iso's header before and after this call and I found out that the iso file was correct just after creating by mkisofs:

  -rw-r--r--. 1 root root 251363328 Feb 15 /var/tmp/...
  Volume size is: 122736

But the iso file was adjusted right after that by "isohybrid" utility (which is a part of syslinux package). After that adjusting the iso file had a new size, but the same iso's header:

  -rw-r--r--. 1 root root 251658240 Feb 15 15:56 /var/tmp/
  Volume size is: 122736

It seems like isohybrid doesn't care about iso's header at all, but I suppose it should. So this failure is not caused by mkisofs/genisoimage and I'm switching the component to syslinux therefore.

---
Just a note about isohybrid's purpose: Post process an ISO 9660 image generated with mkisofs or genisoimage to allow - hybrid booting - as a CD-ROM or as a hard disk.

Spoke with isolinux upstream:

hpa: The size is indeed adjusted; it is padded to a megabyte boundary
hpa: That is intentional and is not a bug
hpa: So I don't really see the problem
Oxf13: I'm not really sure, the reporter seems to think it's a problem that the header doesn't match the size
hpa: Yes, but there isn't anything in the bug report that indicates that it's an actual problem
hpa: xorriso might do it better, for all I know
hpa: I think this is NOTABUG though

So, what is the actual problem here?

Well, besides just the fact that it's wrong to invalidate the correct header information written by another program such as genisoimage (even if no use for it is known!), being able to properly read an ISO off an optical disc automatically depends on the header size being correct. For example see

http://www.troubleshooters.com/linux/coasterless.htm#_Accurately_Reading_a_CD

Without the correct size information, there's no way to know what it is, so the size has to be stored separately (for example by writing it on each disc) and a manual dd command used to read it off. The rawread script in the link assumes that the header info is correct ("you'll be assured of reading the correct number of blocks on the CD").

Should add the qualifier that it would be okay to invalidate the header info if there is no good reason for that info being there at all (no good use for it, and no standard requiring it). In that case, though, the presence of that info should also be reported as a bug against whatever program put it there (genisoimage in this case) so its maintainer knows that it's being invalidated and can remove it. On the other hand, if there IS a reason for the info being there, it should be correct, as wrong info is worse than no info (since people might believe it).

OpenSUSE 11.4 DVDs are being affected by this issue as well (earlier ones were not). Is this eventually expected to happen to Fedora install DVDs?

Since checking for this issue only requires downloading the first 50K or so of an ISO (enough to get the ISO header), it would be easy to check various other distros to see exactly which ones are affected.

(In reply to comment #27)
...
> hpa: I think this is NOTABUG though
>
> So, what is the actual problem here?

The bug here is that isohybrid is changing a valid ISO 9660 image into an image that is not compliant with the ISO 9660 standard.

8.4.8 Volume Space Size (BP 81 to 88)
      This field shall specify as a 32-bit number 
      the number of Logical Blocks 
      in which the Volume Space of the volume is recorded.

      This field shall be recorded according to 7.3.3.

ISO 9660 is also called:

Standard ECMA-119
Volume and File Structure of CDROM for Information Interchange
2nd edition (December 1987)
http://www.ecma-international.org/publications/standards/Ecma-119.htm
http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-119.pdf

The standard say "all" here, not "some":

6.1.3 Volume Space
      The information on a volume shall be recorded in the set of 
      all Logical Sectors on the volume. 
      This set shall be referred to as the Volume Space of the volume.

      The bytes in the Volume Space shall be numbered consecutively. 
      The numbering shall start with 1, 
      which shall be assigned to the first byte of 
      the first Logical Sector of the Volume Space. 
      The numbering shall continue through successive bytes of 
      the first Logical Sector, and then 
      through successive bytes of each successive Logical Sector, 
      of the Volume Space.

(In reply to comment #27)
...
> hpa: The size is indeed adjusted; it is padded to a megabyte boundary

genisoimage pads by default:
-pad   Pad the end of the whole image by 150 sectors (300 kB).
       This option is enabled by default.

You can disable padding with:
-no-pad Do not pad the end by 150 sectors (300 kB) and 
        do not make the the boot partitions start on a multiple of 16 sectors.

And genisoimage computes the correct size in *both* cases.

Tested with:
$ genisoimage -o x1.iso tmp
$ genisoimage -o x2.iso -no-pad tmp

For the record:
$ readlink -ev /usr/bin/mkisofs
/usr/bin/genisoimage

New suggested bug title:

"isohybrid fails to create image in conformance with ISO 9660 standard: incorrect Volume Space Size"

2.1 Conformance of a CD-ROM
    A CD-ROM sconforms to this Standard when 
    all information recorded on it 
    conforms to the requirements of Section II of this Standard.
    A statement of conformance shall identify 
    the lowest level of interchange to which 
    the contents of the CD-ROM conform.

(In reply to comment #2)
> The sizes of the Fedora 11 netinst and live images are all correct.  The sizes
> of the Fedora 12 netinst and live images are all larger than the ISO header
> indicates, with the exception of Fedora-12-ppc-netinst.iso which is the correct
> size.

This could be explained by the fact that the padding can be zero:
padding = (frac > 0) ? cylsize - frac : 0;
http://git.kernel.org/?p=boot/syslinux/syslinux.git;a=blob;f=utils/isohybrid.c

Comment 27:
> hpa: The size is indeed adjusted; it is padded to a megabyte boundary

So a hypothesis would be that some images were a multiple of 1024**2 bytes before isohybrid was run.

(In reply to comment #36)
...
> Comment 27:
> > hpa: The size is indeed adjusted; it is padded to a megabyte boundary
> 
> So a hypothesis would be that some images were a multiple of 1024**2 bytes
> before isohybrid was run.

And that could be tested by reporting which images have a Volume Space Size that is a multiple of 1024**2 bytes and which do not.

The Fedora-11-x86_64-Live CD is not a multiple of 1024**2 bytes.
So was it not padded?

[stephent@walnut F11]$ du -b Fedora-11-x86_64-Live-from-cdrom.iso
724097024	Fedora-11-x86_64-Live-from-cdrom.iso
[stephent@walnut F11]$ isoinfo -d -i Fedora-11-x86_64-Live-from-cdrom.iso | egrep 'Logical block size|Volume size'
Logical block size is: 2048
Volume size is: 353563
[stephent@walnut F11]$ python
Python 2.7 (r27:82500, Sep 16 2010, 18:02:00) 
[GCC 4.5.1 20100907 (Red Hat 4.5.1-3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> 353563*2048
724097024
>>> 724097024.0/1024**2
690.552734375

(In reply to comment #38)
> The Fedora-11-x86_64-Live CD is not a multiple of 1024**2 bytes.
> So was it not padded?

It is not hybrid:
[stephent@walnut F11]$ hexdump -C Fedora-11-x86_64-Live-from-cdrom.iso | head -3
00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00008000  01 43 44 30 30 31 01 00  4c 49 4e 55 58 20 20 20  |.CD001..LINUX   |

Hybrid live images were released starting with F12.
http://fedoraproject.org/wiki/Fedora_12_Announcement

This explains why the F11 images have the correct sizes (Comment 2) --
they never had isohybrid run on them and hence were not padded.

The sizes of all F12 Alpha, Beta, and Final PPC install images, including netinst, match the ISO header size. The sizes of the PPC netinst images are

276062208 Fedora-12-Alpha-ppc-netinst.iso
279496704 Fedora-12-Beta-ppc-netinst.iso
278157312 Fedora-12-ppc-netinst.iso

none of which are a multiple of 1024**2.

(In reply to comment #41)
> The sizes of all F12 Alpha, Beta, and Final PPC install images, including
> netinst, match the ISO header size. The sizes of the PPC netinst images are
> 
> 276062208 Fedora-12-Alpha-ppc-netinst.iso
> 279496704 Fedora-12-Beta-ppc-netinst.iso
> 278157312 Fedora-12-ppc-netinst.iso
> 
> none of which are a multiple of 1024**2.

Thanks for pointing that out. I pulled down Fedora-12-ppc-netinst.iso[1] and confirm. Further, the images do have a boot loader, but it is not isolinux:
$ hexdump -C Fedora-12-ppc-netinst.iso | less

There is a string "yaboot.conf" in it, which suggests a different tool was used.
Possibly yaboot: http://yaboot.ozlabs.org/

[1] $ wget http://archives.fedoraproject.org/pub/archive/fedora/linux/releases/12/Fedora/ppc/iso/Fedora-12-ppc-netinst.iso

(In reply to comment #28)
...
> being able to properly read an ISO off an optical disc
> automatically depends on the header size being correct.
...

This could explain why checkisomd5 passes on the padded images. The command reads the "wrong" size and gets the "right" md5sum, because it never reads the padded sectors.

/* finds primary volume descriptor and returns info from it */
/* mediasum must be a preallocated buffer at least 33 bytes long */
/* fragmentsums must be a preallocated buffer at least FRAGMENT_SUM_LENGTH+1 bytes long */
static int parsepvd(int isofd, char *mediasum, int *skipsectors, long long *isosize, int *supported, char *fragmentsums, long long *fragmentcount) {
...

http://git.fedorahosted.org/git/?p=isomd5sum.git;a=blob_plain;f=libcheckisomd5.c

(In reply to comment #43)
> ... because it never reads the padded sectors. ...

Even zeroed sectors can affect the md5sum:
$ dd if=/dev/zero count=1 > zero-1.dat
$ dd if=/dev/zero count=2 > zero-2.dat
$ md5sum zero-*.dat
bf619eac0cdf3f68d496ea9344137e8b  zero-1.dat
0f343b0931126a20f133d67c2b018a3b  zero-2.dat

(In reply to comment #43)
...
> This could explain why checkisomd5 passes on the padded images.
...

I'm wrong. anaconda runs:
isohybrid $BOOTISO
implantisomd5 $BOOTISO
http://git.fedorahosted.org/git/?p=anaconda.git;a=blob;f=scripts/upd-bootiso

The netinst images for 15 Final RC1 both have the correct ISO header size. Is this bug fixed, or is isohybrid not being used anymore?

Also should note that this is affecting other distros. Besides the openSUSE DVD mentioned earlier, it also affects Mandriva. It does not seem to affect Debian or Ubuntu (at least not yet). These are the only other distros I've checked.

(In reply to comment #46)
> The netinst images for 15 Final RC1 both have the correct ISO header size. Is
> this bug fixed, or is isohybrid not being used anymore?
...

I'm still downloading, but you can try:
$ hexdump -Cv Fedora-15-x86_64-netinst.iso | less

If you see something like this, it is probably an isolinux hybrid image:
00000090  00 e8 83 00 69 73 6f 6c  69 6e 75 78 2e 62 69 6e  |....isolinux.bin|

The DVD should have zeros there.

Can your report the image sizes?
(So we can see if they are a multiple of 1024**2 bytes.)

See comment 17 for example. You only need to download the first 50K or so of the ISO to test it. The sizes are

205494272 Fedora-15-i386-netinst.iso
208332800 Fedora-15-x86_64-netinst.iso

neither of which is a multiple of 1024**2.

(In reply to comment #48)
> See comment 17 for example. You only need to download the first 50K or so of
> the ISO to test it. The sizes are
> 
> 205494272 Fedora-15-i386-netinst.iso
> 208332800 Fedora-15-x86_64-netinst.iso
> 
> neither of which is a multiple of 1024**2.

Thanks the report.

Further, there is no isolinux boot loader:
$ hexdump -C Fedora-15-x86_64-netinst.iso | head -3
00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00008000  01 43 44 30 30 31 01 00  4c 49 4e 55 58 20 20 20  |.CD001..LINUX   |

If isohybrid was not installed in the build environment, it would not have been run, but there would have been an error message.

       eval $mkisocmd
        if [ -x /usr/bin/isohybrid ]; then
            isohybrid $BOOTISO || echo "Unable to make hybrid boot.iso"
        fi
        implantisomd5 $BOOTISO
http://git.fedorahosted.org/git/?p=anaconda.git;a=blob;f=scripts/upd-bootiso

(In reply to comment #50)
> If isohybrid was not installed in the build environment, it would not have been
> run, but there would have been an error message.

Erk! "... and there would NOT have been an error message."

(In reply to comment #50)
> If isohybrid was not installed in the build environment, it would not have been
> run, but there would have been an error message.
> 
>        eval $mkisocmd
>         if [ -x /usr/bin/isohybrid ]; then
>             isohybrid $BOOTISO || echo "Unable to make hybrid boot.iso"
>         fi
>         implantisomd5 $BOOTISO
> http://git.fedorahosted.org/git/?p=anaconda.git;a=blob;f=scripts/upd-bootiso

Fedora 15 no longer uses those anaconda scripts.  Image building is handled by the lorax project.

That said in a quick peek at how lorax does isohybrid, I think I've spotted a logic error leading to no isohybrid.

Netinst images for 15 Final RC2 and RC3 continue to have the correct Volume Space Size. RC3 Live images have the wrong Volume Space Size, as usual.

(In reply to comment #53)
> Netinst images for 15 Final RC2 and RC3 continue to have the correct Volume
> Space Size. RC3 Live images have the wrong Volume Space Size, as usual.

Thanks for your report. The RC2 and RC3 netinst images do not have an isolinux boot loader, so I presume they never had isohybrid run on them:

$ hexdump -C Fedora-15-x86_64-netinst.iso | head -3
00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00008000  01 43 44 30 30 31 01 00  4c 49 4e 55 58 20 20 20  |.CD001..LINUX   |

The RC3 Live image has an isolinux boot loader:

$ hexdump -C Fedora-15-x86_64-Live-Desktop.iso | head | grep isolinux
00000090  00 e8 83 00 69 73 6f 6c  69 6e 75 78 2e 62 69 6e  |....isolinux.bin|

Three images were just added in the RC3 Live directory. Fedora-15-Multi-Boot.iso is 5000081408 bytes and has the correct Volume Space Size. It fails the checkisomd5 test - I don't know if this is expected.

[robatino@secondary01 Live]$ checkisomd5 Fedora-15-Multi-Boot.iso
Press [Esc] to abort check.

The media check is complete, the result is: NA.

No checksum information available, unable to verify media.
[robatino@secondary01 Live]$

Fedora-15-i686-Live-Games.iso is 4203534336 bytes and also has the correct Volume Space Size. Fedora-15-x86_64-Live-Games.iso is 4208984064 bytes and has the wrong Volume Space Size. None of these 3 sizes are multiples of 1024**2.

(In reply to comment #55)
> Three images were just added in the RC3 Live directory.
...

Thanks for your report. I was getting confused, so here is a table.
Columns are:
1: Image name.
2: Image size as reported by wget.
3: Volume space size computed from "isoinfo -d".
4: Image size divided by 1024**2.
5: Whether isolinux is in the first sector of the image.
6: [Not reported: Result of checkisomd5 test.]

F15-RC3:
img_name                        img_sz      vol_sp_sz  img_sz/1024**2 isolinux
Fedora-15-i686-Live-Games.iso   4203534336  4203534336 4008.802734375 yes
Fedora-15-x86_64-Live-Games.iso 4208984064  4208353280 4014.0         yes
Fedora-15-Multi-Boot.iso        5000081408  5000081408 4768.44921875  no

I did this manually, so there could be errors.

(In reply to comment #55)
...
> None of these 3 sizes are multiples of 1024**2.

For Fedora-15-x86_64-Live-Games.iso, I get a multiple of 1024**2:
>>> 4208984064.0/1024**2
4014.0

You're correct, sorry about that. I neglected to check it since the Volume Space Size was wrong anyway.

There is now a separate http://serverbeach1.fedoraproject.org/pub/alt/stage/15.RC3/Multi/ directory containing two ISOs, Fedora-15-Multi-Desktop.iso and Fedora-15-Multi-Install.iso. Both of these have the correct Volume Space Size. They do not contain a built-in mediacheck (and are not expected to in the future). Ideally, the rawread script mentioned in comment 28 could be used to always verify these, but it makes the now unreliable assumption that the ISO actually has the correct Volume Space Size, and I don't know if that's expected to always be true for Multi images (or even for install DVDs). So the simplest way to document verifying these is probably to just post an explicit dd command for each image, with the "count" for each one set to the ISO size in bytes, divided by 2048. For example,

dd if=/dev/dvd bs=2048 count=2441446 conv=notrunc,noerror | sha256sum

for Fedora-15-Multi-Desktop.iso, and

dd if=/dev/dvd bs=2048 count=3639167 conv=notrunc,noerror | sha256sum

for Fedora-15-Multi-Install.iso. If this bug gets fixed, then people could just be told to run

rawread /dev/dvd | sha256sum

for each image (which BTW is my preferred method for verifying ISOs, since it should work on any ISO).

I must be missing something ... wouldn't this work?
$ sha256sum *.iso

(In reply to comment #60)
> I must be missing something ... wouldn't this work?
> $ sha256sum *.iso

Only for the ISO file, not for the burned media (the above is a substitute for the missing mediacheck).

(In reply to comment #61)
> (In reply to comment #60)
> > I must be missing something ... wouldn't this work?
> > $ sha256sum *.iso
> 
> Only for the ISO file, not for the burned media (the above is a substitute for
> the missing mediacheck).

OK, thanks. This seems to work:

$ sha256sum /dev/sr0
abe225d6279b1f6b1b26c9655518d4b0312f1da24e1f674e9209d1a172f1fe54  /dev/sr0

$ grep Fedora-15-Beta-i386-netinst.iso Fedora-15-Beta-i386-CHECKSUM
abe225d6279b1f6b1b26c9655518d4b0312f1da24e1f674e9209d1a172f1fe54 *Fedora-15-Beta-i386-netinst.iso

In general, just asking the drive to read off the image without telling it exactly how far to read won't work. You're fortunate if it works on your hardware. Unless the drive is able to correctly determine exactly how many bytes to read, the checksum won't match. Typically, it will read off the original ISO plus some padding. In general it needs to be done as I described in comment 59.

For the record, I always burn with "-dao":
wodim $DUMMYBURN -v -dao $BLANK dev=$BURNER $ISONAME

Created attachment 500089 [details]
iso-image-analyzer.pl

(In reply to comment #56)
...
> I did this manually, so there could be errors.

Here is a script to analyze ISO images and produce a report suitable for importing into gnumeric. In particular, it reports the image size computed from the ISO image's volume space size and the size of the image file itself. The script can be run before an image is fully downloaded[1], although the image file size will be incorrect.

Usage:
$ ./iso-image-analyzer.pl *.iso > x-1.csv

Here are all the fields in the report.
my @report_fields = (
    'rec_num',      # record number
    'file_name',    # file name as given on command line
    'vol_id',       # volume id

    'lbs',          # logical block size in bytes
    'vol_sz_bks',   # volume size in logical blocks
    'vol_sz_b',     # volume size in bytes (lbs * vol_sz_bks)

    'img_sz_b',     # image size in bytes
    'img_sz_mib',   # image size in MiB (img_sz_b/1024**2)
    'img_mtime',    # image modification time (mtime)
    'img_mtime_h',  # image modification time (mtime), human readable, GMT

    'md5sum',       # whether the image has an md5sum (0 or 1)
    'isolinux',     # whether isolinux was found in boot sector (0 or 1)
);

[1] Thanks the tip, Andre: Comment 48.
You can also get the image size with:
$ curl -Is <URL-of-ISO-image> | grep 'Content-Length:'

FWIW, the script I wrote to generate the Multi-* images doesn't currently invoke isohybrid. It might do so in the future.

FYI, the CentOS 6.0 netinstall images now have the wrong Volume Space Size as well. The 5.6 netinstalls did not, nor did the one 6.0 DVD image that I checked.

There is no ISO 9660 standard violation here. These are hybrid images, i.e. media images which contain several file systems sharing the same space, first of it happened to be an ISO 9660 file system image. There is nothing wrong with Volume Space Size in that particular ISO 9660 image, it's just an unfortunate and icorrect assumption about the structure of the whole concatenated media image that leads to troubles. Just don't assume that there should be only one file system on the media and it should resolve all issues. The only real problem I can see here is that that image format is poorly documented.

(In reply to comment #68)
> There is no ISO 9660 standard violation here. These are hybrid images, i.e.
> media images which contain several file systems sharing the same space, first
> of it happened to be an ISO 9660 file system image. There is nothing wrong with
> Volume Space Size in that particular ISO 9660 image, it's just an unfortunate
> and icorrect assumption about the structure of the whole concatenated media
> image that leads to troubles. Just don't assume that there should be only one
> file system on the media and it should resolve all issues. The only real
> problem I can see here is that that image format is poorly documented.

isohybrid is poorly documented, but from a test, it simply writes 512 bytes at the beginning of the ISO image and zero-pads at the end to 1MB. Also note that genisoimage pads and computes the correct size: Comment 33.

Source is here:
http://git.kernel.org/?p=boot/syslinux/syslinux.git;a=tree;f=utils;hb=HEAD

Some documentation is here:
http://syslinux.zytor.com/wiki/index.php/Doc/isolinux

Test procedure:
$ cp -ip Fedora-15-i386-netinst.iso x1.iso
$ cp -ip x1.iso x2.iso
$ isohybrid x2.iso
$ hexdump -Cv x1.iso > x1.hexdump &
$ hexdump -Cv x2.iso > x2.hexdump &
$ diff -u x1.hexdump x2.hexdump | less

(In reply to comment #68)
> There is no ISO 9660 standard violation here. These are hybrid images, i.e.
> media images which contain several file systems sharing the same space, first
> of it happened to be an ISO 9660 file system image. There is nothing wrong with
> Volume Space Size in that particular ISO 9660 image, it's just an unfortunate
> and icorrect assumption about the structure of the whole concatenated media
> image that leads to troubles. Just don't assume that there should be only one
> file system on the media and it should resolve all issues. The only real
> problem I can see here is that that image format is poorly documented.

If I understand you correctly, you're saying that these images are not really ISO files and therefore shouldn't be expected to comply with that standard. Even so, there's still a problem in that they are masquerading as ISO files - they have a header which is apparently unchanged from the original ISO header, and are even named with a .iso file suffix, so applications can't tell the difference, and will assume that the header information is correct, as it would be in an actual ISO 9660-compliant image. If in fact it is not, then the images need to stop masquerading as ISO files, so this doesn't happen. I suspect it would be far simpler to just modify isohybrid so it writes the correct Volume Space Size when it pads (and the existing genisoimage code for this can be used as a template, so it shouldn't be hard).

(In reply to comment #69)
> isohybrid is poorly documented, but from a test, it simply writes 512 bytes at
> the beginning of the ISO image and zero-pads at the end to 1MB. Also note that
> genisoimage pads and computes the correct size: Comment 33.

It may be seem so, but it's not so simple. According to isolinux documentation:

"The ISO 9660 filesystem is encapsulated in a partition (which starts at offset zero, which may confuse some systems.) This makes it possible for the operating system, once booted, to use the remainder of the device for persistent storage by creating a second partition."

(In reply to comment #70)
> If I understand you correctly, you're saying that these images are not really
> ISO files and therefore shouldn't be expected to comply with that standard.
> Even so, there's still a problem in that they are masquerading as ISO files -
> they have a header which is apparently unchanged from the original ISO header,
> and are even named with a .iso file suffix, so applications can't tell the
> difference, and will assume that the header information is correct, as it would
> be in an actual ISO 9660-compliant image. If in fact it is not, then the images
> need to stop masquerading as ISO files, so this doesn't happen.

May be you are right, but this is common practice. It's done this way to simplify deployment between different media types, e.g. CD, DVD, USB flash drive. Also, many commercial CD/DVD writing applications don't care and add padding with no modification to ISO 9660 filesystem itself.

>I suspect it
> would be far simpler to just modify isohybrid so it writes the correct Volume
> Space Size when it pads (and the existing genisoimage code for this can be used
> as a template, so it shouldn't be hard).

It would defeat the purpose. The padding itself does not belong to ISO 9660 filesystem, but is a part of hard disk partitioning scheme.

The situation I'm mainly concerned with is verifying burned Linux ISO (or hybrid or whatever) files. If there is a built-in mediacheck like the one most Fedora ISOs have, no problem. But sometimes the mediacheck doesn't work (for example bug 692135) and sometimes there is no mediacheck ( for example https://fedoraproject.org/wiki/Test_Results:Fedora_15_Final_Multi_Image_DVD ). In this case, one needs to be able to read off the exact size from the burned disc to match the checksum, and if the Volume Space Size doesn't correspond, one needs to know the size independently. The Fedora checksum files currently don't contain sizes (though they could if necessary) so if you have a disc that's been sitting around a while, and can't find the size (my current workaround for affected images is to write the size on the disc), it's almost impossible to verify. I say "almost" because you have a lower bound on the actual size, and it's a multiple of 2048, so theoretically you could run through all the possibilities to try to get a match.

Another possibility would be if the Fedora checksum files contained hashes for relevant data only, i.e. for ISO 9660 filesystem itself, and not for the whole file. This would still require that it should be documented in the installation guide though.

(In reply to comment #74)
> Another possibility would be if the Fedora checksum files contained hashes for
> relevant data only, i.e. for ISO 9660 filesystem itself, and not for the whole
> file. This would still require that it should be documented in the installation
> guide though.

1. The image verification command checks whole files:
$ sha256sum -c Fedora-15-x86_64-CHECKSUM

2. Malicious code inserted in the image boot sector and in the padded area could not be detected:
Fedora Project - Verify your download
https://fedoraproject.org/verify

(In reply to comment #75)
> 1. The image verification command checks whole files:
> $ sha256sum -c Fedora-15-x86_64-CHECKSUM

That's just coreutils. There are other tools out there. Or there could be a simple script.

> 2. Malicious code inserted in the image boot sector and in the padded area
> could not be detected:
> Fedora Project - Verify your download
> https://fedoraproject.org/verify

Boot sector is part of the ISO 9660 filesystem and is protected. Padded area is actually irrelevant and is not bootable anyway.

(In reply to comment #76)
...
> Boot sector is part of the ISO 9660 filesystem and is protected.
...

OK. The first 512 bytes are part of the System Area:

6.2.1 System Area and Data Area

The Volume Space shall be divided into a System Area and a Data Area.

The System Area shall occupy the Logical Sectors with Logical Sector Numbers 0 to 15. The System Area shall be reserved for system use. Its content is not specified by this Standard.

The Data Area shall occupy the remaining Logical Sectors of the Volume Space.

http://www.ecma-international.org/publications/standards/Ecma-119.htm

Both the downloaded file and the burned disc need to be checked - the downloaded file to verify that it's not evil, and the burned disc for integrity at least. It looks to me like it would be simpler to just use one checksum for both (in a signed checksum file) and if necessary put the size of each ISO in a comment next to the corresponding checksum. I'm still not convinced that the Volume Space Size shouldn't be made to correspond to the full file size. Since these are not genuine ISO files, doesn't that mean that the header information is allowed to be set to anything (including what it would be if the entire file was in fact an ISO file)?

(In reply to comment #77)
> OK. The first 512 bytes are part of the System Area:
[snipped]
> The Data Area shall occupy the remaining Logical Sectors of the Volume Space.

Yes, so? I'm not sure I'm following you here. The padding is past the Data Area if you mean that.

(In reply to comment #78)
> Both the downloaded file and the burned disc need to be checked - the
> downloaded file to verify that it's not evil, and the burned disc for integrity
> at least. It looks to me like it would be simpler to just use one checksum for
> both (in a signed checksum file)

Actually, it doesn't contradict to what I said earlier. You can do all that using just one checksum of ISO 9660 file system. Anything that is not there won't make any difference.

> and if necessary put the size of each ISO in a
> comment next to the corresponding checksum.

You could do that. Or you could look at the size of original file, would it be locally or on the web. I see little difference.

> I'm still not convinced that the
> Volume Space Size shouldn't be made to correspond to the full file size. Since
> these are not genuine ISO files, doesn't that mean that the header information
> is allowed to be set to anything (including what it would be if the entire file
> was in fact an ISO file)?

Well, yes, of course. It depends on the one' point of view. You could count the extra data as part of the file system or you could not. Look at self extracting archives for example. Usually they don't count the payload as part of the executable. And if they would, that could have subtle consequences, just like yours in our case. In other words: you can't have your cake and eat it too, you always pay a price.

I want to emphasize one important issue. Lets keep it modular and independent from each other as much as possible. Just the fact that you could encapsulate the payload (ISO 9660 filesystem in this case) in different containers (hard disk partition in particular) don't have to change its hash sum.

(In reply to comment #80)

> > and if necessary put the size of each ISO in a
> > comment next to the corresponding checksum.
> 
> You could do that. Or you could look at the size of original file, would it be
> locally or on the web. I see little difference.

But if the original file is long gone, and you only have the burned disc and a checksum file, AFAIK there's no way to determine the exact size of the original file from the disc contents, if the Volume Space Size doesn't correspond to the full file size. It may or may not be possible to locate the size online somewhere.

(In reply to comment #82)
> AFAIK there's no way to determine the exact size of the original
> file from the disc contents, if the Volume Space Size doesn't correspond to the
> full file size.

That's right. But I'm not against using Volume Space Size at all. It's just make no sense to account for foreign data there. And you could perfectly verify the ISO 9660 filesystem image while ignoring that garbage at its tail. Think of it another way: what if you would include that ISO 9660 filesystem in the tar archive, would you modify that Volume Space Size to account for the tar structures? Would you modify it if that was zip archive? That's just another container formats. That's the ISO 9660 image which you are interested in.

I took the Fedora-14-x86_64-netinst.iso file, which is 230686720 bytes, although the Volume Space Size is 112193 blocks, or 112193*2048 = 229771264 bytes. This file has a built-in mediacheck (I believe the F15 netinst does not, though). I truncated it to the latter size and ran mediacheck on it. It passes on both the original and the truncated file, either using the built-in mediacheck, or the external check "checkisomd5 -v Fedora-14-x86_64-netinst.iso". So does this mean that the file is perfectly functional even when truncated in this way? If so, then it would be possible to just checksum the Volume Space Size part. But then why was the file padded, and what would happen if one tried to boot from an image with the correct ISO part but with altered padding? Would it be possible to do something malicious?

(In reply to comment #84)
> So does this mean that the file is perfectly
> functional even when truncated in this way?

Yes.

> If so, then it would be possible to
> just checksum the Volume Space Size part. But then why was the file padded,

That was answered above in the comment #71. But I quote again here: "This makes it possible for the operating system, once booted, to use the remainder of the device for persistent storage by creating a second partition." The padding is there to round on the cylinder boundary which is required by partitioning.

> and
> what would happen if one tried to boot from an image with the correct ISO part
> but with altered padding?

Nothing bad would happen. It's just that you would have possible problems on writeable media trying to create a second partition after it.

> Would it be possible to do something malicious?

I beleive it won't be possible. See comment #76

I'm now leaning towards either closing this bug or changing it to something documentation-related, since these "ISOs" are in fact not expected to be ISO files. There's still the issue that the procedure to recover the original checksummed file from the burned disc should be documented somewhere, since the one for actual ISO files doesn't work. I checked several affected images (Fedora live, Fedora netinst, CentOS netinstall, openSUSE DVD) and in each case the change consists of zero-padding up to the next higher multiple of 1 MiB. So as long as isohybrid is the only utility which converts ISOs into non-ISOs, and it always uses the same defaults of h=64, s=32 (isohybrid.pl has the comment "# Fake geometry (zipdrive-style...)" for these values), so that h*s*512=1 MiB, there are only 2 possible candidate images, namely 1) the one read off by the above-mentioned rawread script, which assumes that the image is an ISO, and 2) what one gets by taking the image from 1) and zero-padding it up to the next higher multiple of 1 MiB. It's pretty easy to check both, and if there's a scriptable way to check if isohybrid was applied, simpler still. (Of course if the size is NOT a multiple of 1 MiB, it's not a hybrid image, but that's not an if-and-only-if criterion.)

I just want to be sure, though - is there any way the same result could be gotten without making these files non-ISOs - say by putting the padding _inside_ the ISOs? If there was, the current behavior could still be considered a bug.

(In reply to comment #86)
> as long as isohybrid is the only utility which converts ISOs into non-ISOs

I wouldn't count on it. There is GRUB 2 for example, which can do the same thing (according to its documentation at least).

> I just want to be sure, though - is there any way the same result could be
> gotten without making these files non-ISOs - say by putting the padding
> _inside_ the ISOs? If there was, the current behavior could still be considered
> a bug.

As I've already said it depends on the point of view. But as long as there are different people writing different software you can not expect that particular one to be universally accepted.

For 16-Alpha.TC1, strangely, the x86_64 DVD has the isohybrid padding, but the i386 DVD and netinst and the x86_64 netinst does not. Using hexdump as in comment 54 seems to indicate that both DVDs have an isolinux boot loader, but the size of the i386 DVD agrees with the Volume Space Size and is not a multiple of 1024**2.

We're going to need to put data in the padding region as part of supporting EFI booting of the isohybrid images, at which point there's no way to checksum the image based purely on the iso size. I disagree that the images fail to conform to the standard - the additional data is clearly not part of the Volume Space, and so doesn't need to be accounted for in the ISO header.

In light of the fact that in the future the padding may be arbitrary data and not just zeroes, this makes it necessary to specify the actual size. Filed bug 727387 to have this information added to the checksum files as comments.