From Bugzilla Helper: User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.5) Gecko/20011031 Description of problem: If you create a PGM with maxval=65535 and then use pnmdepth to scale it, the resulting PGM is not scaled properly due to integer overflow in the scaling algorithm. In particular, the following lines in pnndepth.c are the culprit: for ( i = 0; i <= maxval; ++i ) newvals[i] = ( i * newmaxval + maxval / 2 ) / maxval; Version-Release number of selected component (if applicable): How reproducible: Always Steps to Reproduce: Attached in the "additional information" section is a perl script which creates a 16-bit PGM. Run it via vignette.pl > foo16.pgm Use something like xv to view the PGM (xv happily handles 16-bit PGM, the GIMP does not). Scale the PGM via pnmdepth 255 foo16.pgm > foo8.pgm Now view the resulting PGM with xv. Actual Results: The original PGM is a smooth grayscale image running from about 70% white at the edge to 100% white in the center. The pnmdepth scaled image looks like I was playing with some fractal generator. Expected Results: The pnmdepth scaled image should have looked substantially the same as the original. Additional info: The "easy" fix is to promote the calculation to double instead of integer but that is probably a bad thing on a real 386 with no floating-point support. Still, the mapping is calculated only once and them used in a lookup table, so it isn't a total killer. Here is the perl script (watch for spurious line wraps): #! /usr/bin/perl $opt{maxval} = 65535; $opt{xsize} = 160; $opt{ysize} = 120; $opt{minval} = 0.7; printf ("P5 %d %d %d\n", $opt{xsize}, $opt{ysize}, $opt{maxval}); $h = 0.5 * sqrt($opt{xsize} * $opt{xsize} + $opt{ysize} * $opt{ysize}); $d_square = $h * $h * sqrt($opt{minval}) / (1.0 - sqrt($opt{minval})); $d = $h * sqrt(sqrt($opt{minval})) / sqrt(1.0 - sqrt($opt{minval})); for ($j = 0, $y = 0.5*($opt{ysize}-1); $j < $opt{ysize}; $j++, $y -= 1.0) { for ($i = 0, $x = -0.5*($opt{xsize}-1); $i < $opt{xsize}; $i++, $x += 1.0) { $h_square = $x*$x + $y*$y; $v = $h_square / ($d_square + $h_square); $v = 1 - $v; $v *= $v; $v *= $opt{maxval}; print pack('S', int($v)); } }
My analysis of the bug is wrong; the code in question can't be overflowing as newval[] should be of type unsigned long is more than big enough to handle the 65535*255+65535/2 calculation for i=maxval. I'll run it under the debugger and see what I can figure out....
It's an endian problem. But there is no endian setting which will result in an image which displays correctly via xv or ee as a 16-bit grayscale and still display correctly as an 8-bit grayscale when converted via pnmdepth. I *can* get a correct image by forcing keeping everything as ascii format pgm files (i.e., P2 instead of P5).
Apparently xv doesn't read 16-bit pgm format correctly. both ee and display (ImageMagick) agree on the image content and a careful reading of the netpbm code indicates that 16-bit raw should be in network byte-order. So the problem is *not* in netpbm but rather in xv.