Bug 1006865

Thank you for your bugreport and willing make free software better!

Reported upstream: http://www.imagemagick.org/discourse-server/viewtopic.php?f=3&t=24093

We close bug now, as it related to upstream developing. But we continue track changes and whatever it will be fixed we consider make update in Fedora.

John could you please provide reproduce image?

Created attachment 797977 [details]
repro image 1

Created attachment 797978 [details]
repro image 2

(In reply to Pavel Alexeev (aka Pahan-Hubbitus) from comment #2)
> John could you please provide reproduce image?

Sure, though I showed where the image came from above, I don't think there's anything special about that particular one.

Here are two similar images taken from a different machine. The smaller image (boot-quick.svg) is from a normal, working boot sequence that completed in 30s. This is a bit faster than the original machine which has a more complicated boot sequence, so the resulting SVG is a bit smaller. display still takes 10s to show a window and just under 30s to render the SVG.

The first time I tried this I actually hit a snag in one of the services that stalled in the presence of a stale lock file. Thus the boot sequence there is 5 minutes long, resulting in a much wider SVG (boot-slow.svg). display again takes 10s to display a blank window, then things get much worse. The original machine has a lot of RAM so I didn't notice, but the second machine is a laptop with 3GB, and display rapidly eats all of that then goes in to swap, then takes even longer to chew through the SVG, before failing to render it at all: CPU usage eventually falls off, the window starts responding, the "pan window" allows you to move around the "image", but the main display window remains grey.

And again, Firefox opens both images in less than a second, displays them perfectly, maintains responsiveness and a reasonable (for Firefox!) memory usage.

Thank you John.

(Following the upstream discussion - I don't have an account there. Feel free to repost/quote/point to this if you feel appropriate.)

On the original machine with the original image:

bash$ convert -list format | grep SVG
     MSVG  SVG       rw+   ImageMagick's own SVG internal renderer
      SVG  SVG       rw+   Scalable Vector Graphics (RSVG 2.36.1)
     SVGZ  SVG       rw+   Compressed Scalable Vector Graphics (RSVG 2.36.1)

bash$ convert /tmp/boot.svg show

This spins at 100% of a single CPU. Memory usage is a fairly constant 1GB throughout, which is about what firefox uses to open the image (from a clean start with no other tabs/windows open.)

After 20 minutes I started digging and realised that that is not a command to "show" the image, but just a normal image->image conversion which by default writes a new SVG to a file called show. At the point I killed it the file was 4.8GB in size, though still showing the same SVG dimensions as the original 256kB SVG file. The start of the file is:

<?xml version="1.0" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 20010904//EN"
  "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd">
<svg width="23991" height="3790">
  <circle cx="0" cy="0" r="1" fill="white"/>
  <circle cx="1" cy="0" r="1" fill="white"/>
  <circle cx="2" cy="0" r="1" fill="white"/>
  <circle cx="3" cy="0" r="1" fill="white"/>
  <circle cx="4" cy="0" r="1" fill="white"/>

And this carries on for cx=0..23990, then for cy=1, cy=2 etc.

Yes, it has decided to convert SVG to SVG by emitting a one-pixel radius circle for every pixel location in the original image... Adding "-density 9" reduces the size, allowing it to complete in 32 seconds, leaving a 90MB file. Which takes 18 seconds to crash rsvg-convert with the error "internal error: Huge input lookup".

Even if that had worked, "If the intention is to display the entire image scaled down to the screen, rendering all the pixels is a waste of time, and specifying a low density will save time and memory" is incorrect. A scaled version of these images (any of them) is completely illegible and useless. What is needed is a full size, dot-for-dot, pannable window. (It would be nice if IM's pan window could be made a bit bigger, it tends to get unwieldy/inaccurate for very large zoom factors, but that's a separate matter.)

Note that if I install the xloadimage package and use the xview program, with the following added to ~/.xloadimagerc:

filter = "rsvg-convert" .svg

/usr/bin/rsvg-convert: Thu 02 May 2013 15:36:21 BST  	librsvg2-2.36.4-1.fc18.x86_64.rpm

So this is using RSVG to do the rasterisation, just like IM.

(In the following, image windows are closed as soon as render is complete, so the times shown are the time to first render.)

bash$ time rsvg-convert </tmp/boot.svg >/tmp/boot-rsvgout.png
real	0m9.685s
user	0m9.487s
sys	0m0.136s
orthrus$ file /tmp/boot-rsvgout.png 
/tmp/boot-rsvgout.png: PNG image data, 23991 x 3790, 8-bit/color RGBA, non-interlaced

bash$ time xview /tmp/boot.svg
/tmp/boot.svg is 23991x3790 PNG image, color type RGB_ALPHA, 8 bit [after 2 seconds]
  Building XImage...done [after
real	0m11.325s
user	0m12.243s
sys	0m0.491s

bash$ time display /tmp/boot-rsvgout.png 
real	0m59.482s
user	1m6.736s
sys	0m2.634s

orthrus$ time xview /tmp/boot-rsvgout.png 
/tmp/boot-rsvgout.png is 23991x3790 PNG image, color type RGB_ALPHA, 8 bit
  Building XImage...done
real	0m3.948s
user	0m3.046s
sys	0m0.213s

So while RSVG's conversion time of 10s is clearly much longer than Firefox's subsecond response, the bulk of the time is in loading/displaying the rasterised image, which even xview manages reasonably quickly (slower than Firefox still, but xview/xloadimage have never been known for their efficiency.)

One last try, this time forcing the IM internal MSVG handler:

orthrus$ time display msvg:/tmp/boot.svg
real	0m33.766s
user	0m53.200s
sys	0m1.543s

Still slow, but faster than RSVG (note the bigger ratio between elapsed and CPU time here)... but the main graph portion of the image is one big black rectangle. Presumable some transparency effect it's getting wrong or not handling there.

Oh, and on the issue of the files being "big". 20 years ago they would probably have been considered beyond the capabilities of most machines. These days, not so much. 4k/8k imagery is gaining ground in photography and film, and even a 24kx4k image such as this, weighing in at ~300MB uncompressed, is no longer an unreasonable amount of data to expect modern machines to be able to handle. The fact that above we have multiple other programs (xview, rsvg-convert, Firefox, you said Chrome too) apparently able to deal gracefully with it proves this.

---

I've just realised the "convert ... show" command should have been "convert ... show:", to force the "show" handler rather than a filename of show. Doing that instead, convert returns to the shell after 8 seconds (presumably this is the rasterisation step), after having spawned a separate display process on a file in /tmp. display then sits there at a grey window for 1m04s before rendering. So a little faster, again, but still too slow.

And, why? convert is just delegating to display for actual display, both are part of IM, no special conversion options were specified to either, surely they should behave pretty much identically?