Bug 1829533 - ImageMagick 6.9 Upgrade Causes Huge Increase in CPU and Memory Usage
Summary: ImageMagick 6.9 Upgrade Causes Huge Increase in CPU and Memory Usage
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: ImageMagick
Version: 7.8
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: rc
: ---
Assignee: Jan Horak
QA Contact: Desktop QE
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-04-29 17:47 UTC by Redhat RHN
Modified: 2020-05-15 07:48 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-05-15 07:48:19 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
CentOS 17308 None None None 2020-04-29 17:47:22 UTC

Description Redhat RHN 2020-04-29 17:47:22 UTC
Description of problem:

Yesterday we upgraded our test hosts to CentOS 7.8, which included an upgrade to ImageMagick from 6.7 to 6.9. This upgrade caused a noticeable and detrimental effect in the use of the convert command. Processing time went from ~3 seconds to ~26 seconds in our 5425x5424 pixel images. The delays scaled by a factor of 4x, given the X by Y increase in pixels.


Version-Release number of selected component (if applicable):
  ImageMagick.x86_64 0:6.9.10.68-3.el7                                                                                  


How reproducible:

Easily reproducible.

Steps to Reproduce:
cd /dev/shm

wget https://cdn.star.nesdis.noaa.gov/GOES16/ABI/FD/09/5424x5424.jpg

convert -version

time convert -define registry:temporary-path=/dev/shm/ -limit memory 2gb 20201201440_GOES16-ABI-FD-02-LOWLIGHT4-5424x5424.jpg -gravity south -chop 0x12 -font Liberation-Sans-Regular -pointsize 40 -fill black -gravity South label:'April 29 2020 14:40Z GOES-East ABI Band 1o - Full Disk' -gravity Center -append /opt/deployed/goes16-imagery//noaa_logo//noaa_logo_350px.png -gravity SouthWest -geometry +24+42 -composite -resize 5424x5424! OLD_IM.jpg

sudo yum versionlock clear && sudo yum -y update ImageMagick 1</dev/null 2</dev/null

convert -version

time convert -define registry:temporary-path=/dev/shm/ -limit memory 2gb 20201201440_GOES16-ABI-FD-02-LOWLIGHT4-5424x5424.jpg -gravity south -chop 0x12 -font Liberation-Sans-Regular -pointsize 40 -fill black -gravity South label:'April 29 2020 14:40Z GOES-East ABI Band 10 - Full Disk' -gravity Center -append /opt/deployed/goes16-imagery//noaa_logo//noaa_logo_350px.png -gravity SouthWest -geometry +24+42 -composite -resize 5424x5424! NEW_IM.jpg



Actual results:
Version 6.9

-bash-4.2$  convert -version
Version: ImageMagick 6.9.10-68 Q16 x86_64 2020-04-01 https://imagemagick.org
Copyright: © 1999-2019 ImageMagick Studio LLC
License: https://imagemagick.org/script/license.php
Features: Cipher DPC Modules OpenMP(3.1) 
Delegates (built-in): bzlib cairo fontconfig freetype gslib jng jpeg lcms ltdl lzma openexr pangocairo png ps rsvg tiff wmf x xml zlib
-bash-4.2$  time convert -define registry:temporary-path=/dev/shm/ -limit memory 2gb 20201201440_GOES16-ABI-FD-02-LOWLIGHT4-5424x5424.jpg -gravity south -chop 0x12 -font Liberation-Sans-Regular -pointsize 40 -fill black -gravity South label:'April 29 2020 14:40Z GOES-East ABI Band 10 - Full Disk' -gravity Center -append /opt/deployed/goes16-imagery//noaa_logo//noaa_logo_350px.png -gravity SouthWest -geometry +24+42 -composite -resize 5424x5424! NEW_IM.jpg

real	0m26.948s
user	0m11.192s
sys	0m38.080s


Expected results:
-bash-4.2$  convert -version
Version: ImageMagick 6.7.8-9 2019-08-08 Q16 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2012 ImageMagick Studio LLC
Features: OpenMP    

-bash-4.2$  time convert -define registry:temporary-path=/dev/shm/ -limit memory 2gb 20201201440_GOES16-ABI-FD-02-LOWLIGHT4-5424x5424.jpg -gravity south -chop 0x12 -font Liberation-Sans-Regular -pointsize 40 -fill black -gravity South label:'April 29 2020 14:40Z GOES-East ABI Band 10 - Full Disk' -gravity Center -append /opt/deployed/goes16-imagery//noaa_logo//noaa_logo_350px.png -gravity SouthWest -geometry +24+42 -composite -resize 5424x5424! NEW_OLD.jpg

real	0m3.337s
user	0m4.276s
sys	0m0.779s


Additional info:

We produce nearly 1 million JPGs every day from our satellite data. Any increase in CPU usage, memory usage, or overall processing time has a ripple effect that is hard to recover from, given the rapidity volume(MB) of data.

Comment 2 Jan Horak 2020-04-30 07:53:18 UTC
I'm having the same problem on ImageMagick-6.9.10.86-2.fc32.x86_64, so we could consider to move it to upstream. Removing "-limit memory 2gb" helped though.

Comment 3 Redhat RHN 2020-04-30 12:13:26 UTC
Thanks, Jan,  I removed, before posting, the -sharpen option too, it was hogging even more resources.  I thought about reporting upstream, but wasn't sure if it was a compiling issue or would be easier to move up/down a point release on the RH side.

Comment 4 Jan Horak 2020-05-15 07:48:19 UTC
Sorry, we won't address it in the RHEL7 unless there's a strong business need for that. If you see it in latest upstream version, please fill a bugreport to the ImageMagick upstream issues.


Note You need to log in before you can comment on or make changes to this bug.