Bug 1637677

Summary: [abrt] gnome-shell: raise(): gnome-shell killed by SIGTRAP, maybe related to libst blur_pixels
Product: Red Hat Enterprise Linux 7 Reporter: Paulo Andrade <pandrade>
Component: gnome-shellAssignee: Florian Müllner <fmuellner>
Status: CLOSED WONTFIX QA Contact: Desktop QE <desktop-qa-list>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.5CC: aubin.guillemette, cww, jadahl
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-06-12 14:55:03 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Paulo Andrade 2018-10-09 18:26:17 UTC 
Customer is experiencing bug #1526164 that should have been fixed in
https://src.fedoraproject.org/cgit/rpms/gnome-shell.git/commit/?h=f27&id=76c9c60c2d35a28fed835ae6da937e935366d3a5
  The crash matches the pattern in the f27 report. While not understanding
some internals, it appears it is using a released object, with memory
likely reused for something else:

(gdb) frame 10
#10 0x00007f04e2b2fe1d in _st_create_shadow_pipeline (
    shadow_spec=shadow_spec@entry=0x4aa2c40, 
    src_texture=src_texture@entry=0x11ecf70) at st/st-private.c:372
372	  pixels_out = blur_pixels (pixels_in, width_in, height_in, rowstride_in,
(gdb) p shadow_spoec
No symbol "shadow_spoec" in current context.
(gdb) p shadow_spec
$1 = (StShadow *) 0x4aa2c40
(gdb) p* shadow_spec
$2 = {color = {red = 0 '\000', green = 0 '\000', blue = 0 '\000', 
    alpha = 0 '\000'}, xoffset = 0, yoffset = 0, blur = 33554432.000000253, 
  spread = 33554440.1875, inset = 1098907648, ref_count = 0}
(gdb) 

because it sues shadow_spec->blur to calculate the image size, it ends
up requesting a huge amount of memory...

257	  sigma = blur / 2.;
[...]
273	      n_values = (gint) 5 * sigma;
274	      half = n_values / 2;
275	
276	      *width_out  = width_in  + 2 * half;
277	      *height_out = height_in + 2 * half;
278	      *rowstride_out = (*width_out + 3) & ~3;
279	
280	      pixels_out = g_malloc0 (*rowstride_out * *height_out);
281	      line       = g_malloc0 (*rowstride_out);
282	
(gdb) p *width_out
$3 = 83886096
(gdb) p *height_out
$4 = 83886096
(gdb) p width_in
$5 = 16
(gdb) p height_in
$6 = 16