Bug 735794 - i965: segfault in prepare_depthbuffer when switching to full screen with virtualbox
i965: segfault in prepare_depthbuffer when switching to full screen with virt...
Status: CLOSED WONTFIX
Product: Fedora
Classification: Fedora
Component: mesa (Show other bugs)
16
Unspecified Unspecified
unspecified Severity high
: ---
: ---
Assigned To: Adam Jackson
Fedora Extras Quality Assurance
[cat:crash]
: Patch, Triaged
: 766078 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2011-09-05 08:27 EDT by Alon Levy
Modified: 2014-08-04 18:08 EDT (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-02-13 02:56:25 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Alon Levy 2011-09-05 08:27:45 EDT
Description of problem:
Running virtualbox 4.1.2 with a win7 32 guest which has the wddm and direct3d driver installed, and changing to full screen, host is using i965 driver, triggers a segfault in i965, stack trace in further details. This is a result of a NULL dereference.

Version-Release number of selected component (if applicable):
mesa-dri-drivers-7.11-2.fc16.x86_64

How reproducible:
100%

Steps to Reproduce:
1. install a win7 guest with guest addons, choose to enable WDDM driver (might be reproducible with direct3d too. Note that there are at least two other bugs with standard packages in fedora already, gnome-shell bug 734183, compiz bug 717140, with the same symptoms, or I wouldn't have reported this)
2. launch guest
3. open ie9 in guest
4. browse to a sight with a video playing
5. switch to full screen (Host+F, Host defaults to right ctrl)
  
Actual results:
segfault, stack trace below

Expected results:
switches successfully to full screen

Additional info:
(gdb) bt
#0  0x00007f6b33a85f95 in prepare_depthbuffer (brw=0x7f6b384901e0) at brw_misc_state.
c:208
#1  0x00007f6b33a8f61d in brw_validate_state (brw=0x7f6b384901e0) at brw_state_upload
.c:480
#2  0x00007f6b33a7d8fb in brw_try_draw_prims (max_index=<optimized out>, min_index=<o
ptimized out>, ib=0x0, nr_prims=1, prim=0x7f6b384ebaec, arrays=0x7f6b384ed4d0, ctx=0x
7f6b384901e0) at brw_draw.c:340
#3  brw_draw_prims (ctx=0x7f6b384901e0, arrays=0x7f6b384ed4d0, prim=0x7f6b384ebaec, n
r_prims=1, ib=0x0, index_bounds_valid=<optimized out>, min_index=0, max_index=15) at
brw_draw.c:431
#4  0x00007f6b33b6ee18 in vbo_exec_vtx_flush (exec=0x7f6b384eb800, keepUnmapped=1 '\0
01') at vbo/vbo_exec_draw.c:390
#5  0x00007f6b33b6833c in vbo_exec_FlushVertices_internal (exec=0x7f6b384eb800, unmap
=<optimized out>) at vbo/vbo_exec_api.c:545
#6  0x00007f6b33b6ccac in vbo_exec_FlushVertices (ctx=0x7f6b384901e0, flags=<optimize
d out>) at vbo/vbo_exec_api.c:996
#7  0x00007f6b33b4384e in _mesa_BindTexture (target=34037, texName=939525102) at main
/texobj.c:1102
#8  0x00007f6b48ed0c6a in crUnpack () from /usr/lib/virtualbox/VBoxSharedCrOpenGL.so
#9  0x00007f6b48e6aa9c in crServerServiceClients () from /usr/lib/virtualbox/VBoxShar
edCrOpenGL.so
#10 0x00007f6b48e5e022 in crVBoxServerClientWrite () from /usr/lib/virtualbox/VBoxSha
redCrOpenGL.so
#11 0x00007f6b48e5c569 in ?? () from /usr/lib/virtualbox/VBoxSharedCrOpenGL.so
#12 0x00007f6b49e5fc45 in ?? () from /usr/lib/virtualbox/components/VBoxC.so
#13 0x00007f6b49e5e3ec in ?? () from /usr/lib/virtualbox/components/VBoxC.so
#14 0x00007f6b63bf240c in ?? () from /usr/lib/virtualbox/VBoxRT.so
#15 0x00007f6b63c3d383 in ?? () from /usr/lib/virtualbox/VBoxRT.so
#16 0x0000003d4a607d11 in start_thread (arg=0x7f6b49194700) at pthread_create.c:305
#17 0x0000003d4a2ea7dd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:115

(gdb) l
203        struct gl_framebuffer *fb = ctx->DrawBuffer;
204        struct intel_renderbuffer *drb = intel_get_renderbuffer(fb, BUFFER_DEPTH);
205        struct intel_renderbuffer *srb = intel_get_renderbuffer(fb, BUFFER_STENCIL
);
206
207        if (drb)
208           brw_add_validated_bo(brw, drb->region->buffer);
209        if (drb && drb->hiz_region)
210           brw_add_validated_bo(brw, drb->hiz_region->buffer);
211        if (srb)
212           brw_add_validated_bo(brw, srb->region->buffer);

Problem expression is on line 208, drb->region == NULL: (BUFFER_DEPTH == 4)

(gdb) p ((struct intel_renderbuffer*)fb->Attachment[4].Renderbuffer)->region
$1 = (struct intel_region *) 0x0

A simple fix that I'm just testing is:

commit a1251e89244f8e5237647e118a8ec129ed42f682
Author: Alon Levy <alevy@redhat.com>
Date:   Mon Sep 5 14:28:14 2011 +0300

    i965: prepare_depthbuffer: fix segfault, rhbz#717140
    
    Signed-off-by: Alon Levy <alevy@redhat.com>

diff --git a/src/mesa/drivers/dri/i965/brw_misc_state.c b/src/mesa/drivers/dri/i965/brw_misc_state.c
index f7e6e7c..785fb37 100644
--- a/src/mesa/drivers/dri/i965/brw_misc_state.c
+++ b/src/mesa/drivers/dri/i965/brw_misc_state.c
@@ -204,7 +204,7 @@ static void prepare_depthbuffer(struct brw_context *brw)
    struct intel_renderbuffer *drb = intel_get_renderbuffer(fb, BUFFER_DEPTH);
    struct intel_renderbuffer *srb = intel_get_renderbuffer(fb, BUFFER_STENCIL);
 
-   if (drb)
+   if (drb && drb->region)
       brw_add_validated_bo(brw, drb->region->buffer);
    if (drb && drb->hiz_region)
       brw_add_validated_bo(brw, drb->hiz_region->buffer);
Comment 1 Alon Levy 2011-09-05 08:43:44 EDT
that patch is not enough since srb->region == NULL too, so now trying this patch:

commit 426dc8fe8f45082841773382e24dc42a2d2dc22d
Author: Alon Levy <alevy@redhat.com>
Date:   Mon Sep 5 14:28:14 2011 +0300

    i965: prepare_depthbuffer: fix segfault, rhbz#735794
    
    Also fixes a segfault immediatelly after in the same case, i.e.
    srb->region is also NULL in the run of virtualbox described in
    the bug report in the subject.
    
    Signed-off-by: Alon Levy <alevy@redhat.com>

diff --git a/src/mesa/drivers/dri/i965/brw_misc_state.c b/src/mesa/drivers/dri/i965/brw_misc_state.c
index f7e6e7c..479cf82 100644
--- a/src/mesa/drivers/dri/i965/brw_misc_state.c
+++ b/src/mesa/drivers/dri/i965/brw_misc_state.c
@@ -204,11 +204,11 @@ static void prepare_depthbuffer(struct brw_context *brw)
    struct intel_renderbuffer *drb = intel_get_renderbuffer(fb, BUFFER_DEPTH);
    struct intel_renderbuffer *srb = intel_get_renderbuffer(fb, BUFFER_STENCIL);
 
-   if (drb)
+   if (drb && drb->region)
       brw_add_validated_bo(brw, drb->region->buffer);
    if (drb && drb->hiz_region)
       brw_add_validated_bo(brw, drb->hiz_region->buffer);
-   if (srb)
+   if (srb && srb->region)
       brw_add_validated_bo(brw, srb->region->buffer);
 }
Comment 2 Alon Levy 2011-09-05 09:15:35 EDT
That just makes it fail again later, at intel_renderbuffer_tile_offsets



(gdb) bt
#0  intel_renderbuffer_tile_offsets (irb=0x7f98ceb39ec0, tile_x=0x7f98dddcc898, tile_y=0x7f98dddcc89c)
at intel_fbo.c:659
#1  0x00007f98dcb48dca in brw_update_renderbuffer_surface (unit=0, rb=0x7f98ceb39ec0, brw=0x7f98cc4901e
0) at brw_wm_surface_state.c:480
#2  upload_wm_surfaces (brw=0x7f98cc4901e0) at brw_wm_surface_state.c:588
#3  0x00007f98dcb32729 in brw_upload_state (brw=0x7f98cc4901e0) at brw_state_upload.c:553
#4  0x00007f98dcb20930 in brw_try_draw_prims (max_index=<optimized out>, min_index=<optimized out>, ib=
0x0, nr_prims=1, prim=0x7f98cc4ebaec, arrays=0x7f98cc4ed4d0, ctx=0x7f98cc4901e0) at brw_draw.c:370
#5  brw_draw_prims (ctx=0x7f98cc4901e0, arrays=0x7f98cc4ed4d0, prim=0x7f98cc4ebaec, nr_prims=1, ib=0x0,
 index_bounds_valid=<optimized out>, min_index=0, max_index=15) at brw_draw.c:431
#6  0x00007f98dcc11e28 in vbo_exec_vtx_flush (exec=0x7f98cc4eb800, keepUnmapped=1 '\001') at vbo/vbo_ex
ec_draw.c:390
#7  0x00007f98dcc0b34c in vbo_exec_FlushVertices_internal (exec=0x7f98cc4eb800, unmap=<optimized out>)
at vbo/vbo_exec_api.c:545
#8  0x00007f98dcc0fcbc in vbo_exec_FlushVertices (ctx=0x7f98cc4901e0, flags=<optimized out>) at vbo/vbo
_exec_api.c:996
#9  0x00007f98dcbe685e in _mesa_BindTexture (target=34037, texName=622) at main/texobj.c:1102
#10 0x00007f98ddb09c6a in crUnpack () from /usr/lib/virtualbox/VBoxSharedCrOpenGL.so

So I think the right approach would be to make intel_renderbuffer return NULL when irb->region == NULL. I'll try that next.

Alon
Comment 3 Alon Levy 2011-09-05 10:35:20 EDT
Tried something else, namely: (this patch is for the mesa package, not upstream - upstream broke a line in to to preserve max columns, so it didn't apply to mesa)

From fd46e2c7bee1d0c30a66fdee37f0c7d192322481 Mon Sep 17 00:00:00 2001
From: Alon Levy <alevy@redhat.com>
Date: Mon, 5 Sep 2011 16:38:43 +0300
Subject: [PATCH] i965: prepare_depthbuffer: don't update NULL region'ed
 surface, rhbz#735794

---
 src/mesa/drivers/dri/i965/brw_wm_surface_state.c |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)

diff --git a/src/mesa/drivers/dri/i965/brw_wm_surface_state.c b/src/mesa/drivers/dri/i965/brw_wm_surface_state.c
index 89fea9c..8cc1e91 100644
--- a/src/mesa/drivers/dri/i965/brw_wm_surface_state.c
+++ b/src/mesa/drivers/dri/i965/brw_wm_surface_state.c
@@ -439,6 +439,9 @@ brw_update_renderbuffer_surface(struct brw_context *brw,
    uint32_t tile_x, tile_y;
    uint32_t format = 0;
 
+   if (irb->region == NULL)
+      return;
+
    surf = brw_state_batch(brw, 6 * 4, 32, &brw->wm.surf_offset[unit]);
 
    switch (irb->Base.Format) {
-- 
1.7.6.1


And then it failed a bit later at:

(gdb) up
#1  0x00007ffb9f86a729 in brw_upload_state (brw=0x7ffba44901e0) at brw_state_upl
oad.c:553
(gdb) down
#0  emit_depthbuffer (brw=0x7ffba44901e0) at brw_misc_state.c:318
(gdb) bt
#0  emit_depthbuffer (brw=0x7ffba44901e0) at brw_misc_state.c:318
#1  0x00007ffb9f86a729 in brw_upload_state (brw=0x7ffba44901e0) at brw_state_upl
oad.c:553
#2  0x00007ffb9f858930 in brw_try_draw_prims (max_index=<optimized out>, min_ind
ex=<optimized out>, ib=0x0, nr_prims=1, prim=0x7ffba44ebaec, arrays=0x7ffba44ed4
d0, ctx=0x7ffba44901e0) at brw_draw.c:370
#3  brw_draw_prims (ctx=0x7ffba44901e0, arrays=0x7ffba44ed4d0, prim=0x7ffba44eba
ec, nr_prims=1, ib=0x0, index_bounds_valid=<optimized out>, min_index=0, max_ind
ex=15) at brw_draw.c:431
#4  0x00007ffb9f949e38 in vbo_exec_vtx_flush (exec=0x7ffba44eb800, keepUnmapped=
1 '\001') at vbo/vbo_exec_draw.c:390

So now really trying what I suggested at comment 2.
Comment 4 Alon Levy 2011-09-05 14:27:21 EDT
comment 2 was wrong, since glClear's intel implementation relies on intel_get_renderbuffer being non NULL. But combining the two previous patches in comment 1 and comment 3, together with the following patch, fixes the problem:

commit 1ecdae72917f4485bd7fea204d346d05a18ee055
Author: Alon Levy <alevy@redhat.com>
Date:   Mon Sep 5 19:25:19 2011 +0300

    i965: emit_depthbuffer: fix segfault, rhbz#735794
    
    Signed-off-by: Alon Levy <alevy@redhat.com>

diff --git a/src/mesa/drivers/dri/i965/brw_misc_state.c b/src/mesa/drivers/dri/i965/brw_misc_state.c
index 479cf82..f102bc6 100644
--- a/src/mesa/drivers/dri/i965/brw_misc_state.c
+++ b/src/mesa/drivers/dri/i965/brw_misc_state.c
@@ -248,6 +248,9 @@ static void emit_depthbuffer(struct brw_context *brw)
    else
       len = 5;
 
+   if (depth_irb && depth_irb->region == NULL)
+      depth_irb = NULL;
+
    if (!depth_irb && !stencil_irb) {
       BEGIN_BATCH(len);
       OUT_BATCH(_3DSTATE_DEPTH_BUFFER << 16 | (len - 2));
Comment 5 Alon Levy 2011-09-05 14:28:44 EDT
put a scratch build with those three patches:

http://koji.fedoraproject.org/koji/taskinfo?taskID=3325476

Alon
Comment 6 Alon Levy 2011-09-05 15:11:23 EDT
better scratch build - same patches, but release incremented:

http://koji.fedoraproject.org/koji/taskinfo?taskID=3325495
Comment 7 Mamoru TASAKA 2011-12-11 20:23:39 EST
*** Bug 766078 has been marked as a duplicate of this bug. ***
Comment 8 Fedora End Of Life 2013-01-16 04:56:45 EST
This message is a reminder that Fedora 16 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 16. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '16'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 16's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 16 is end of life. If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora, you are encouraged to click on 
"Clone This Bug" and open it against that version of Fedora.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 9 Fedora End Of Life 2013-02-13 02:56:28 EST
Fedora 16 changed to end-of-life (EOL) status on 2013-02-12. Fedora 16 is 
no longer maintained, which means that it will not receive any further 
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of 
Fedora please feel free to reopen this bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.

Note You need to log in before you can comment on or make changes to this bug.