Bug 959001

Summary: libtheora: Segfault in oc_dec_init (upstream patch available)
Product: [Fedora] Fedora Reporter: Andrew Price <anprice>
Component: libtheoraAssignee: Adam Jackson <ajax>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: 19CC: ajax, bruce, hdegoede, kparal
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-05-04 01:45:36 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Andrew Price 2013-05-02 23:31:29 UTC
Description of problem:

#0  0x0000003275c034cb in oc_dec_init (_setup=<optimized out>, _info=0x7fffffffcb70, _dec=0x28d1690) at decode.c:400
400	      qsum+=_dec->state.dequant_tables[qti][pli][qi][12]+

Version-Release number of selected component (if applicable):

libtheora-1.1.1-5.fc19

How reproducible:

100%

Steps to Reproduce:
1. Compiled ags
2. Ran Resonance with it

Actual results:

Segfault as above

Expected results:

No segfault

Additional info:

There is an upstream patch for this problem available in libtheora's svn trunk:

[andy@phanto lib]$ svn log -c 18268
------------------------------------------------------------------------
r18268 | tterribe | 2012-05-08 03:51:57 +0100 (Tue, 08 May 2012) | 8 lines

Fix pp_sharp_mod calculation.

This was broken when the dequant_tables indexing changed in commit
 r16102, but it only affected post-processing quality, so we never
 noticed.
With gcc 4.8.0, this can now trigger a segfault during decoder
 initialization.

------------------------------------------------------------------------
[andy@phanto lib]$ svn diff -c 18268
Index: decode.c
===================================================================
--- decode.c	(revision 18267)
+++ decode.c	(revision 18268)
@@ -400,10 +400,10 @@
     int qsum;
     qsum=0;
     for(qti=0;qti<2;qti++)for(pli=0;pli<3;pli++){
-      qsum+=_dec->state.dequant_tables[qti][pli][qi][12]+
-       _dec->state.dequant_tables[qti][pli][qi][17]+
-       _dec->state.dequant_tables[qti][pli][qi][18]+
-       _dec->state.dequant_tables[qti][pli][qi][24]<<(pli==0);
+      qsum+=_dec->state.dequant_tables[qi][pli][qti][12]+
+       _dec->state.dequant_tables[qi][pli][qti][17]+
+       _dec->state.dequant_tables[qi][pli][qti][18]+
+       _dec->state.dequant_tables[qi][pli][qti][24]<<(pli==0);
     }
     _dec->pp_sharp_mod[qi]=-(qsum>>11);
   }

Comment 1 Hans de Goede 2013-05-03 13:05:02 UTC
*** Bug 924435 has been marked as a duplicate of this bug. ***

Comment 2 Fedora Update System 2013-05-03 13:47:42 UTC
libtheora-1.1.1-6.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/libtheora-1.1.1-6.fc19

Comment 3 Bruce Cowan 2013-05-03 15:29:18 UTC
*** Bug 957228 has been marked as a duplicate of this bug. ***

Comment 4 Fedora Update System 2013-05-04 01:45:38 UTC
libtheora-1.1.1-6.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.