Bug 704674 - Stack corruption when built with -O2
Summary: Stack corruption when built with -O2
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: dssi-vst
Version: 15
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Orcan Ogetbil
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-05-14 00:36 UTC by Brendan Jones
Modified: 2011-07-06 21:37 UTC (History)
5 users (show)

Fixed In Version: dssi-vst-0.9.2-4.fc15
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2011-07-06 21:37:58 UTC
Type: ---


Attachments (Terms of Use)
vsthost stacktrace (5.33 KB, text/plain)
2011-05-14 00:36 UTC, Brendan Jones
no flags Details
diff O2 flags gcc 4.5.1 vs 4.6.1 (1.68 KB, text/plain)
2011-05-14 00:38 UTC, Brendan Jones
no flags Details

Description Brendan Jones 2011-05-14 00:36:25 UTC
Created attachment 498881 [details]
vsthost stacktrace

Description of problem:

currently dssi-vst page faults on all VST plugins (see attached backtrace).
Using the package built for F14 appears to instantiate the plugin without error.

After doing some debugging it appears that there is some kind of stack corruption occurring at line 1272 dssi-vst-server.cpp
1240     AEffect *(__stdcall* getInstance)(audioMasterCallback);
1241 
1242     getInstance = (AEffect*(__stdcall*)(audioMasterCallback))
1243     GetProcAddress(libHandle, NEW_PLUGIN_ENTRY_POINT);
1244 
1245     if (!getInstance) {
1246     if (debugLevel > 0) {
1247         cerr << "dssi-vst-server[1]: VST 2.4 entrypoint \""
1248          << NEW_PLUGIN_ENTRY_POINT << "\" not found in DLL \""
1249          << libname << "\", looking for \""
1250          << OLD_PLUGIN_ENTRY_POINT << "\"" << endl;
1251     }
1252 
1253     getInstance = (AEffect*(__stdcall*)(audioMasterCallback))
1254         GetProcAddress(libHandle, OLD_PLUGIN_ENTRY_POINT);
1255 
1256     if (!getInstance) {
1257         cerr << "dssi-vst-server: ERROR: VST entrypoints \""
1258          << NEW_PLUGIN_ENTRY_POINT << "\" or \""
1259          << OLD_PLUGIN_ENTRY_POINT << "\" not found in DLL \""
1260          << libname << "\"" << endl;
1261         return 1;
1262     } else if (debugLevel > 0) {
1263         cerr << "dssi-vst-server[1]: VST entrypoint \""
1264          << OLD_PLUGIN_ENTRY_POINT << "\" found" << endl;
1265     }
1266 
1267     } else if (debugLevel > 0) {
1268     cerr << "dssi-vst-server[1]: VST entrypoint \""
1269          << NEW_PLUGIN_ENTRY_POINT << "\" found" << endl;
1270     }
1271 
1272     AEffect *plugin = getInstance(hostCallback);
1273 

After this point the memory pointed to by fileInfo has changed. The program crashes at some point after

Version-Release number of selected component (if applicable):
dssi-vst-wine-0.9.2
wine-1.3.19.i686
gcc-4.6.0

I can get dssi-vst-wine to work by rebuilding without %{optflags}, specifically without the -O2 flag. The flags governed by O2 have changed between gcc versions in F15 and F14 and I'm yet to narrow it down to one is causing the problem. 


How reproducible:

Always

Steps to Reproduce:

vsthost [some plugin]

Comment 1 Brendan Jones 2011-05-14 00:38:50 UTC
Created attachment 498882 [details]
diff O2 flags gcc 4.5.1 vs 4.6.1

diff <(/fedora14/usr/bin/gcc -c -Q -O2 --help=optimizers) <(/usr/bin/gcc -c -Q -O2 --help=optimizers)

Comment 2 Orcan Ogetbil 2011-05-14 02:51:16 UTC
I am assigning this to gcc maintainers. Their opinion might help us track the issue.

Comment 3 Jakub Jelinek 2011-05-14 06:49:23 UTC
If code doesn't work with -O2, but works with -O0, it might be of course a compiler bug, but many times more likely it is just a bug in the program being compiled, where it relies on undefined behavior somewhere.  Therefore, I'm not going to debug arbitrarily large wrong-code issues unless the package maintainers do their homework and reduce it significantly.  You can use a binary search in between objects compiled with -O0 and -O2 (if -O0 works) to narrow it down to a particular object, in that object try -fno-inline to see if inlining isn't irrelevant, use __attribute__((optimize (0))) on some routines in the file to narrow it down to a particular routine (or you can alternatively use a debugger to see where things go wrong).  Try -fno-strict-aliasing, if that helps, you might be violating strict aliasing, check -W -Wall output, etc.
See http://gcc.gnu.org/bugs.html for more details about proper gcc bugreporting.
As I said, most of the time you just find a bug in your package.

Comment 4 Orcan Ogetbil 2011-05-14 14:51:11 UTC
Thanks Jakub. I am aware that such bugs usually occur due the code relying on undefined behavior (why the compiler changes its behavior between releases in such circumstances is beyond my understanding). I didn't assign the bug to you for requesting to go through the code. I was just wondering if this is one of the common issues you encounter, and if there are specific things to look for. Again thanks for the feedback.

Comment 5 Brendan Jones 2011-05-15 01:10:53 UTC
Rebuilding with %{optflags} -fno-omit-frame-pointer -Ivestige -fPIC results in a working dssi-vst-wine.

I'm not across wine but a little research has found that some apps require -fno-omit-frame-pointer to work correctly (eg. LMMS sets this when compiling for VST support).

Comment 6 Lars G 2011-06-17 23:28:07 UTC
*Bump*
Please rebuild for Fedora 15.

Thanks Lars

Comment 7 Fedora Update System 2011-06-18 16:57:43 UTC
dssi-vst-0.9.2-4.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/dssi-vst-0.9.2-4.fc15

Comment 8 Orcan Ogetbil 2011-06-18 16:59:08 UTC
I built on F-15+ with the -fno-omit-frame-pointer flag as it is the only solution we have right now. Let us know how it goes.

Comment 9 Lars G 2011-06-18 17:20:17 UTC
Thanks!
Works ok here.

Comment 10 Brendan Jones 2011-06-18 17:26:22 UTC
FWIW I posted this upstream a while ago. No response as yet.

http://code.breakfastquay.com/issues/6

Comment 11 Orcan Ogetbil 2011-06-18 17:34:07 UTC
I had the same issue with this upstream tracker in the past. I think it is best to email the author directly in this case. See:
   http://lists.linuxaudio.org/pipermail/linux-audio-user/2010-May/069096.html

Comment 12 Fedora Update System 2011-06-21 17:11:50 UTC
Package dssi-vst-0.9.2-4.fc15:
* should fix your issue,
* was pushed to the Fedora 15 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing dssi-vst-0.9.2-4.fc15'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/dssi-vst-0.9.2-4.fc15
then log in and leave karma (feedback).

Comment 13 Fedora Update System 2011-07-06 21:37:52 UTC
dssi-vst-0.9.2-4.fc15 has been pushed to the Fedora 15 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.