Description of problem: exec-shield (mode 2) kills mplayer in some cases (depending on used codec). In original announcement I've read that I can disable exec-shield per application, if switched to mode 2: http://people.redhat.com/mingo/exec-shield/ANNOUNCE-exec-shield There was also described the utility "./chstk" (which is no longer available on main site, but found on an "mirror" somewhere. I've tried different settings, but this doesn't help. Only disabling exec-shield (mode 0) helps. Are the flags no longer checked? Version-Release number of selected component (if applicable): kernel-2.4.22-1.2115.nptl How reproducible: Always Steps to Reproduce: 1.activate exec-shield (default) 2.play a movie with mplayer which requires [dmo] DMO video codecs Actual Results: mplayer crashes before start showing the movie Expected Results: no crash, like in exec-shield mode 0 Additional info: Mplayer versions tested: 0.92, 1.0pre2
chkstk is the wrong app to mark binaries; you need /usr/bin/execstack for that. It seems mplayer has a bug where it doesn't use PROT_EXEC for the codecs it loads, mplayer will need to be fixed for real... Also the "2" you mention is not applicable to this version of execshield, read the fedora release notes for the real information
Ok, read the RL, played around with execstack, doesn't help in any case except setting kernel.exec-shield=0. Note: mplayer was rebuilt on this FC1 system from SRPMS. # execstack -s /usr/bin/mplayer # execstack -q /usr/bin/mplayer X /usr/bin/mplayer # sysctl kernel.exec-shield kernel.exec-shield = 1 -> still crashes: Opening video decoder: [dmo] DMO video codecs Segmentation fault Note that the mplayer process itself is still in process table (sleeping). Do you have any additional hint or have mplayer developers now some work to do and meanwhile I have to disable exec-shield system-wide?
it's quite likely that they forget to use the PROT_EXEC flag on the mmap that allocates teh memory for the dmo decoder, or don't mprotect() the memory with that flag. Also if they would use dlopen it would already just work...
You can't really use dlopen on win32 dlls :)
then the mplayer dlopen equivalent needs to make sure the permissions for the memory it allocates are set for execute (eg PROT_EXEC)