Bug 1118504

Summary: executable stack violation in librados.so.2
Product: [Fedora] Fedora Reporter: Adam Williamson <awilliam>
Component: cephAssignee: Boris Ranto <branto>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: rawhideCC: crobinso, david, decathorpe, dmick, elad, fedora, jdulaney, pschindl, rjones, robatino, steve, vbenes
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-08-23 10:22:15 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 910269, 1043124    

Description Adam Williamson 2014-07-10 21:42:16 UTC
Trying to launch a virtual machine from virt-manager in current Rawhide, I get this:

libvirtError: internal error: process exited while connecting to monitor: /usr/bin/qemu-system-x86_64: error while loading shared libraries: librados.so.2: cannot enable executable stack as shared object requires: Permission denied

if I run with setenforce Permissive, it works. AIUI, this is a Fedora policy violation on the part of the library: Fedora libraries are not supposed to require an executable stack.

Nominating as a Beta blocker, as it violates https://fedoraproject.org/wiki/Fedora_21_Beta_Release_Criteria#Self_hosting_virtualization - "The release must be able host virtual guest instances of the same release." - in the case of SELinux being enabled (which is the default configuration).

Comment 1 Adam Williamson 2014-07-10 22:25:13 UTC
Note: I have librados2-0.81.0-4.fc21.x86_64 installed, i.e. one of the builds with lots of radical changes compared to -2 that were reverted in -5. But the -5 reversion seems to be planned to be temporary at least as regards F22, so this bug is likely to persist when those changes are re-applied.

Comment 2 Adam Williamson 2014-07-10 22:40:02 UTC
Just confirmed that -5 doesn't have this bug. So it's inconvenient for people who got the -3 or -4 packages due to https://bugzilla.redhat.com/show_bug.cgi?id=1118510 , but not a Beta blocker because it won't affect fresh F21 installs, assuming the -3/-4 changes are not applied to F21 again later. I'll leave it open in the assumption that you're intending to apply the -3/-4 changes to Rawhide (F22) again; this bug should be fixed when that happens.

Comment 3 Andre Robatino 2014-07-11 01:53:40 UTC
Although I can't find now which day it was, the ceph package was removed from Rawhide recently, so there's no longer a librados2 binary package (I use "yum list extras" to identify packages that are no longer in the repo). I removed the binary package on July 9, so ceph must have been removed a day or two before that, though I can't find it in any Rawhide report.

Comment 4 Adam Williamson 2014-07-11 02:55:14 UTC
andre: see https://bugzilla.redhat.com/show_bug.cgi?id=1118510 .

Comment 5 Cole Robinson 2014-08-08 15:04:28 UTC
*** Bug 1122283 has been marked as a duplicate of this bug. ***

Comment 6 Petr Schindler 2014-08-12 08:49:29 UTC
Description of problem:
I run newly created virtual machine with virt-manager.

Version-Release number of selected component:
selinux-policy-3.13.1-71.fc21.noarch

Additional info:
reporter:       libreport-2.2.3
hashmarkername: setroubleshoot
kernel:         3.16.0-1.fc21.x86_64
type:           libreport

Comment 7 Boris Ranto 2014-08-17 17:01:50 UTC
Hi Adam,

could you please retest this with the latest package in the latest rawhide (or f21)? (that is with 1:ceph-0.80.5-6.fc21/fc22 package)

-Boris

Comment 8 Elad Alfassa 2014-08-19 21:01:47 UTC
With these ceph related packages installed the issue is still there.

libcephfs1-0.80.5-6.fc21.x86_64
ceph-libs-compat-0.80.5-6.fc21.x86_64

Comment 9 Adam Williamson 2014-08-20 15:05:20 UTC
After all the messing about with ceph lately, I have these packages:

ceph-libs-compat-0.80.5-6.fc21.x86_64
libcephfs1-0.80.5-6.fc21.x86_64
librados2-0.80.5-6.fc21.x86_64

and the bug still exists.

Comment 10 Adam Williamson 2014-08-20 15:08:10 UTC
this is now rather worse as the 'live' version of the package in F21 suffers from the bug. Once again proposing as a Beta blocker per criterion https://fedoraproject.org/wiki/Fedora_21_Beta_Release_Criteria#Self_hosting_virtualization - "The release must be able host virtual guest instances of the same release." - in the case of SELinux being enabled (which is the default configuration).

Comment 11 Boris Ranto 2014-08-20 15:36:12 UTC
Hm, I'm looking at this, currently I've got a scratch build scheduled that will hopefully fix this.

In the meantime, could you try doing

execstack -c /usr/lib64/librados.so.2

and see if that fixes your problem? (this is not the way I want to fix it but I'm looking for a backup solution if the build won't fix this)

Comment 12 Fabio Valentini 2014-08-21 15:30:48 UTC
Description of problem:
Start a VM in GNOME boxes, this AVD denial pops up.

Version-Release number of selected component:
selinux-policy-3.13.1-73.fc21.noarch

Additional info:
reporter:       libreport-2.2.3
hashmarkername: setroubleshoot
kernel:         3.16.1-300.fc21.x86_64
type:           libreport

Comment 13 Boris Ranto 2014-08-21 15:52:30 UTC
I've scheduled the builds that should fix this (based on my scratch build), should be in f22/f21, soon:

http://koji.fedoraproject.org/koji/taskinfo?taskID=7435648
http://koji.fedoraproject.org/koji/taskinfo?taskID=7435653

Please re-test once the packages are built.

Comment 14 Cole Robinson 2014-08-21 16:32:51 UTC
*** Bug 1132467 has been marked as a duplicate of this bug. ***

Comment 15 Cole Robinson 2014-08-21 16:33:05 UTC
*** Bug 1131651 has been marked as a duplicate of this bug. ***

Comment 16 Boris Ranto 2014-08-21 16:35:06 UTC
Discard my previous comment, the build won't fix the issue.

Comment 17 Mairi Dulaney 2014-08-21 17:16:50 UTC
CCing myself as I was hitting the same bug (mine was closed as a dupe)

Comment 18 Boris Ranto 2014-08-22 12:54:40 UTC
The latest package (1:ceph-0.80.5-8) contains a fix for this bug (at least librados no longer has the execmem flag). Could anyone hitting this please retest with the package?

Comment 19 Richard W.M. Jones 2014-08-22 17:21:50 UTC
librados2-0.80.5-6.fc22.x86_64 does not fix this.

Since this breaks all qemu guests, would be great to fix this, and soon.

Comment 20 Adam Williamson 2014-08-22 18:58:23 UTC
rwmj: he asked about -8, not -6.

Comment 21 Adam Williamson 2014-08-22 19:01:49 UTC
Seems fixed with -8, at least I can launch a VM from virt-manager with enforcing enabled. Can anyone else confirm?

Comment 22 Richard W.M. Jones 2014-08-22 19:19:37 UTC
Yes the -8 package downloaded from Koji fixes it for me.

Seems as if this bug can now be CLOSED -> RAWHIDE?

Comment 23 Mairi Dulaney 2014-08-22 19:35:53 UTC
I can confirm, fixed in F21 with -8.  Other problems I had with virt-manager seem to have dissapeared.

Comment 24 Boris Ranto 2014-08-23 10:22:15 UTC
Thanks for testing, closing.

Comment 25 Dan Mick 2014-11-15 00:01:35 UTC
Is there, like, any information on what the actual cause or resolution to this issue was?

Comment 26 Dan Mick 2014-11-15 00:06:36 UTC
(apparently for f21 it was "remove yasm", which is not acceptable; the code paths that need the assembler really really need it for speed.)

Comment 27 Dan Mick 2014-11-15 01:20:39 UTC
It appears as though annotating the assembly source will allow the linker to put the correct program header on the binary to avoid this issue.  Ubuntu suggests, for yasm:

section .note.GNU-stack noalloc noexec nowrite progbits

trying that now.

Comment 28 Boris Ranto 2014-11-18 12:55:33 UTC
Yep, in fedora, the "solution" was to remove yasm as build dependency altogether. There was no time to investigate the proper solution as for fedora, this was a release blocker (you need to be able to run the release of fedora in VM and this made VMs crash).

Please, keep me posted on whether that particular solution (adding GNU-stack section) fixed the issue for you.

Comment 29 Dan Mick 2014-11-18 19:29:43 UTC
Reiterating what I tried to send in email:

The fix appears good and has been merged upstream; see http://tracker.ceph.com/issues/10114 and https://github.com/ceph/ceph/commit/5c0562610b059c9c1e2ab16c994749eba07f18aa.  We'll be backporting it to firefly and giant branches (at least).