Bug 492737

Summary: segfault in libpython2.6.so.1.0[7fa932218000+169000]
Product: [Fedora] Fedora Reporter: sangu <sangu.fedora>
Component: pygobject2Assignee: Matthew Barnes <mbarnes>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: low    
Version: rawhideCC: ackistler, chemobejk, dcantrell, dwalsh, fedora, hez, ivazqueznet, james.antill, jdennis, jonathansteffan, katzj, lkundrak, madko, mbarnes, mclasen, mgrepl, pertusus, rdieter
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2009-05-11 16:59:30 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 446452, 517000    

Description sangu 2009-03-29 01:38:21 UTC
Description of problem:
$ dmesg
[skip]
setroubleshootd[8023]: segfault at 7 ip 00007fa9322b1884 sp 00007fff3a7df860 error 4 in libpython2.6.so.1.0[7fa932218000+169000]
sealert[8084] general protection ip:7f191cd9090d sp:7fff252c2f60 error:0 in libpython2.6.so.1.0[7f191ccf7000+169000]

# tail -f /var/log/audit/audit.log
[skip]
type=ANOM_ABEND msg=audit(1238289476.348:27288): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 pid=8023 comm="setroubleshootd" sig=11
type=ANOM_ABEND msg=audit(1238289758.776:27289): auid=500 uid=500 gid=500 ses=2 subj=unconfined_u:unconfined_r:unconfined_t:s0 pid=8084 comm="sealert" sig=11

Version-Release number of selected component (if applicable):
2.1.6-2.fc11.x86_64

How reproducible:
always

Steps to Reproduce:
1. launch sealert
2.
3.
  
Actual results:


Expected results:


Additional info:
python-2.6-7.fc11.x86_64
audit-1.7.12-3.fc11.x86_64

Comment 1 Daniel Walsh 2009-03-30 14:18:49 UTC
setroubleshoot should not be able to trigger a segfault in a python library.

Comment 2 Stefan Becker 2009-04-02 15:17:30 UTC
Confirmed, setroubleshootd generates a lot of messages and keeps the CPUs busy until the parent gives up starting it:

with enforcing:

Apr  2 18:14:29 localhost setroubleshoot: [program.ERROR] setroubleshoot generat
ed AVC, exiting to avoid recursion, context=system_u:system_r:setroubleshootd_t:
s0-s0:c0.c1023, AVC scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023
Apr  2 18:14:29 localhost setroubleshoot: [program.ERROR] audit event#012node=ba
raddur type=AVC msg=audit(1238685206.215:573): avc:  denied  { execstack } for
pid=4464 comm="setroubleshootd" scontext=system_u:system_r:setroubleshootd_t:s0-
s0:c0.c1023 tcontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tclass=p
rocess#012#012node=baraddur type=SYSCALL msg=audit(1238685206.215:573): arch=c00
0003e syscall=10 success=no exit=-1491468328 a0=7fff1648a000 a1=1000 a2=1000007
a3=7fd70e286781 items=0 ppid=4463 pid=4464 auid=4294967295 uid=0 gid=0 euid=0 su
id=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="setroubleshoo
td" exe="/usr/bin/python" subj=system_u:system_r:setroubleshootd_t:s0-s0:c0.c102
3 key=(null)

with permissive

Apr  2 18:10:51 localhost kernel: setroubleshootd[4214] general protection ip:3a
ea299884 sp:7fff329579d0 error:0 in libpython2.6.so.1.0[3aea200000+169000]

selinux-policy-targeted-3.6.10-5.fc11.noarch
policycoreutils-2.0.62-7.fc11.x86_64
audit-1.7.12-3.fc11.x86_64
python-2.6-7.fc11.x86_64

I think the crash in Python makes selinuxtroubleshootd starting up and the parent keeps on spawning it.

Comment 3 James Antill 2009-04-02 15:35:55 UTC
setroubleshootd isn't just python code, is it?

In all cases where I see stuff kill python (or die in libpython), the code that is causing the problem is C code external to the python package.

Comment 4 Alex Hudson (Fedora Address) 2009-04-02 16:18:56 UTC
I'm seeing this bug too. gdb tells me:

Program received signal SIGSEGV, Segmentation fault.
0x000000370129990d in PyType_IsSubtype () from /usr/lib64/libpython2.6.so.1.0

Backtrace:

(gdb) bt
#0  0x000000370129990d in PyType_IsSubtype ()
   from /usr/lib64/libpython2.6.so.1.0
#1  0x00007ffff0e6ede6 in ?? ()
   from /usr/lib64/python2.6/site-packages/gtk-2.0/gobject/_gobject.so
#2  0x00007ffff0e6f0ed in ?? ()
   from /usr/lib64/python2.6/site-packages/gtk-2.0/gobject/_gobject.so
#3  0x00007ffff0e6fef8 in ?? ()
   from /usr/lib64/python2.6/site-packages/gtk-2.0/gobject/_gobject.so
#4  0x00000037012df473 in PyEval_EvalFrameEx ()
   from /usr/lib64/libpython2.6.so.1.0
#5  0x00000037012e0805 in PyEval_EvalFrameEx ()
   from /usr/lib64/libpython2.6.so.1.0
#6  0x00000037012e1123 in PyEval_EvalCodeEx ()
   from /usr/lib64/libpython2.6.so.1.0
#7  0x000000370126e67f in ?? () from /usr/lib64/libpython2.6.so.1.0
#8  0x0000003701243ce3 in PyObject_Call () from /usr/lib64/libpython2.6.so.1.0
#9  0x000000370125903f in ?? () from /usr/lib64/libpython2.6.so.1.0
#10 0x0000003701243ce3 in PyObject_Call () from /usr/lib64/libpython2.6.so.1.0
#11 0x000000370129c78e in ?? () from /usr/lib64/libpython2.6.so.1.0
#12 0x000000370129b1de in ?? () from /usr/lib64/libpython2.6.so.1.0
#13 0x0000003701243ce3 in PyObject_Call () from /usr/lib64/libpython2.6.so.1.0
#14 0x0000003701243fb8 in PyObject_CallFunctionObjArgs ()
   from /usr/lib64/libpython2.6.so.1.0
#15 0x00000037012dcceb in PyEval_EvalFrameEx ()
   from /usr/lib64/libpython2.6.so.1.0
#16 0x00000037012e1123 in PyEval_EvalCodeEx ()
   from /usr/lib64/libpython2.6.so.1.0
#17 0x00000037012e1232 in PyEval_EvalCode ()
   from /usr/lib64/libpython2.6.so.1.0
#18 0x00000037012f15e4 in PyImport_ExecCodeModuleEx ()
   from /usr/lib64/libpython2.6.so.1.0
#19 0x00000037012f1a08 in ?? () from /usr/lib64/libpython2.6.so.1.0
#20 0x00000037012f2f75 in ?? () from /usr/lib64/libpython2.6.so.1.0
#21 0x00000037012f3204 in ?? () from /usr/lib64/libpython2.6.so.1.0
#22 0x00000037012f38be in ?? () from /usr/lib64/libpython2.6.so.1.0
#23 0x00000037012f3e04 in PyImport_ImportModuleLevel ()
   from /usr/lib64/libpython2.6.so.1.0
#24 0x00000037012d90af in ?? () from /usr/lib64/libpython2.6.so.1.0
#25 0x0000003701243ce3 in PyObject_Call () from /usr/lib64/libpython2.6.so.1.0
#26 0x00000037012d95f3 in PyEval_CallObjectWithKeywords ()
   from /usr/lib64/libpython2.6.so.1.0
#27 0x00000037012dc276 in PyEval_EvalFrameEx ()
   from /usr/lib64/libpython2.6.so.1.0
#28 0x00000037012e1123 in PyEval_EvalCodeEx ()
   from /usr/lib64/libpython2.6.so.1.0
#29 0x00000037012e1232 in PyEval_EvalCode ()
   from /usr/lib64/libpython2.6.so.1.0
#30 0x00000037012fc8dc in ?? () from /usr/lib64/libpython2.6.so.1.0
#31 0x00000037012fc9b0 in PyRun_FileExFlags ()
   from /usr/lib64/libpython2.6.so.1.0
#32 0x00000037012fde0e in PyRun_SimpleFileExFlags ()
   from /usr/lib64/libpython2.6.so.1.0
#33 0x000000370130a8c9 in Py_Main () from /usr/lib64/libpython2.6.so.1.0
#34 0x00007ffff71e86cd in __libc_start_main () from /lib64/libc.so.6
#35 0x0000000000400649 in _start ()

Looks to me like gobject is passing something bad through to libpython, causing the segfault.

Comment 5 Daniel Walsh 2009-04-02 17:39:02 UTC
Fixed in libselinux-2.0.79-5.fc11

Please update from Koji and make sure this fixes the problem.  (It has on my machine).

Comment 6 Stefan Becker 2009-04-02 18:19:31 UTC
I think there is still something missing. I updated the packages from koji:

libselinux-2.0.79-5.fc11.x86_64
libselinux-utils-2.0.79-5.fc11.x86_64
libselinux-python-2.0.79-5.fc11.x86_64

and after a "service auditd restart" the problem disappeared.


But after the next reboot it is back again until I restart auditd. I already did a "new-kernel-pkg --mkinitrd --update $(uname -r)". I verified that libselinux.so.1 has the same sha1sum in /lib64 and the initrd.

Comment 7 Alex Hudson (Fedora Address) 2009-04-02 18:29:29 UTC
I still have problems after updating from Koji too; no crash any more but I do get an assertion now:

# sealert 
python: Objects/typeobject.c:1144: PyType_IsSubtype: Assertion `((((((PyObject*)(mro))->ob_type))->tp_flags & ((1L<<26))) != 0)' failed.
Aborted

Comment 8 Stefan Becker 2009-04-02 19:29:00 UTC
As normal user I get the same Python assertion as in comment #7

As root I get a segfault:

# sealert
Segmentation fault

Apr  2 22:09:38 localhost kernel: sealert[3696]: segfault at 7 ip 0000003aea299884 sp 00007fff95e04350 error 4 in libpython2.6.so.1.0[3aea200000+169000]

(gdb) bt                                                                        
#0  PyType_IsSubtype (a=0x2501ef0, b=0x7f0908504aa0)                            
    at Objects/typeobject.c:1144                                                
#1  0x00007f09082f0de6 in ?? ()                                                 
   from /usr/lib64/python2.6/site-packages/gtk-2.0/gobject/_gobject.so          
#2  0x00007f09082f10ed in ?? ()                                                 
   from /usr/lib64/python2.6/site-packages/gtk-2.0/gobject/_gobject.so          
#3  0x00007f09082f1ef8 in ?? ()                                                 
   from /usr/lib64/python2.6/site-packages/gtk-2.0/gobject/_gobject.so          
#4  0x0000003aea2df473 in call_function (oparg=<value optimized out>,           
    pp_stack=<value optimized out>) at Python/ceval.c:3679                      
#5  PyEval_EvalFrameEx (oparg=<value optimized out>,                            
    pp_stack=<value optimized out>) at Python/ceval.c:2370                      
#6  0x0000003aea2e0805 in fast_function (nk=<value optimized out>,              
    na=<value optimized out>, n=<value optimized out>,                          
    pp_stack=<value optimized out>, func=<value optimized out>)                 
    at Python/ceval.c:3765                                                      
#7  call_function (nk=<value optimized out>, na=<value optimized out>,          
    n=<value optimized out>, pp_stack=<value optimized out>,                    
    func=<value optimized out>) at Python/ceval.c:3700                          
#8  PyEval_EvalFrameEx (nk=<value optimized out>, na=<value optimized out>,     
    n=<value optimized out>, pp_stack=<value optimized out>,                    
    func=<value optimized out>) at Python/ceval.c:2370                          
#9  0x0000003aea2e1123 in PyEval_EvalCodeEx (co=0x1d7a6c0,                      
    globals=<value optimized out>, locals=<value optimized out>, args=0x0,      
    argcount=<value optimized out>, kws=0x4, kwcount=0, defs=0x0, defcount=0,   
    closure=0x0) at Python/ceval.c:2942                                         
#10 0x0000003aea26e67f in function_call (func=0x1e21b18, arg=0x20aad08, kw=0x0) 
    at Objects/funcobject.c:524                                                 
#11 0x0000003aea243ce3 in PyObject_Call (func=0x1e21b18, arg=0x7f0908504aa0,    
    kw=0xffffffffffffffff) at Objects/abstract.c:2487                           
#12 0x0000003aea25903f in instancemethod_call (func=0x1e21b18, arg=0x20aad08,   
    kw=0x0) at Objects/classobject.c:2579                                       
#13 0x0000003aea243ce3 in PyObject_Call (func=0x1f12460, arg=0x7f0908504aa0,    
    kw=0xffffffffffffffff) at Objects/abstract.c:2487                           
#14 0x0000003aea29c78e in slot_tp_init (self=<value optimized out>,             
    args=0x2503fa0, kwds=0x0) at Objects/typeobject.c:5595                      
#15 0x0000003aea29b1de in type_call (type=<value optimized out>,                
    args=0x2503fa0, kwds=0x0) at Objects/typeobject.c:747                       
#16 0x0000003aea243ce3 in PyObject_Call (func=0x3aea584740,                     
    arg=0x7f0908504aa0, kw=0xffffffffffffffff) at Objects/abstract.c:2487       
#17 0x0000003aea243fb8 in PyObject_CallFunctionObjArgs (callable=0x3aea584740)  
    at Objects/abstract.c:2718                                                  
#18 0x0000003aea2dcceb in build_class (name=<value optimized out>,              
    bases=<value optimized out>, methods=<value optimized out>)                 
    at Python/ceval.c:4276                                                      
#19 PyEval_EvalFrameEx (name=<value optimized out>,                             
    bases=<value optimized out>, methods=<value optimized out>)
    at Python/ceval.c:1754
#20 0x0000003aea2e1123 in PyEval_EvalCodeEx (co=0x1d7a648,
    globals=<value optimized out>, locals=<value optimized out>, args=0x0,
    argcount=<value optimized out>, kws=0x0, kwcount=0, defs=0x0, defcount=0,
    closure=0x0) at Python/ceval.c:2942
#21 0x0000003aea2e1232 in PyEval_EvalCode (co=0x2501ef0,
    globals=0x7f0908504aa0, locals=0xffffffffffffffff) at Python/ceval.c:515
#22 0x0000003aea2fc8dc in run_mod (mod=<value optimized out>,
    filename=<value optimized out>, globals=0x1d15560, locals=0x1d15560,
    flags=<value optimized out>, arena=<value optimized out>)
    at Python/pythonrun.c:1330
#23 0x0000003aea2fc9b0 in PyRun_FileExFlags (fp=0x1dacdd0,
    filename=0x7fff173fe8bf "/usr/bin/sealert", start=<value optimized out>,
    globals=<value optimized out>, locals=0x1d15560, closeit=1,
    flags=0x7fff173fd930) at Python/pythonrun.c:1316
#24 0x0000003aea2fde0e in PyRun_SimpleFileExFlags (fp=<value optimized out>,
    filename=0x7fff173fe8bf "/usr/bin/sealert", closeit=1,
    flags=0x7fff173fd930) at Python/pythonrun.c:926
#25 0x0000003aea30a8c9 in Py_Main (argc=255647904, argv=<value optimized out>)
    at Modules/main.c:597
#26 0x00007f090e5e86cd in __libc_start_main () from /lib64/libc.so.6
#27 0x0000000000400649 in _start ()

Comment 9 Stefan Becker 2009-04-03 16:03:56 UTC
Updating to

 libselinux-2.0.79-6.fc11.x86_64
 setroubleshoot-2.1.7-1.fc11.x86_64

fixed the setroubleshoot problem on my machine. Thanks Dan!

sealert still segfaults though :-(

Comment 10 Daniel Walsh 2009-04-03 17:46:12 UTC
I don't have the sealert seg faulting problem.

Could this be a language thing?

LANG=en_US.UTF-8 sealert

Works

LANG=c sealert
Segmentation fault
[Exit 139 (SIGSEGV)]

Comment 11 Stefan Becker 2009-04-04 12:04:12 UTC
Yes, just the other way around for me, i.e. "LANG=C sealert" worked OK.

Just updated to

 setroubleshoot-2.1.8-1.fc11.x86_64
 setroubleshoot-plugins-2.0.15-1.fc11.noarch

and the sealert crash is gone.

Comment 12 James Antill 2009-04-06 13:47:11 UTC
*** Bug 494263 has been marked as a duplicate of this bug. ***

Comment 13 Allen Kistler 2009-04-06 23:49:27 UTC
I still get a segfault in libpython when I launch sealert.

python-2.6-7.fc11.i586
libselinux-2.0.79-6.fc11.i586
setroubleshoot-2.1.8-1.fc11.i586
setroubleshoot-plugins-2.0.15-1.fc11.noarch

Realizing that some of their files are in initrd, I rebuilt initrd and rebooted.
No help.

I also tried setting the LANG as in the comments above.
No help there, either.

Comment 14 Allen Kistler 2009-04-06 23:50:02 UTC
I appear not to have the proper access to reopen this bug.  If no one else can reopen it, I'll reopen and edit Bug 494263.

Comment 15 Daniel Walsh 2009-04-07 12:34:53 UTC
What lang are you using?

Comment 16 Allen Kistler 2009-04-07 19:30:48 UTC
echo $LANG
says en_US.UTF-8

I also tried both LANG=C and LANG=en_US.UTF-8 on the command line, too, since I wasn't sure what got passed to sealert when I ran it from the System-Tools menu.

Comment 17 Lubomir Rintel 2009-04-10 06:31:34 UTC
  File "/usr/sbin/setroubleshootd", line 112, in <module>
    from setroubleshoot.server import RunFaultServer
  File "/usr/lib/python2.6/site-packages/setroubleshoot/server.py", line 45, in <module>
    from setroubleshoot.analyze import (PluginReportReceiver,
  File "/usr/lib/python2.6/site-packages/setroubleshoot/analyze.py", line 463, in <module>
    gobject.GObject):
  File "/usr/lib/python2.6/site-packages/gtk-2.0/gobject/__init__.py", line 67, in __init__
    cls._type_register(cls.__dict__)
  File "/usr/lib/python2.6/site-packages/gtk-2.0/gobject/__init__.py", line 121, in _type_register
    type_register(cls, namespace.get('__gtype_name__'))
  File "<string>", line 1, in <module>

In python code, the comment #7 assertion fail traces to the very same place in code as the segfault. I reported it as a separate bug #495181, before realizing that it is probably a duplicate.

[root@bimbo lkundrak]# LANG=en_US /usr/sbin/setroubleshootd -f
Segmentation fault
[root@bimbo lkundrak]# LANG=en_US.UTF8 /usr/sbin/setroubleshootd -f
python: Objects/typeobject.c:1144: PyType_IsSubtype: Assertion `((((((PyObject*)(mro))->ob_type))->tp_flags & ((1L<<26))) != 0)' failed.
Aborted
[root@bimbo lkundrak]# 

Dan, both of these are reliably reproducible on today's i586 rawhide.

Comment 18 Daniel Walsh 2009-04-13 11:45:26 UTC
I think these are being triggered by some change in gtk, And I don't see how python code should be able to trigger a segfault.  Unless it is libselinux that is causing the problem.  Need someone from gtk to look at it.

Comment 19 Rex Dieter 2009-04-13 12:40:05 UTC
->gtk2

Comment 20 Matthias Clasen 2009-04-13 16:49:24 UTC
I don't see anything from gtk or even gobject in the stacktrace, so not sure what I am expected to do here...

Comment 21 Lubomir Rintel 2009-04-13 18:01:27 UTC
Matthias, see the python traceback in comment #17.
Maybe this needs to be reassigned to pygobject2.

Comment 22 Jesse Keating 2009-04-16 19:20:56 UTC
Adding to the blocker, it seems a lot of people will be seeing this.

Comment 23 Matthew Barnes 2009-04-22 17:23:04 UTC
Crash happens when registering base types for the ServerConnectionHandler class. SETroubleshootDatabaseInterface appears to be the troublesome base class.  Its C struct type is a PyClassObject, but it's getting cast to a PyTypeObject in pyg_type_add_interfaces() (gobjectmodule.c from pygobject).

Comment 24 Matthew Barnes 2009-04-22 18:03:08 UTC
Assuming I'm on the right trail, I traced the faulty code back to:
http://bugzilla.gnome.org/show_bug.cgi?id=566571

Might just need to revert that change for the time being and live with whatever bug it was supposed to fix.

Comment 25 John Dennis 2009-04-22 18:04:58 UTC
I think what might be happening is there is a mix of "classic classes" and "new style classes" in the inheritence chain. The reason I think that is because I seem to recall a classic class derives from a type object and a new style class derives from a class object (or visa versa :-). I'm not sure at the moment what the implication of this might be in the context of multiple inheritance and how gobject wants manage things or whether it's legal or illegal but it's the only thing I can think of off the top of my head given the observation in comment #23 from Matt. HTH, John

Comment 26 John Dennis 2009-04-22 18:10:14 UTC
comment #8 in http://bugzilla.gnome.org/show_bug.cgi?id=566571
 seems to reinforce the observation I made in comment #25. The issues seems to be a mix of classic and new style classes in the inheritance chain, or so I believe.

Comment 27 Matthew Barnes 2009-04-22 21:22:39 UTC
(In reply to comment #25)
> I think what might be happening is there is a mix of "classic classes" and "new
> style classes" in the inheritence chain. The reason I think that is because I
> seem to recall a classic class derives from a type object and a new style class
> derives from a class object (or visa versa :-). I'm not sure at the moment what
> the implication of this might be in the context of multiple inheritance and how
> gobject wants manage things or whether it's legal or illegal but it's the only
> thing I can think of off the top of my head given the observation in comment
> #23 from Matt.

That sounds correct based on my rusty memory of Python semantics.  Classes and types are now one and the same.

The proposed upstream correction now appears to simply skip over classic base classes, which would avoid the crash but probably still break setroubleshootd (or wherever that Python code lives).

Comment 28 Matthew Barnes 2009-04-22 21:39:56 UTC
I packaged the upstream correction as pygobject2-2.16.1-4.fc11.

If that doesn't work I can still back out all changes related to http://bugzilla.gnome.org/show_bug.cgi?id=566571.

Comment 29 Fedora Update System 2009-05-05 23:03:09 UTC
pygobject2-2.16.1-4.fc11 has been submitted as an update for Fedora 11.
http://admin.fedoraproject.org/updates/pygobject2-2.16.1-4.fc11

Comment 30 Allen Kistler 2009-05-06 18:48:27 UTC
I updated to the package in Comment #29.

I generated SELinux AVC violations on purpose.
I can verify that:
1. seapplet works again - a warning showed up in the notification area
2. sealert works again - the SE troubleshooter opens and displays content
3. setroubleshootd works again - I got mail on the denials
4. selinux-polgengui runs again - it opens a window (I didn't look any harder)

Those are all things that were broken before.

Comment 31 Lubomir Rintel 2009-05-07 04:54:00 UTC
(In reply to comment #29)
> pygobject2-2.16.1-4.fc11 has been submitted as an update for Fedora 11.
> http://admin.fedoraproject.org/updates/pygobject2-2.16.1-4.fc11  

Has a freeze break been proposed for this? This is probably serious enough to go into the release.

Comment 32 Daniel Walsh 2009-05-07 14:04:57 UTC
I believe I opened a ticket for this and koji shows it in dist-f11

koji latest-pkg dist-f11 pygobject2
Build                                     Tag                   Built by
----------------------------------------  --------------------  ----------------
pygobject2-2.16.1-4.fc11                  dist-f11              mbarnes

Comment 33 Jesse Keating 2009-05-11 16:59:30 UTC
This is in the release now, closing.