Description of problem: Setup: Apache+mod_python+trac+python-tidy A trac plugin that uses python-tidy causes Apache to segfault when the function parseString is invoked. The segfault happens in the libtidy tidySetErrorSink function. This only happens in 64bit systems. The same trac plugin executes correctly in 32bit systems. Version-Release number of selected component (if applicable): libtidy-0.99.0-14.20070615.el5 How reproducible: In 64bit systems only. Must be run under apache/mod_python to cause the segmentation fault; the code executes correctly under tracd. Actual results: ... Core was generated by `httpd -X -f /etc/httpd/conf/httpd.conf'. Program terminated with signal 11, Segmentation fault. #0 tidySetErrorSink (tdoc=0x7e245de0, sink=0x2b7f7e196210) at tidylib.c:732 732 uint outenc = cfg( impl, TidyOutCharEncoding ); Additional info: Another report of the same problem: http://labs.creativecommons.org/2008/08/02/64-bit-woes-almost-cleared-up/ I have also updated tidy to the latest CVS version to see if the problem had been resolved: wishful thinking, the problem persisted.
Partial stack trace: --------- .... Core was generated by `httpd -X -f /etc/httpd/conf/httpd.conf'. Program terminated with signal 11, Segmentation fault. #0 tidySetErrorSink (tdoc=0x263eb170, sink=0x2b3421bb6c10) at tidylib.c:732 732 uint outenc = cfg( impl, TidyOutCharEncoding ); (gdb) where 20 #0 tidySetErrorSink (tdoc=0x263eb170, sink=0x2b3421bb6c10) at tidylib.c:732 #1 0x00002b34203b76c4 in ffi_call_unix64 () from /usr/lib64/python2.4/site-packages/_ctypes.so #2 0x00002b34203b7234 in ffi_call () from /usr/lib64/python2.4/site-packages/_ctypes.so #3 0x00002b34203b2cd1 in _CallProc () from /usr/lib64/python2.4/site-packages/_ctypes.so #4 0x00002b34203adbe3 in CData_AtAddress () from /usr/lib64/python2.4/site-packages/_ctypes.so #5 0x00002b3418213fb0 in PyObject_Call (func=0x263eb170, arg=0x2b3421bb6c10, kw=0x0) at Objects/abstract.c:1795 #6 0x00002b34182701ee in PyEval_EvalFrame (f=0x2b342632ffb0) at Python/ceval.c:3771 #7 0x00002b3418273905 in PyEval_EvalCodeEx (co=0x2b3420369420, globals=<value optimized out>, locals=<value optimized out>, args=0x2b341db65ca8, argcount=1, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:2736 #8 0x00002b341822a259 in function_call (func=0x2b3420379d70, arg=0x2b341db65c90, kw=0x0) at Objects/funcobject.c:548 #9 0x00002b3418213fb0 in PyObject_Call (func=0x263eb170, arg=0x2b3421bb6c10, kw=0x0) at Objects/abstract.c:1795 #10 0x00002b341821a03f in instancemethod_call (func=<value optimized out>, arg=0x2b341db65c90, kw=0x0) at Objects/classobject.c:2447 #11 0x00002b3418213fb0 in PyObject_Call (func=0x263eb170, arg=0x2b3421bb6c10, kw=0x0) at Objects/abstract.c:1795 #12 0x00002b341825153c in slot_tp_init (self=<value optimized out>, args=0x2b341891e050, kwds=0x0) at Objects/typeobject.c:4759 #13 0x00002b341824d518 in type_call (type=<value optimized out>, args=0x2b341891e050, kwds=0x0) at Objects/typeobject.c:435 #14 0x00002b3418213fb0 in PyObject_Call (func=0x263eb170, arg=0x2b3421bb6c10, kw=0x0) at Objects/abstract.c:1795 #15 0x00002b34182701ee in PyEval_EvalFrame (f=0x2b3425dec570) at Python/ceval.c:3771 #16 0x00002b3418273905 in PyEval_EvalCodeEx (co=0x2b3420369880, globals=<value optimized out>, locals=<value optimized out>, args=0x2b34262bfc90, argcount=1, kws=0x2b34262bfc10, kwcount=9, defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:2736 #17 0x00002b341822a1f7 in function_call (func=0x2b34203801b8, arg=0x2b3421d1d2d0, kw=0x2b34263b80e0) at Objects/funcobject.c:548 #18 0x00002b3418213fb0 in PyObject_Call (func=0x263eb170, arg=0x2b3421bb6c10, kw=0x0) at Objects/abstract.c:1795 #19 0x00002b3418270e4c in PyEval_EvalFrame (f=0x2b34263d0bf0) at Python/ceval.c:3840 (More stack frames follow...) ---------
Removed Trac from que equation. The following python CGI executes without problems in both 32/64 systems: ---------- #!/usr/bin/python # -*- coding: utf-8 -*- import tidy options = dict(char_encoding='utf8') print 'Status: 200 OK' print 'Content-Type: text/plain' print print tidy.parseString(u'<p>Kaboom ção'.encode('utf8'), **options) ---------- But the following mod_python script fails in 64 bit systems (produces the correct solution in 32 bit systems): ---------- # -*- coding: utf-8 -*- from mod_python import apache import tidy def handler(req): options = dict(char_encoding='utf8') req.content_type = "text/plain" req.write(str(tidy.parseString(u'<p>Kaboom ção'.encode('utf8'), **options))) return apache.OK ---------- Gdb output: ---------- ... Reading symbols from /usr/lib64/libtidy.so...Reading symbols from /usr/lib/debug/usr/lib64/libtidy-0.99.so.0.0.0.debug...done. done. Loaded symbols for /usr/lib64/libtidy.so Core was generated by `httpd -X -f /etc/httpd/conf/httpd.conf'. Program terminated with signal 11, Segmentation fault. #0 tidySetErrorSink (tdoc=0x142661e0, sink=0x2ad013315e10) at tidylib.c:732 732 uint outenc = cfg( impl, TidyOutCharEncoding ); (gdb) ----------
Boy, haven't looked at this one in awhile. Looks like upstream "doesn't do releases anymore", yay. anyone, feel free to jump in and help in any way you can. Soonish, I'll see about whipping up a new snapshot.
Oh, missed (from comment #1): > I have also updated tidy to the latest CVS version to see if the problem had > been resolved: wishful thinking, the problem persisted. boo. In the meantime, looks like reporting upstream is one good way forward. Please do so.
(In reply to comment #4) > Oh, missed (from comment #1): > > I have also updated tidy to the latest CVS version to see if the problem had > > been resolved: wishful thinking, the problem persisted. > > boo. In the meantime, looks like reporting upstream is one good way forward. > Please do so. Done. * [ tidy-Bugs-2154022 ] Segmentation fault in tidySetErrorSink (64 bit) https://sourceforge.net/tracker/?func=detail&atid=390963&aid=2154022&group_id=27659
Upstream already made a couple of good debug suggestions in the sourceforge ticket (he thinks the problem is in the python bindings - python-tidy package). I will try to look into it again (most likely during the weekend).
Hi, The problem resides in the python bindings: basically they aren't 64bit safe. As everything defaults to ctypes.c_int, this truncates everything to 32 bits in 64 bit systems (even ctypes.c_void_p is only 4 bytes in 64bit systems). This one line patch to uTidyLib fixed my problem: ---------- diff -ruN uTidylib-0.2-orig/tidy/lib.py uTidylib-0.2/tidy/lib.py --- uTidylib-0.2-orig/tidy/lib.py 2004-02-24 08:12:24.000000000 +0000 +++ uTidylib-0.2/tidy/lib.py 2008-10-16 14:45:56.000000000 +0100 @@ -130,6 +130,8 @@ sinkfactory=SinkFactory() +_tidy.Create.restype = ctypes.POINTER(ctypes.c_void_p) + class _Document(object): def __init__(self): self.cdoc = _tidy.Create() ---------- Note1: more information in the sourceforge ticket mentioned in comment #5. Note2: Platforms used for testing: CentOS 5.2 x86_64 (python2.4), Centos 5.2 i386, Fedora 9 x86_64 (python2.5).
(In reply to comment #3) > ... > Soonish, I'll see about whipping up a new snapshot. I've been using the tidy 20081006 snapshot in one of my 64bit systems: no problems detected so far.
* Python-tidy bug report (against Fedora 9 x86_64): https://bugzilla.redhat.com/show_bug.cgi?id=467246
Thanks for all your efforts... so strictly speaking there is no "tidy" bug here? If so, can we close this? That won't stop me from updating tidy... eventually. If you don't want to wait on me, feel free to do so in cvs (acls' should be open).
Closing this ticket as this problem was caused by the python bindings in utidylib (they aren't 64bit safe). Sorry for the noise, jpo