Bug 466069 - tidy: Segmentation fault in tidySetErrorSink
tidy: Segmentation fault in tidySetErrorSink
Status: CLOSED NOTABUG
Product: Fedora EPEL
Classification: Fedora
Component: tidy (Show other bugs)
el5
All Linux
medium Severity medium
: ---
: ---
Assigned To: Rex Dieter
Fedora Extras Quality Assurance
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2008-10-07 22:33 EDT by Jose Pedro Oliveira
Modified: 2008-10-16 13:25 EDT (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2008-10-16 13:25:07 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Jose Pedro Oliveira 2008-10-07 22:33:53 EDT
Description of problem:
Setup: Apache+mod_python+trac+python-tidy

A trac plugin that uses python-tidy causes Apache to segfault when the function
parseString is invoked. The segfault happens in the libtidy tidySetErrorSink function.  This only happens in 64bit systems.  The same trac plugin executes correctly in 32bit systems.


Version-Release number of selected component (if applicable):
libtidy-0.99.0-14.20070615.el5


How reproducible:
In 64bit systems only.  Must be run under apache/mod_python to cause the segmentation fault; the code executes correctly under tracd.


Actual results:
...
Core was generated by `httpd -X -f /etc/httpd/conf/httpd.conf'.
Program terminated with signal 11, Segmentation fault.
#0  tidySetErrorSink (tdoc=0x7e245de0, sink=0x2b7f7e196210) at tidylib.c:732
732             uint outenc = cfg( impl, TidyOutCharEncoding );


Additional info:
Another report of the same problem: http://labs.creativecommons.org/2008/08/02/64-bit-woes-almost-cleared-up/

I have also updated tidy to the latest CVS version to see if the problem had been resolved: wishful thinking, the problem persisted.
Comment 1 Jose Pedro Oliveira 2008-10-08 12:41:51 EDT
Partial stack trace:
---------
....
Core was generated by `httpd -X -f /etc/httpd/conf/httpd.conf'.
Program terminated with signal 11, Segmentation fault.
#0  tidySetErrorSink (tdoc=0x263eb170, sink=0x2b3421bb6c10) at tidylib.c:732
732             uint outenc = cfg( impl, TidyOutCharEncoding );
(gdb) where 20
#0  tidySetErrorSink (tdoc=0x263eb170, sink=0x2b3421bb6c10) at tidylib.c:732
#1  0x00002b34203b76c4 in ffi_call_unix64 () from /usr/lib64/python2.4/site-packages/_ctypes.so
#2  0x00002b34203b7234 in ffi_call () from /usr/lib64/python2.4/site-packages/_ctypes.so
#3  0x00002b34203b2cd1 in _CallProc () from /usr/lib64/python2.4/site-packages/_ctypes.so
#4  0x00002b34203adbe3 in CData_AtAddress () from /usr/lib64/python2.4/site-packages/_ctypes.so
#5  0x00002b3418213fb0 in PyObject_Call (func=0x263eb170, arg=0x2b3421bb6c10, kw=0x0)
    at Objects/abstract.c:1795
#6  0x00002b34182701ee in PyEval_EvalFrame (f=0x2b342632ffb0) at Python/ceval.c:3771
#7  0x00002b3418273905 in PyEval_EvalCodeEx (co=0x2b3420369420, globals=<value optimized out>, 
    locals=<value optimized out>, args=0x2b341db65ca8, argcount=1, kws=0x0, kwcount=0, defs=0x0, 
    defcount=0, closure=0x0) at Python/ceval.c:2736
#8  0x00002b341822a259 in function_call (func=0x2b3420379d70, arg=0x2b341db65c90, kw=0x0)
    at Objects/funcobject.c:548
#9  0x00002b3418213fb0 in PyObject_Call (func=0x263eb170, arg=0x2b3421bb6c10, kw=0x0)
    at Objects/abstract.c:1795
#10 0x00002b341821a03f in instancemethod_call (func=<value optimized out>, arg=0x2b341db65c90, kw=0x0)
    at Objects/classobject.c:2447
#11 0x00002b3418213fb0 in PyObject_Call (func=0x263eb170, arg=0x2b3421bb6c10, kw=0x0)
    at Objects/abstract.c:1795
#12 0x00002b341825153c in slot_tp_init (self=<value optimized out>, args=0x2b341891e050, kwds=0x0)
    at Objects/typeobject.c:4759
#13 0x00002b341824d518 in type_call (type=<value optimized out>, args=0x2b341891e050, kwds=0x0)
    at Objects/typeobject.c:435
#14 0x00002b3418213fb0 in PyObject_Call (func=0x263eb170, arg=0x2b3421bb6c10, kw=0x0)
    at Objects/abstract.c:1795
#15 0x00002b34182701ee in PyEval_EvalFrame (f=0x2b3425dec570) at Python/ceval.c:3771
#16 0x00002b3418273905 in PyEval_EvalCodeEx (co=0x2b3420369880, globals=<value optimized out>, 
    locals=<value optimized out>, args=0x2b34262bfc90, argcount=1, kws=0x2b34262bfc10, kwcount=9, 
    defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:2736
#17 0x00002b341822a1f7 in function_call (func=0x2b34203801b8, arg=0x2b3421d1d2d0, kw=0x2b34263b80e0)
    at Objects/funcobject.c:548
#18 0x00002b3418213fb0 in PyObject_Call (func=0x263eb170, arg=0x2b3421bb6c10, kw=0x0)
    at Objects/abstract.c:1795
#19 0x00002b3418270e4c in PyEval_EvalFrame (f=0x2b34263d0bf0) at Python/ceval.c:3840
(More stack frames follow...)
---------
Comment 2 Jose Pedro Oliveira 2008-10-08 14:13:31 EDT
Removed Trac from que equation.

The following python CGI executes without problems in both 32/64 systems:
----------
#!/usr/bin/python
# -*- coding: utf-8 -*-

import tidy

options = dict(char_encoding='utf8')

print 'Status: 200 OK'
print 'Content-Type: text/plain'
print
print tidy.parseString(u'<p>Kaboom ção'.encode('utf8'),  **options)
----------

But the following mod_python script fails in 64 bit systems (produces the correct solution in 32 bit systems):
----------
# -*- coding: utf-8 -*-

from mod_python import apache
import tidy

def handler(req):
    options = dict(char_encoding='utf8')
    req.content_type = "text/plain"
    req.write(str(tidy.parseString(u'<p>Kaboom ção'.encode('utf8'),  **options)))
    return apache.OK
----------


Gdb output:
----------
...
Reading symbols from /usr/lib64/libtidy.so...Reading symbols from /usr/lib/debug/usr/lib64/libtidy-0.99.so.0.0.0.debug...done.
done.
Loaded symbols for /usr/lib64/libtidy.so
Core was generated by `httpd -X -f /etc/httpd/conf/httpd.conf'.
Program terminated with signal 11, Segmentation fault.
#0  tidySetErrorSink (tdoc=0x142661e0, sink=0x2ad013315e10) at tidylib.c:732
732             uint outenc = cfg( impl, TidyOutCharEncoding );
(gdb)
----------
Comment 3 Rex Dieter 2008-10-08 14:25:52 EDT
Boy, haven't looked at this one in awhile.  Looks like upstream "doesn't do releases anymore", yay.

anyone, feel free to jump in and help in any way you can.

Soonish, I'll see about whipping up a new snapshot.
Comment 4 Rex Dieter 2008-10-08 14:28:12 EDT
Oh, missed (from comment #1):
> I have also updated tidy to the latest CVS version to see if the problem had
> been resolved: wishful thinking, the problem persisted.

boo.  In the meantime, looks like reporting upstream is one good way forward.  Please do so.
Comment 5 Jose Pedro Oliveira 2008-10-08 20:18:13 EDT
(In reply to comment #4)
> Oh, missed (from comment #1):
> > I have also updated tidy to the latest CVS version to see if the problem had
> > been resolved: wishful thinking, the problem persisted.
> 
> boo.  In the meantime, looks like reporting upstream is one good way forward. 
> Please do so.

Done.

 * [ tidy-Bugs-2154022 ] Segmentation fault in tidySetErrorSink (64 bit)
   https://sourceforge.net/tracker/?func=detail&atid=390963&aid=2154022&group_id=27659
Comment 6 Jose Pedro Oliveira 2008-10-09 10:06:43 EDT
Upstream already made a couple of good debug suggestions in the sourceforge ticket (he thinks the problem is in the python bindings - python-tidy package).

I will try to look into it again (most likely during the weekend).
Comment 7 Jose Pedro Oliveira 2008-10-16 10:45:49 EDT
Hi,

The problem resides in the python bindings: basically they aren't 64bit safe. As everything defaults to ctypes.c_int, this truncates everything to 32 bits in 64 bit systems (even ctypes.c_void_p is only 4 bytes in 64bit systems).

This one line patch to uTidyLib fixed my problem:
----------
diff -ruN uTidylib-0.2-orig/tidy/lib.py uTidylib-0.2/tidy/lib.py
--- uTidylib-0.2-orig/tidy/lib.py       2004-02-24 08:12:24.000000000 +0000
+++ uTidylib-0.2/tidy/lib.py    2008-10-16 14:45:56.000000000 +0100
@@ -130,6 +130,8 @@
 
 sinkfactory=SinkFactory()
 
+_tidy.Create.restype = ctypes.POINTER(ctypes.c_void_p)
+
 class _Document(object):
     def __init__(self):
         self.cdoc = _tidy.Create()
----------

Note1: more information in the sourceforge ticket mentioned in comment #5.
Note2: Platforms used for testing: CentOS 5.2 x86_64 (python2.4), Centos 5.2 i386, Fedora 9 x86_64 (python2.5).
Comment 8 Jose Pedro Oliveira 2008-10-16 11:04:12 EDT
(In reply to comment #3)
> ...
> Soonish, I'll see about whipping up a new snapshot.

I've been using the tidy 20081006 snapshot in one of my 64bit systems: no problems detected so far.
Comment 9 Jose Pedro Oliveira 2008-10-16 11:12:47 EDT
 * Python-tidy bug report (against Fedora 9 x86_64):
   https://bugzilla.redhat.com/show_bug.cgi?id=467246
Comment 10 Rex Dieter 2008-10-16 11:29:42 EDT
Thanks for all your efforts... so strictly speaking there is no "tidy" bug here?  If so, can we close this?

That won't stop me from updating tidy... eventually.  If you don't want to wait on me, feel free to do so in cvs (acls' should be open).
Comment 11 Jose Pedro Oliveira 2008-10-16 13:25:07 EDT
Closing this ticket as this problem was caused by the python bindings in utidylib
(they aren't 64bit safe). 

Sorry for the noise,
jpo

Note You need to log in before you can comment on or make changes to this bug.