Bug 739593

Summary: Python plugin doesn't work
Product: [Fedora] Fedora Reporter: Jonathan Barber <jonathan.barber>
Component: collectdAssignee: Alan Pevec <apevec>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: unspecified    
Version: rawhideCC: apevec, apevec, gjansen, kevin, lkundrak, mail, virt-maint, zxvdr.au
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: collectd-4.10.4-1.fc16 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-12-22 22:25:42 EST Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
Attachments:
Description Flags
collectd configuration
none
Plugin that causes the problem none

Description Jonathan Barber 2011-09-19 11:28:25 EDT
Description of problem:
Collectd doesn't work with the pymongo module, it is unable to find the symbol _Py_ZeroStruct.

Configure collectd with the following /etc/collectd.conf configuration:
Hostname    "test"
FQDNLookup   false
BaseDir     "/tmp/foo"

<LoadPlugin python>
    Globals true
</LoadPlugin>

<Plugin python>
    ModulePath "/tmp/plugins"

    Import "mongodb"
    <Module mongodb>
        Host "127.0.0.1"
    </Module>
</Plugin>

The /tmp/plugins/mongodb.py plugin is:
import collectd
from pymongo import Connection

class MongoDB(object):

    def __init__(self):
        self.plugin_name = "mongo"

mongodb = MongoDB()

Then run collected:
collectd -C collectd.conf -P /tmp/collectd.pid

Gives the error:
Unhandled python exception in importing module: ImportError: /usr/lib64/python2.4/lib-dynload/datetime.so: undefined symbol: _Py_ZeroStruct

and doesn't run the mongodb plugin.

Version-Release number of selected component (if applicable):
rpm -qa pymongo python collectd --qf '%{NAME} %{VERSION} %{RELEASE} %{ARCH}\n'
python 2.4.3 27.el5 x86_64
collectd 4.10.3 1.el5 x86_64
pymongo 1.9 8.el5 x86_64

How reproducible:
Always

Steps to Reproduce:
1. Configure as above
2. Run collectd
3.
  
Actual results:
The following error is generated and the python plugin isn't used:
plugin_load_file: The global flag is not supported, libtool 2 is required for this.
python plugin: Error importing module "mongodb".
Unhandled python exception in importing module: ImportError: /usr/lib64/python2.4/lib-dynload/datetime.so: undefined symbol: _Py_ZeroStruct
python plugin: Found a configuration for the "mongodb" plugin, but the plugin isn't loaded or didn't register a configuration callback.

Expected results:
No undefined symbol message.

Additional info:
By explicitly setting LD_PRELOAD to point at libpython, the error is suppressed and the module appears to function correctly.

LD_PRELOAD=/usr/lib64/libpython2.4.so collectd -C collectd.conf -P /tmp/collectd.pid
plugin_load_file: The global flag is not supported, libtool 2 is required for this.
python plugin: Found a configuration for the "mongodb" plugin, but the plugin isn't loaded or didn't register a configuration callback.
Comment 1 Jonathan Barber 2011-09-19 11:29:13 EDT
Created attachment 523853 [details]
collectd configuration
Comment 2 Jonathan Barber 2011-09-19 11:29:59 EDT
Created attachment 523857 [details]
Plugin that causes the problem
Comment 3 Kevin Fenzi 2011-09-19 12:35:31 EDT
Does it work correctly with python-2.4.3-44.el5 ?
That is the version it was built against...
Comment 4 Jonathan Barber 2011-09-19 12:56:16 EDT
Doesn't appear to, I've "yum updated python" and now have the following versions:
python-devel 2.4.3 44.el5 i386
python-libs 2.4.3 44.el5 x86_64
python-devel 2.4.3 44.el5 x86_64
collectd 4.10.3 1.el5 x86_64
pymongo 1.9 8.el5 x86_64
python 2.4.3 44.el5 x86_64

and I still get the same error without the LD_PRELOAD.
Comment 5 David Robinson 2011-09-20 03:16:40 EDT
I'm experiencing the same problem, but on RHEL 6.

collectd-4.10.3-1.el6.x86_64
python-2.6.6-20.el6.x86_64

You can also reproduce the problem via an "import httplib":

python plugin: Error importing module "example".
Unhandled python exception in importing module: ImportError: /usr/lib64/python2.6/lib-dynload/arraymodule.so: undefined symbol: _Py_ZeroStruct
Traceback (most recent call last):
  File "/usr/local/libexec/example.py", line 4, in <module>#012    import httplib
  File "/usr/lib64/python2.6/httplib.py", line 69, in <module>#012    from array import array
ImportError: /usr/lib64/python2.6/lib-dynload/arraymodule.so: undefined symbol: _Py_ZeroStruct

collectd-4.10.2-2.el6 works as expected - looks like its a regression in 4.10.3-1.
Comment 6 David Robinson 2011-09-20 19:57:04 EDT
Looks like others have stumbled across this too:

https://github.com/indygreg/collectd-carbon
http://mailman.verplant.org/pipermail/collectd/2008-March/001616.html
Comment 7 Alan Pevec 2011-09-21 04:50:01 EDT
So workaround is:
-%configure \
+%configure CFLAGS="%{optflags} -DLT_LAZY_OR_NOW='RTLD_LAZY|RTLD_GLOBAL'" \

Kevin, Lubomir - any objections to push that?
Comment 8 Kevin Fenzi 2011-09-21 11:02:07 EDT
Sounds ok to me if it fixes it.
Comment 9 Alan Pevec 2011-11-28 19:19:41 EST
*** Bug 655305 has been marked as a duplicate of this bug. ***
Comment 10 Fedora Update System 2011-11-29 05:07:11 EST
collectd-4.10.4-1.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/collectd-4.10.4-1.fc16
Comment 11 Fedora Update System 2011-11-29 21:03:35 EST
Package collectd-4.10.4-1.fc16:
* should fix your issue,
* was pushed to the Fedora 16 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing collectd-4.10.4-1.fc16'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2011-16624/collectd-4.10.4-1.fc16
then log in and leave karma (feedback).
Comment 12 Fedora Update System 2011-12-22 22:25:42 EST
collectd-4.10.4-1.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.