Bug 739593 - Python plugin doesn't work
Summary: Python plugin doesn't work
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: collectd
Version: rawhide
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Alan Pevec
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 655305 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-09-19 15:28 UTC by Jonathan Barber
Modified: 2011-12-23 03:25 UTC (History)
8 users (show)

Fixed In Version: collectd-4.10.4-1.fc16
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2011-12-23 03:25:42 UTC


Attachments (Terms of Use)
collectd configuration (266 bytes, application/octet-stream)
2011-09-19 15:29 UTC, Jonathan Barber
no flags Details
Plugin that causes the problem (157 bytes, application/octet-stream)
2011-09-19 15:29 UTC, Jonathan Barber
no flags Details

Description Jonathan Barber 2011-09-19 15:28:25 UTC
Description of problem:
Collectd doesn't work with the pymongo module, it is unable to find the symbol _Py_ZeroStruct.

Configure collectd with the following /etc/collectd.conf configuration:
Hostname    "test"
FQDNLookup   false
BaseDir     "/tmp/foo"

<LoadPlugin python>
    Globals true
</LoadPlugin>

<Plugin python>
    ModulePath "/tmp/plugins"

    Import "mongodb"
    <Module mongodb>
        Host "127.0.0.1"
    </Module>
</Plugin>

The /tmp/plugins/mongodb.py plugin is:
import collectd
from pymongo import Connection

class MongoDB(object):

    def __init__(self):
        self.plugin_name = "mongo"

mongodb = MongoDB()

Then run collected:
collectd -C collectd.conf -P /tmp/collectd.pid

Gives the error:
Unhandled python exception in importing module: ImportError: /usr/lib64/python2.4/lib-dynload/datetime.so: undefined symbol: _Py_ZeroStruct

and doesn't run the mongodb plugin.

Version-Release number of selected component (if applicable):
rpm -qa pymongo python collectd --qf '%{NAME} %{VERSION} %{RELEASE} %{ARCH}\n'
python 2.4.3 27.el5 x86_64
collectd 4.10.3 1.el5 x86_64
pymongo 1.9 8.el5 x86_64

How reproducible:
Always

Steps to Reproduce:
1. Configure as above
2. Run collectd
3.
  
Actual results:
The following error is generated and the python plugin isn't used:
plugin_load_file: The global flag is not supported, libtool 2 is required for this.
python plugin: Error importing module "mongodb".
Unhandled python exception in importing module: ImportError: /usr/lib64/python2.4/lib-dynload/datetime.so: undefined symbol: _Py_ZeroStruct
python plugin: Found a configuration for the "mongodb" plugin, but the plugin isn't loaded or didn't register a configuration callback.

Expected results:
No undefined symbol message.

Additional info:
By explicitly setting LD_PRELOAD to point at libpython, the error is suppressed and the module appears to function correctly.

LD_PRELOAD=/usr/lib64/libpython2.4.so collectd -C collectd.conf -P /tmp/collectd.pid
plugin_load_file: The global flag is not supported, libtool 2 is required for this.
python plugin: Found a configuration for the "mongodb" plugin, but the plugin isn't loaded or didn't register a configuration callback.

Comment 1 Jonathan Barber 2011-09-19 15:29:13 UTC
Created attachment 523853 [details]
collectd configuration

Comment 2 Jonathan Barber 2011-09-19 15:29:59 UTC
Created attachment 523857 [details]
Plugin that causes the problem

Comment 3 Kevin Fenzi 2011-09-19 16:35:31 UTC
Does it work correctly with python-2.4.3-44.el5 ?
That is the version it was built against...

Comment 4 Jonathan Barber 2011-09-19 16:56:16 UTC
Doesn't appear to, I've "yum updated python" and now have the following versions:
python-devel 2.4.3 44.el5 i386
python-libs 2.4.3 44.el5 x86_64
python-devel 2.4.3 44.el5 x86_64
collectd 4.10.3 1.el5 x86_64
pymongo 1.9 8.el5 x86_64
python 2.4.3 44.el5 x86_64

and I still get the same error without the LD_PRELOAD.

Comment 5 David Robinson 2011-09-20 07:16:40 UTC
I'm experiencing the same problem, but on RHEL 6.

collectd-4.10.3-1.el6.x86_64
python-2.6.6-20.el6.x86_64

You can also reproduce the problem via an "import httplib":

python plugin: Error importing module "example".
Unhandled python exception in importing module: ImportError: /usr/lib64/python2.6/lib-dynload/arraymodule.so: undefined symbol: _Py_ZeroStruct
Traceback (most recent call last):
  File "/usr/local/libexec/example.py", line 4, in <module>#012    import httplib
  File "/usr/lib64/python2.6/httplib.py", line 69, in <module>#012    from array import array
ImportError: /usr/lib64/python2.6/lib-dynload/arraymodule.so: undefined symbol: _Py_ZeroStruct

collectd-4.10.2-2.el6 works as expected - looks like its a regression in 4.10.3-1.

Comment 6 David Robinson 2011-09-20 23:57:04 UTC
Looks like others have stumbled across this too:

https://github.com/indygreg/collectd-carbon
http://mailman.verplant.org/pipermail/collectd/2008-March/001616.html

Comment 7 Alan Pevec 2011-09-21 08:50:01 UTC
So workaround is:
-%configure \
+%configure CFLAGS="%{optflags} -DLT_LAZY_OR_NOW='RTLD_LAZY|RTLD_GLOBAL'" \

Kevin, Lubomir - any objections to push that?

Comment 8 Kevin Fenzi 2011-09-21 15:02:07 UTC
Sounds ok to me if it fixes it.

Comment 9 Alan Pevec 2011-11-29 00:19:41 UTC
*** Bug 655305 has been marked as a duplicate of this bug. ***

Comment 10 Fedora Update System 2011-11-29 10:07:11 UTC
collectd-4.10.4-1.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/collectd-4.10.4-1.fc16

Comment 11 Fedora Update System 2011-11-30 02:03:35 UTC
Package collectd-4.10.4-1.fc16:
* should fix your issue,
* was pushed to the Fedora 16 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing collectd-4.10.4-1.fc16'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2011-16624/collectd-4.10.4-1.fc16
then log in and leave karma (feedback).

Comment 12 Fedora Update System 2011-12-23 03:25:42 UTC
collectd-4.10.4-1.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.