Bug 623854

Summary:	Use PyCapsule API instead of deprecated PyCObject API in python bindings
Product:	[Fedora] Fedora	Reporter:	Dave Malcolm <dmalcolm>
Component:	swig	Assignee:	Adam Tkac <atkac>
Status:	CLOSED ERRATA	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	medium	Docs Contact:
Priority:	low
Version:	14	CC:	atkac, dwalsh, john.ellson, mgrepl, ovasik
Target Milestone:	---
Target Release:	---
Hardware:	All
OS:	Linux
URL:	https://sourceforge.net/tracker/?func=detail&aid=3047039&group_id=1645&atid=301645
Whiteboard:
Fixed In Version:	swig-2.0.0-5.fc14	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:
Clones:	624674 (view as bug list)		Environment:
Last Closed:	2010-09-14 05:00:19 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	620842, 624674

Description Dave Malcolm 2010-08-13 00:52:58 UTC

Description of problem:
Python 2.7 deprecated the PyCObject API in favor of a new "capsule" API.
  http://docs.python.org/dev/whatsnew/2.7.html#capsules

The deprecations are set to "ignore" by default, so in theory the API still works.

However the deprecation has caused at least one app to fail (virt-manager, see bug 620216, due to it modifying the warning settings), so I've been doublechecking the scope of usage of the PyCObject API, and am filing bugs against components that are possibly affected.

You can trigger a hard failure of the API via:
>>> import warnings
>>> warnings.filterwarnings('error')
>>> import selinux
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python2.7/site-packages/selinux/__init__.py", line 25, in <module>
    _selinux = swig_import_helper()
  File "/usr/lib64/python2.7/site-packages/selinux/__init__.py", line 21, in swig_import_helper
    _mod = imp.load_module('_selinux', fp, pathname, description)
PendingDeprecationWarning: The CObject type is marked Pending Deprecation in Python 2.7.  Please use capsule objects instead.
>>> dir (selinux)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'selinux' is not defined

The usage comes via SWIG:
./libselinux-2.0.96/src/selinuxswig_wrap.c:2427:    type_pointer = PyCObject_Import((char*)"swig_runtime_data" SWIG_RUNTIME_VERSION,
./libselinux-2.0.96/src/selinuxswig_wrap.c:2497:  PyObject *pointer = PyCObject_FromVoidPtr((void *) swig_module, SWIG_Python_DestroyModule);
./libselinux-2.0.96/src/selinuxswig_wrap.c:2520:    descriptor = (swig_type_info *) PyCObject_AsVoidPtr(obj);
./libselinux-2.0.96/src/selinuxswig_wrap.c:2525:      obj = PyCObject_FromVoidPtr(descriptor, NULL);

I'm not yet sure what the best approach is to fixing this.

Version-Release number of selected component (if applicable):
libselinux-python-2.0.96-3.fc14.x86_64

Comment 1 Daniel Walsh 2010-08-13 18:36:09 UTC

Seems like this should be a bug reported to SWIG?

Comment 2 Adam Tkac 2010-08-17 13:04:33 UTC

(In reply to comment #1)
> Seems like this should be a bug reported to SWIG?

Right you are. I've just build swig-2.0.0-3.fc15 (https://kojiweb.fedoraproject.org/koji/buildinfo?buildID=190601), can you verify it really fixes the problem, please? I tested libselinux-python built with the new swig only with short script written below and it seems it works fine.

$ cat test.py 
#!/usr/bin/python

import warnings;
warnings.filterwarnings('error');
import selinux;

print "Enabled: %d" % selinux.is_selinux_enabled();

Comment 3 Dave Malcolm 2010-08-17 13:20:26 UTC

Thanks for looking at this.  I'm a little concerned about runtime compatibility between modules built with a version of SWIG that uses one API versus other modules built with a different SWIG using the other API.

Looking at swig200-rh623854.patch [1] I see:
  Py_InitModule((char*)"swig_runtime_data" SWIG_RUNTIME_VERSION
which looks like there's a fake module named "swig_runtime_data", which exposes "type_pointer".

It looks like existing modules built with the old code expect this value to be a PyCObject, whereas new modules built with the new code will expect it to be a PyCapsuleObject.  So I'm concerned about lingering bugs which will vary based on when the module was built, and which module gets imported first.  Does this work?

[1] http://pkgs.fedoraproject.org/gitweb/?p=swig.git;a=blob;f=swig200-rh623854.patch;h=ee53c0262ab7db5fe667c0c979f84b342f0049bb;hb=ca6fe127cf56ca735e30bc66898c6c55b09beba5

Comment 4 Adam Tkac 2010-08-17 14:07:57 UTC

(In reply to comment #3)
> Thanks for looking at this.  I'm a little concerned about runtime compatibility
> between modules built with a version of SWIG that uses one API versus other
> modules built with a different SWIG using the other API.
> 
> Looking at swig200-rh623854.patch [1] I see:
>   Py_InitModule((char*)"swig_runtime_data" SWIG_RUNTIME_VERSION
> which looks like there's a fake module named "swig_runtime_data", which exposes
> "type_pointer".
> 
> It looks like existing modules built with the old code expect this value to be
> a PyCObject, whereas new modules built with the new code will expect it to be a
> PyCapsuleObject.  So I'm concerned about lingering bugs which will vary based
> on when the module was built, and which module gets imported first.  Does this
> work?

Hm, I didn't remind scenario when multiple modules built with different swigs are simultaneously used. It seems that SWIG_RUNTIME_VERSION must be bumped. However I must discuss this change with upstream first.

Comment 5 Fedora Update System 2010-08-20 10:26:53 UTC

swig-2.0.0-4.fc14 has been submitted as an update for Fedora 14.
http://admin.fedoraproject.org/updates/swig-2.0.0-4.fc14

Comment 6 Fedora Update System 2010-08-20 18:39:52 UTC

swig-2.0.0-4.fc14 has been pushed to the Fedora 14 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update swig'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/swig-2.0.0-4.fc14

Comment 7 John Ellson 2010-08-25 17:00:19 UTC

Bug #627310 might have been caused by this change?

Comment 8 Adam Tkac 2010-08-27 09:08:44 UTC

(In reply to comment #7)
> Bug #627310 might have been caused by this change?

Right, bug #627310 should be fixed in swig-2.0.0-5.fc14.

Comment 9 Fedora Update System 2010-09-02 22:07:19 UTC

swig-2.0.0-5.fc14 has been pushed to the Fedora 14 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update swig'.  You can provide feedback for this update here: https://admin.fedoraproject.org/updates/swig-2.0.0-5.fc14

Comment 10 Fedora Update System 2010-09-14 05:00:09 UTC

swig-2.0.0-5.fc14 has been pushed to the Fedora 14 stable repository.  If problems still persist, please make note of it in this bug report.