[root@hamzy-fedora17-build ~]# uname -a Linux hamzy-fedora17-build.localdomain 3.3.4-5.fc17.x86_64 #1 SMP Mon May 7 17:29:34 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux [root@hamzy-fedora17-build ~]# yum install -y pygobject3 [root@hamzy-fedora17-build ~]# cat << __EOF__ | python - #!/usr/bin/python from gi.repository import NMClient if __name__ == "__main__": client = NMClient.Client.new() for con in client.get_active_connections(): print con device = con.get_devices()[0] print device print device.get_device_type() __EOF__ <ActiveConnection object at 0x7f5a611291e0 (NMActiveConnection at 0xe06010)> <DeviceEthernet object at 0x7f5a61129230 (NMDeviceEthernet at 0xe0b000)> <enum NM_DEVICE_TYPE_ETHERNET of type DeviceType> [root@hamzy-fedora17-build ~]# nmcli con NAME UUID TYPE TIMESTAMP-REAL Boot Disk f4b02c17-5a5c-49aa-bc30-b4ad5926b8d3 802-3-ethernet Thu 19 Jul 2012 08:56:12 AM CDT [root@bluebill ~]# uname -a Linux bluebill.rchland.ibm.com 3.3.4-5.fc17.ppc64 #1 SMP Mon May 14 10:18:37 MST 2012 ppc64 ppc64 ppc64 GNU/Linux [root@bluebill ~]# yum install -y pygobject3 [root@bluebill ~]# cat << __EOF__ | python - > #!/usr/bin/python > from gi.repository import NMClient > > if __name__ == "__main__": > client = NMClient.Client.new() > > for con in client.get_active_connections(): > print con > device = con.get_devices()[0] > print device > print device.get_device_type() > __EOF__ <ActiveConnection object at 0xfffa02a6190 (NMActiveConnection at 0x1003002c800)> <DeviceEthernet object at 0xfffa02a61e0 (NMDeviceEthernet at 0x10030034000)> Traceback (most recent call last): File "<stdin>", line 11, in <module> File "/usr/lib64/python2.7/site-packages/gi/types.py", line 43, in function return info.invoke(*args, **kwargs) ValueError: invalid enum value: 4294967296 [root@bluebill ~]# nmcli con NAME UUID TYPE TIMESTAMP-REAL Wired connection 1 34edb1f4-82bc-4471-ab69-4d01893d3293 802-3-ethernet Fri 13 Jul 2012 01:57:48 AM CDT Boot Disk bd10e0ca-184a-45d2-aa2a-769f973d54b6 802-3-ethernet Thu 19 Jul 2012 09:03:57 AM CDT
Might even be a NM bug as get_device_type is from NM, I'll CC: the maintainer <DeviceEthernet object at 0x1001318a4b0 (NMDeviceEthernet at 0x10013256070)> <gi.FunctionInfo object (get_device_type) at 0x0x100131c1c68> get_device_type NMClient Traceback (most recent call last): File "<stdin>", line 10, in <module> File "/usr/lib64/python2.7/site-packages/gi/types.py", line 46, in function return info.invoke(*args, **kwargs) ValueError: invalid enum value: 4294967296
looks like a big-endian or 32bit vs 64bit issue, 4294967296 = 0x100000000 a check of build logs for "implicit declaration" warnings can be useful
1) Many testcases for python3-pygopject3 fail. (cd ~/rpmbuild/BUILD/python3-pygobject3-3.2.2-1.fc17/tests; make check) 2) gobject-introspection testcase seems incorrectly written. Where/What is the official standard? [root@sharpie ~]# rpm -qf /usr/share/gobject-introspection-1.0/tests/gimarshallingtests.c gobject-introspection-devel-1.32.1-1.fc17.ppc64 I see stuff like this: test_glist_int_none_in (test_gi.TestGList) ... g_list_length(list) = 4 list = 0x1000606f620 g_list_nth_data(list, 0) = 0xffffffff70ddde1c GPOINTER_TO_INT(g_list_nth_data(list, 0)) = 1893588508 g_list_nth_data(list, 1) = 0x70ddde1c GPOINTER_TO_INT(g_list_nth_data(list, 1)) = 1893588508 g_list_nth_data(list, 2) = 0x170ddde1c GPOINTER_TO_INT(g_list_nth_data(list, 2)) = 1893588508 g_list_nth_data(list, 3) = 0x270ddde1c GPOINTER_TO_INT(g_list_nth_data(list, 3)) = 1893588508 it is looking for -1, 0, 1, 2, and 3. But there is garbage in half of it.
Filed upstream: https://bugzilla.gnome.org/show_bug.cgi?id=680692
Created attachment 601594 [details] Work in progress patch to fix enum issues on 64-bit bigendian archs Enums appear to be thoroughly broken on ppc64, with all enum values coming out as 0; the upstream selftest suite shows many failures, coredumping before running to completion. The root-cause for this appears to be this implicit cast to "long" within: pygobject/gi/pygi-marshal-from-py.c:_pygi_marshal_from_py_interface_enum where the enum value is extracted as a C "long" (OK) but then written through to the "v_long" member of the union, thus truncating the value to the most-significant 32-bits on a 64-bit big-endian box, thus leaving most values as 0: 1295 >>1296 arg->v_long = PYGLIB_PyLong_AsLong (int_); 1297 Py_DECREF (int_); 1298 I believe this will affect any big-endian box where sizeof(long) > 4, such as ppc64, where sizeof(long) == 8 gobject-introspection's girepository/gifieldinfo.c:g_field_info_get_field has some logic for writing an enum value into a GIArgument using an appropriate size (which appears to be computed in girepository/giroffsets.c), so I've written similar code (perhaps should be a new helper function in gobject-introspection). I'm attaching the work-in-progress patch I have for this. With this patch, the TESTNAMES=test_gi.TestEnum subset of the test suite actually completes (rather than coredumping), with only 1 failure: ====================================================================== ERROR: test_enum_out (test_gi.TestEnum) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/dmalcolm/pygobject/tests/test_gi.py", line 1136, in test_enum_out enum = GIMarshallingTests.enum_out() File "/home/dmalcolm/pygobject/gi/types.py", line 47, in function return info.invoke(*args, **kwargs) ValueError: invalid enum value: 180388626432 ---------------------------------------------------------------------- Ran 6 tests in 68.107s FAILED (errors=1) This appears to be a similar issue in _pygi_marshal_to_py_interface_enum, but I need to head out now, so am posting what I've got (in case my dev box gets accidentally reprovisioned).
Created attachment 601820 [details] Work in progress patch against upstream master to fix various issues on ppc64
I fixed the marshalling from/to GList and GSList in trunk: http://git.gnome.org/browse/pygobject/commit/?id=770e6abfd5bc5dad7d5f56a18f1ef63f9754ada9
FYI, yesterday I committed http://git.gnome.org/browse/gobject-introspection/commit/?id=d0ffee9bf which makes the assertion failures a lot more useful.
When running "xvfb-run make check" in the porter mock chroot, I also got two breakages because Pango threw a warning. Fixed in http://git.gnome.org/browse/pygobject/commit/?id=dbc6df6aad This shouldn't affect your porting on desktop machines, but if it does, you can grab the patch. Now I only have one abort left: test_constructor (test_overrides.TestGdk) ... (runtests.py:35764): Gdk-WARNING **: gdkwindow.c:1406cannot make windows of type 0 make[2]: *** [check-local] Trace/Breakpoint ausgelöst This might be a bug in Gdk or PyGObject, I haven't investigated this yet. When I disable the test (test_overrides.py TestGdk.test_constructor), there are no C-level assertions any more, and I get Ran 575 tests in 6.506s FAILED (failures=9, errors=1, expected failures=1) which still points out some serious errors, but at least there's some visible progress. I need to stop looking at this now, I'm afraid. I'll be back from holidays on August 20, so I guess I'll let you continue working on the currently attached patch version for the remaining issues. You can hop into #python on GNOME IRC to find some people to review/commit patches if you need it in the next two weeks (in particular, tomeu and pbor). Thanks!
Created attachment 603090 [details] Work in progress patch to fix various issues on ppc64 Attached patch is a revision of Dave's patch including the following fixes: a) Fix enum/flags conversion to/from PyObject in gi/pygi-argument.c b) Fix closure return value storage in gi/pygi-closure.c When using girffi you must be careful to use libffi conventions. It means that the closure function passed to g_callable_info_prepare_closure() must store integral return values into a storage of size sizeof(ffi_arg) even for smaller-sized return values.
Note that I've been working on top of Fedora 17's code, I didn't try to apply this on upstream, it may need some rework (though previous patches were flawless portable between f17 and upstream).
I added an invocation of the upstream selftest suite (just for Python 2 for now) to pygobject.spec's %check in f18 and master (now f19) as: http://pkgs.fedoraproject.org/cgit/pygobject3.git/commit/?id=4b2a8915cf06909568537feb2bbd3cbac392ee18 This will fail the build if any unexpected selftest failures happen (I had to mark one of the tests as failing, as it fails on i686, though not on x86_64). I successfully rebuilt on the primary archs into f18: Building pygobject3-3.3.4-4.fc18 for f18-candidate Task info: http://koji.fedoraproject.org/koji/taskinfo?taskID=4370760 (I'm not running it for the python3 build yet; this appears to require additional fixes, but it isn't necessary for getting anaconda running, hence I believe it's a lower priority than the python2 build).
(In reply to comment #11) > Created attachment 603090 [details] > Work in progress patch to fix various issues on ppc64 > > Attached patch is a revision of Dave's patch including the following fixes: > a) Fix enum/flags conversion to/from PyObject in gi/pygi-argument.c > b) Fix closure return value storage in gi/pygi-closure.c > When using girffi you must be careful to use libffi conventions. It > means that the closure function passed to g_callable_info_prepare_closure() > must store integral return values into a storage of size sizeof(ffi_arg) > even for smaller-sized return values. Patch looks reasonable. I tried scratch rebuilding the f19 package with this on the primary architectures, but ran into this new error (on both archs): ====================================================================== FAIL: test_python_calls_sync (test_gdbus.TestGDBusClient) ---------------------------------------------------------------------- Traceback (most recent call last): File "/builddir/build/BUILD/pygobject-3.3.4/tests/test_gdbus.py", line 140, in test_python_calls_sync self.assertTrue('Timeout' in str(e), str(e)) AssertionError: GDBus.Error:org.freedesktop.DBus.Error.NameHasNoOwner: Could not get PID of name '1': no such name ----------------------------------------------------------------------
Created attachment 603367 [details] Work in progress patch to fix various issues on ppc64 A new revision of previous patch including a fix for the conversion of GHashTable from/to PyObject. I had to move the functions _pygi_hash_pointer_to_arg() and _pygi_arg_to_hash_pointer() to gi/pygi-argument.c
(In reply to comment #15) > Created attachment 603367 [details] > Work in progress patch to fix various issues on ppc64 > > A new revision of previous patch including a fix for the conversion of > GHashTable from/to PyObject. > I had to move the functions _pygi_hash_pointer_to_arg() and > _pygi_arg_to_hash_pointer() to gi/pygi-argument.c Thanks. I've updated "endianness-fixes.patch" within f18 and master to attachment 603367 [details] as: http://pkgs.fedoraproject.org/cgit/pygobject3.git/commit/?id=58e8a6c275c75dc06d3b2baef623932569c82240 and am rebuilding into f18-candidate as http://koji.fedoraproject.org/koji/taskinfo?taskID=4374917 (it successfully scratch built for both i686 and x86_64). I attempted a scratch build of this change into f18-candidate for ppc-koji as: http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=662395 and there was a single remaining failure: ====================================================================== ERROR: test_interface3impl (test_gi.TestPythonGObject) ---------------------------------------------------------------------- Traceback (most recent call last): File "/builddir/build/BUILD/pygobject-3.3.4/tests/test_gi.py", line 1752, in test_interface3impl self.assertEqual(iface3.variants[0].unpack(), 27) IndexError: list index out of range ---------------------------------------------------------------------- Ran 568 tests in 6.311s FAILED (errors=1, expected failures=1, unexpected successes=2)
Getting late here, but this failure seems to be due to this assignment within _pygi_argument_to_array within "case GI_ARRAY_TYPE_C": 769 /* FIXME: Take into account the type of the length argument */ 770 length = args[length_arg_pos]->v_int; (gdb) p *args[length_arg_pos] $22 = {v_boolean = 0, v_int8 = 0 '\000', v_uint8 = 0 '\000', v_int16 = 0, v_uint16 = 0, v_int32 = 0, v_uint32 = 0, v_int64 = 2, v_uint64 = 2, v_float = 0, \ v_double = 9.8813129168249309e-324, v_short = 0, v_ushort = 0, v_int = 0, v_uint = 0, v_long = 2, v_ulong = 2, v_ssize = 2, v_size = 2, v_string = 0x2 <Add\ ress 0x2 out of bounds>, v_pointer = 0x2} Hence the python list passed as "variants" to the do_test_variant_array_in closure is being constructed with length 0; do_test_variant_array_in is called with variants: [] n_variants: 2L and this causes the IndexError looking up iface3.variants[0]
Created attachment 603634 [details] Fix "test_interface3impl (test_gi.TestPythonGObject)" failure This applies on top of attachment 603367 [details] to fix the error described in comment #16 and comment #17
(In reply to comment #18) > Created attachment 603634 [details] > Fix "test_interface3impl (test_gi.TestPythonGObject)" failure > > This applies on top of attachment 603367 [details] to fix the error > described in comment #16 and comment #17 Committed to dist-git (f18 and master) as: http://pkgs.fedoraproject.org/cgit/pygobject3.git/commit/?id=794d0789064aa6fb5bbfe111393037e3c0e623bb Building pygobject3-3.3.4-8.fc18 for f18-candidate Task info: http://koji.fedoraproject.org/koji/taskinfo?taskID=4378195 Built successfully on a test ppc64 box: Ran 576 tests in 30.793s (skipped=68, expected failures=1) but I wasn't able to build into koji-ppc due to an unrelated build.log issue: http://ppc.koji.fedoraproject.org/koji/getfile?taskID=663123&name=root.log
I successfully rebuilt pygobject3-3.3.4-8.fc18 into koji-ppc as: http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=664109 On ppc: Ran 568 tests in 5.994s OK (expected failures=2, unexpected successes=1) On ppc64: Ran 568 tests in 6.255s OK (expected failures=1, unexpected successes=2) Are there any remaining endianness issues with pygobject3? (still need to get the patches upstream, of course, but Martin Pitt is still on vacation for another week IIRC).
Backported to the 3.2.2 in f17 and built for ppc into f17-candidate as pygobject3-3.2.2-6.fc17: http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=664311
Thanks Malcolm, great work! I did some minor cleanup in the patch, and pushed it upstream: http://git.gnome.org/browse/pygobject/commit/?id=1c5d497d3c354f4d02f1d4570df2c61d6f47300c With this I only get one test failure on ppc64 (test_gi.TestPythonGObject.test_interface3impl) which is covered and fixed by bug 842880.
This is still reproducible on http://ppc.koji.fedoraproject.org/stage/f18-20120820.0/18/ppc64/iso/Fedora-18-ppc64-DVD.iso
(In reply to comment #23) > This is still reproducible on > http://ppc.koji.fedoraproject.org/stage/f18-20120820.0/18/ppc64/iso/Fedora- > 18-ppc64-DVD.iso Looks like a tagging mishap (perhaps due to koji-shadow) caused the wrong version of pygobject3 to be included. I'll make sure this is fixed for our next ISO spin.
As we haven't encountered any more pygobject3 bugs in the last 3 months i'm closing this bug as fixed for now. Should we hit any more we'll create a new bug. Thanks everyone for fixing all those issues! Regards, Phil