Description of problem: PyGObject3 fails to build in self-check on PPC64 with the following error message: (process:39448): GLib-GObject-CRITICAL **: g_object_class_install_property: assertion `pspec->flags & (G_PARAM_READABLE | G_PARAM_WRITABLE)' failed /bin/sh: line 4: 39448 Trace/breakpoint trap (core dumped) PYTHONPATH=..:../tests:${PYTHONPATH:+:$PYTHONPATH} LD_LIBRARY_PATH=./.libs:$LD_LIBRARY_PATH GI_TYPELIB_PATH=.:$GI_TYPELIB_PATH XDG_DATA_DIRS=$XDG_DATA_DIRS:/usr/share MALLOC_PERTURB_=85 MALLOC_CHECK_=3 G_SLICE=debug-blocks TESTS_BUILDDIR=. /usr/bin/python -Wd ./runtests.py Version-Release number of selected component (if applicable): pygobject3-3.7.92-1.fc19 How reproducible: Always Steps to Reproduce: 1. ppc-koji build --scratch f19 pygobject3-3.7.92-1.fc19.src.rpm 2. 3. Actual results: http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=1004307 Expected results: Additional info:
I'm able to reproduce this on coho ppc64 with pygobject3-3.8.0-1.fc19.src.rpm within gdb and am investigating. Note to self: [dmalcolm@coho ~]$ mock -r fedora-rawhide-ppc64-dmalcolm --configdir=/home/dmalcolm/mymockdir shell <mock-chroot>[root@coho /]# cd builddir/build/BUILD/pygobject-3.8.0/tests [root@coho tests]# /tmp/dmalcolm-xvfb-run make check.gdb Python level backtrace at the point of the crash is: (gdb) py-bt #12 Frame 0x1029ef40, for file /builddir/build/BUILD/pygobject-3.8.0/gi/_gobject/__init__.py, line 58, in _type_register (cls=<GObjectMeta(uint64=<Property(fset=<instancemethod at remote 0x3fffacc02910>, name='uint64', default=0, maximum=18446744073709551615L, nick='', minimum=0, flags=<PyGObjectParamFlags at remote 0x3fffacc0f520>, fget=<instancemethod at remote 0x3fffaccc5e60>, _exc=None, type=<gobject.GType at remote 0x3fffb78e26d8>, __doc__='', blurb='') at remote 0x3fffacc12890>, __module__='test_properties', boxed=<Property(fset=<instancemethod at remote 0x3fffacc02a50>, name='boxed', default=None, maximum=None, nick='', minimum=None, flags=<PyGObjectParamFlags at remote 0x3fffacc0f788>, fget=<instancemethod at remote 0x3fffacc02a00>, _exc=None, type=<gobject.GType at remote 0x3fffaccb1360>, __doc__='', blurb='') at remote 0x3fffacc129d0>, construct_only=<Property(fset=<instancemethod at remote 0x3fffaccc5e10>, name='construct_only', default='', maximum=None, nick='', minimum=None, flags=<PyGObjectParamFlag...(truncated) _gobject.type_register(cls, namespace.get('__gtype_name__')) #15 Frame 0x1029e470, for file /builddir/build/BUILD/pygobject-3.8.0/gi/_gobject/__init__.py, line 46, in __init__ (cls=<GObjectMeta(uint64=<Property(fset=<instancemethod at remote 0x3fffacc02910>, name='uint64', default=0, maximum=18446744073709551615L, nick='', minimum=0, flags=<PyGObjectParamFlags at remote 0x3fffacc0f520>, fget=<instancemethod at remote 0x3fffaccc5e60>, _exc=None, type=<gobject.GType at remote 0x3fffb78e26d8>, __doc__='', blurb='') at remote 0x3fffacc12890>, __module__='test_properties', boxed=<Property(fset=<instancemethod at remote 0x3fffacc02a50>, name='boxed', default=None, maximum=None, nick='', minimum=None, flags=<PyGObjectParamFlags at remote 0x3fffacc0f788>, fget=<instancemethod at remote 0x3fffacc02a00>, _exc=None, type=<gobject.GType at remote 0x3fffaccb1360>, __doc__='', blurb='') at remote 0x3fffacc129d0>, construct_only=<Property(fset=<instancemethod at remote 0x3fffaccc5e10>, name='construct_only', default='', maximum=None, nick='', minimum=None, flags=<PyGObjectParamFlags at r...(truncated) cls._type_register(cls.__dict__) #18 Frame 0x1029e250, for file /builddir/build/BUILD/pygobject-3.8.0/gi/types.py, line 283, in __init__ (cls=<GObjectMeta(uint64=<Property(fset=<instancemethod at remote 0x3fffacc02910>, name='uint64', default=0, maximum=18446744073709551615L, nick='', minimum=0, flags=<PyGObjectParamFlags at remote 0x3fffacc0f520>, fget=<instancemethod at remote 0x3fffaccc5e60>, _exc=None, type=<gobject.GType at remote 0x3fffb78e26d8>, __doc__='', blurb='') at remote 0x3fffacc12890>, __module__='test_properties', boxed=<Property(fset=<instancemethod at remote 0x3fffacc02a50>, name='boxed', default=None, maximum=None, nick='', minimum=None, flags=<PyGObjectParamFlags at remote 0x3fffacc0f788>, fget=<instancemethod at remote 0x3fffacc02a00>, _exc=None, type=<gobject.GType at remote 0x3fffaccb1360>, __doc__='', blurb='') at remote 0x3fffacc129d0>, construct_only=<Property(fset=<instancemethod at remote 0x3fffaccc5e10>, name='construct_only', default='', maximum=None, nick='', minimum=None, flags=<PyGObjectParamFlags at remote 0x3ff...(truncated) super(GObjectMeta, cls).__init__(name, bases, dict_) #29 Frame 0x104b7fd0, for file /builddir/build/BUILD/pygobject-3.8.0/tests/test_properties.py, line 38, in <module> () class PropertyObject(GObject.GObject): #42 Frame 0x1010fbf0, for file /usr/lib64/python2.7/unittest/loader.py, line 91, in loadTestsFromName (self=<TestLoader at remote 0x3fffb0da0150>, name='test_properties', module=None, parts=['test_properties'], parts_copy=['test_properties']) module = __import__('.'.join(parts_copy)) #46 Frame 0x10151df0, for file /usr/lib64/python2.7/unittest/loader.py, line 128, in loadTestsFromNames (self=<TestLoader at remote 0x3fffb0da0150>, names=['test_overrides_gtk', 'test_option', 'test_overrides_gdk', 'test_atoms', 'test_source', 'test_properties', 'test_overrides_pango', 'test_thread', 'test_object_marshaling', 'test_internal_api', 'test_generictreemodel', 'test_signal', 'test_overrides_glib', 'test_gdbus', 'test_interface', 'test_glib', 'test_gtype', 'test_subprocess', 'test_gi', 'test_gobject', 'test_overrides', 'test_mainloop', 'test_everything', 'test_gio', 'test_iochannel'], module=None, name='test_properties') suites = [self.loadTestsFromName(name, module) for name in names] #50 Frame 0x1010c3c0, for file ./runtests.py, line 98, in <module> () suite = loader.loadTestsFromNames(names)
The assert in question was added to glib in 2011 in: https://git.gnome.org/browse/glib/commit/gobject/gobject.c?id=b2371871097ef2b52bdb4688d702438c9e3f1787 (as part of https://bugzilla.gnome.org/show_bug.cgi?id=666616 )
Putting in print statements into gi/overrides/GObject.py shows: PARAM_READABLE: <flags 0 of type PyGObjectParamFlags> PARAM_WRITABLE: <flags 0 of type PyGObjectParamFlags> PARAM_READWRITE: <flags 0 of type PyGObjectParamFlags> Somehow these flags are coming out with value 0 on this arch. (gdb) p sizeof(GParamFlags) $1 = 4 though they're going through PYGLIB_PyLong which for Python 2 is a PyIntObject which uses a C long internally: (gdb) p sizeof(long) $76 = 8 I'm guessing that this is an int vs long endianness issue *somewhere* in the flag handling.
I can't access my ppc64 test machine right now, but looking at git logs, my hunch is that it's this commit: https://git.gnome.org/browse/pygobject/commit/gi/_gobject/pygflags.c?id=caeeeb7e4282e183eefc3c53b2d53c8c2bb7de89 which was due to https://bugzilla.gnome.org/show_bug.cgi?id=693121 In particular, quoting: https://bugzilla.gnome.org/show_bug.cgi?id=693121#c12 > So I added an extra zero padding to the struct, which helps: > > typedef struct { > PYGLIB_PyLongObject parent; > + int zero_pad; /* must always be 0 */ > GType gtype; > } PyGFlags; > I'm not sure whether that's guaranteed to succeed for all possible corner > cases, but at least the test cases now cover representation and arithmetic with > flags whose values exceed MAX_INT. This seems likely to break on a big-endian machine: any code paths that on rely on that zero-padding for the most-significant 4 bytes on a little-endian box are going to read it for the least-significant on a big-endian box.
I tried reverting that patch (the full version, rather than just the one scoped to pygflags.c in the link in comment #4), and it behaves much better: PARAM_READABLE: <flags G_PARAM_READABLE of type PyGObjectParamFlags> PARAM_WRITABLE: <flags G_PARAM_WRITABLE of type PyGObjectParamFlags> PARAM_READWRITE: <flags G_PARAM_READABLE | G_PARAM_WRITABLE of type PyGObjectParamFlags> and much of the test suite actually runs (albeit with lots of output of the form: WARNING **: expected enumeration type GtkWindowType, but got (null) instead )
Hi Dave, any further progress here? Thanks, Dan
After rebuilding 3.8.0 with the patch mentioned in: https://bugzilla.gnome.org/show_bug.cgi?id=693121#c15 specifically: https://git.gnome.org/browse/pygobject/commit/?h=pygobject-3-8&id=c1fb6516031d3c32abd6 everything works much better: the test suite starts and runs without aborting the python process. There is however a single failure ====================================================================== FAIL: test_vfunc_return_enum (test_gi.TestEnumVFuncResults) ---------------------------------------------------------------------- Traceback (most recent call last): File "/builddir/build/BUILD/pygobject-3.8.0/tests/test_gi.py", line 1512, in test_vfunc_return_enum self.assertEqual(tester.vfunc_return_enum(), GIMarshallingTests.Enum.VALUE2) AssertionError: <enum GI_MARSHALLING_TESTS_ENUM_VALUE1 of type PyGIMarshallingTestsEnum> != <enum GI_MARSHALLING_TESTS_ENUM_VALUE2 of type PyGIMarshallingTestsEnum> ---------------------------------------------------------------------- Ran 875 tests in 13.266s FAILED (failures=1, expected failures=4, unexpected successes=2)
(In reply to comment #7) > After rebuilding 3.8.0 with the patch mentioned in: > https://bugzilla.gnome.org/show_bug.cgi?id=693121#c15 > specifically: > > https://git.gnome.org/browse/pygobject/commit/?h=pygobject-3- > 8&id=c1fb6516031d3c32abd6 > everything works much better: the test suite starts and runs without > aborting the python process. > > There is however a single failure > ====================================================================== > FAIL: test_vfunc_return_enum (test_gi.TestEnumVFuncResults) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/builddir/build/BUILD/pygobject-3.8.0/tests/test_gi.py", line 1512, > in test_vfunc_return_enum > self.assertEqual(tester.vfunc_return_enum(), > GIMarshallingTests.Enum.VALUE2) > AssertionError: <enum GI_MARSHALLING_TESTS_ENUM_VALUE1 of type > PyGIMarshallingTestsEnum> != <enum GI_MARSHALLING_TESTS_ENUM_VALUE2 of type > PyGIMarshallingTestsEnum> > > ---------------------------------------------------------------------- > Ran 875 tests in 13.266s > > FAILED (failures=1, expected failures=4, unexpected successes=2) Thanks, Dave. Could send the srpm as scratch build to the s390 koji? I remember there were some ENUM specifics on ppc in the past, but not on s390, maybe it will build there.
(In reply to comment #7) > ====================================================================== > FAIL: test_vfunc_return_enum (test_gi.TestEnumVFuncResults) > ---------------------------------------------------------------------- I've reported this upstream as: https://bugzilla.gnome.org/show_bug.cgi?id=697138
Successful scratch build on ppc/ppc64 as: http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=1023126 Workaround committed downstream to git master (for f20) as: http://pkgs.fedoraproject.org/cgit/pygobject3.git/commit/? id=2655f4b74ce18266c5a053f2a53c49b1a378f1f3 and to f19 as: http://pkgs.fedoraproject.org/cgit/pygobject3.git/commit/?h=f19&id=2655f4b74ce18266c5a053f2a53c49b1a378f1f3 Building pygobject3-3.8.0-2.fc20 for rawhide Task info: http://koji.fedoraproject.org/koji/taskinfo?taskID=5200645 Building pygobject3-3.8.0-2.fc19 for f19-candidate Task info: http://koji.fedoraproject.org/koji/taskinfo?taskID=5200641
(In reply to comment #10) [...] > Building pygobject3-3.8.0-2.fc20 for rawhide > Task info: http://koji.fedoraproject.org/koji/taskinfo?taskID=5200645 Succeeded > Building pygobject3-3.8.0-2.fc19 for f19-candidate > Task info: http://koji.fedoraproject.org/koji/taskinfo?taskID=5200641 Failed [1], however on retrying as: http://koji.fedoraproject.org/koji/taskinfo?taskID=5201006 it succeeded. [1] with: (moo:9325): Gdk-WARNING **: moo: Fatal IO error 0 (Success) on X server :99. /bin/sh: line 4: 9325 Trace/breakpoint trap PYTHONPATH=..:../tests:${PYTHONPATH:+:$PYTHONPATH} LD_LIBRARY_PATH=./.libs:$LD_LIBRARY_PATH GI_TYPELIB_PATH=.:$GI_TYPELIB_PATH XDG_DATA_DIRS=$XDG_DATA_DIRS:/usr/share MALLOC_PERTURB_=85 MALLOC_CHECK_=3 G_SLICE=debug-blocks TESTS_BUILDDIR=. /usr/bin/python -Wd ./runtests.py make[2]: *** [check-local] Error 133 make[2]: Leaving directory `/builddir/build/BUILD/pygobject-3.8.0/tests'
pygobject3-3.8.0-2.fc19 has been submitted as an update for Fedora 19. https://admin.fedoraproject.org/updates/pygobject3-3.8.0-2.fc19
Package pygobject3-3.8.0-2.fc19: * should fix your issue, * was pushed to the Fedora 19 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing pygobject3-3.8.0-2.fc19' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2013-4743/pygobject3-3.8.0-2.fc19 then log in and leave karma (feedback).
proposing as F19 Alpha blocker, both ppc/ppc64 and s390/s390x are affected
Why is this a release blocker? What is the practical effect?
The secondary arches build packages within buildroots that contains ideally only the same NVR as the buildroots in primary, practically there can be few newer NVRs for packages that fail to build and must be fixed. But definitely no older NVRs are allowed. Not having pygobject3-3.8.0-2.fc19 (and latest python-pillow) tagged in f19 means secondary arches can't build cca 300 packages that existed in f19 before the alpha freeze eg. anaconda or the latest GNOME stuff. The release engineering tag syncing script ensure that same NVRs are tagged between primary and secondaries effectively meaning no newer NVR is possible in secondary. This won't allow to make an Alpha compose on secondaries that matches primary. Risk for inclusion of pygobject3-3.8.0-2.fc19 instead of pygobject3-3.8.0-1.fc19 in primary is close to zero.
It's definitely not a blocker as what current blocker definition says, but I think we could grant Freeze Exception for secondary architectures blockers as we do for non-release-blocking images - this one would fit: "Bugs which entirely prevent the composition of one or more of the images expected to be built for a currently-pending (pre-)release". Secondary architectures as Dan explained are trying to follow the primary architecture as much as possible and I can see benefits of helping them.
switching to Freeze Exception which is more appropriate
I can see FE a bit more, but still, "close to zero" is not "zero". Is there any way this can be finessed by releng without bumping the frozen package set for Alpha?
in the past secondary blockers were primary nice to have. we need to get this pushed through as a freeze exception
if that's how we handled it before, then sure, +1 FE.
(In reply to comment #19) > I can see FE a bit more, but still, "close to zero" is not "zero". Is there > any way this can be finessed by releng without bumping the frozen package > set for Alpha? There is one added patch taken as a bugfix from upstream, all tests are still passing, we probably can't do more.
Discussed at 2013-04-08 freeze exception review meeting: http://meetbot.fedoraproject.org/fedora-blocker-review/2013-04-08/f19alpha-blocker-review-5.2013-04-08-16.01.log.txt . Accepted as a freeze exception issue due to blocking a secondary arch.
pygobject3-3.8.0-2.fc19 has been pushed to the Fedora 19 stable repository. If problems still persist, please make note of it in this bug report.