Bug 1792990

This bug appears to have been reported against 'rawhide' during the Fedora 32 development cycle.
Changing version to 32.

When line 25 of test_cvp.py is reached:

        v1 = IntegerMatrix.from_iterable(1, A.nrows, map(lambda x: int(round(x)), v1))

control passes into Cython-generated code.  The source is src/fpylll/fplll/integer_matrix.pyx.  I set a breakpoint in gdb on the generated function, __pyx_pf_6fpylll_5fplll_14integer_matrix_13IntegerMatrix_4from_iterable.  The Cython definition looks like this (minus comments):

    @classmethod
    def from_iterable(cls, nrows, ncols, it, **kwds):
        A = cls(nrows, ncols, **kwds)
        A.set_iterable(it)
        return A

So cls is supposed to be a reference to the IntegerMatrix class, right?  It is, in python 3.8 and earlier, but not in python 3.9.  GDB says:

(gdb) print *__pyx_v_cls
$1 = {ob_base = {ob_base = {ob_refcnt = 54, 
      ob_type = 0x7ffff7dd52e0 <PyType_Type>}, ob_size = 0}, 
  tp_name = 0x7ffff7cdf35b "type", tp_basicsize = 872, tp_itemsize = 40, 
  tp_dealloc = 0x7ffff7c68030 <type_dealloc>, tp_vectorcall_offset = 400, 
  tp_getattr = 0x0, tp_setattr = 0x0, tp_as_async = 0x0, 
  tp_repr = 0x7ffff7c7cca0 <type_repr>, tp_as_number = 0x0, 
  tp_as_sequence = 0x0, tp_as_mapping = 0x0, 
  tp_hash = 0x7ffff7c03c30 <_Py_HashPointer>, 
  tp_call = 0x7ffff7bd79c0 <type_call>, tp_str = 0x7ffff7c16640 <object_str>, 
  tp_getattro = 0x7ffff7bd70e0 <type_getattro>, 
  tp_setattro = 0x7ffff7c547b0 <type_setattro>, tp_as_buffer = 0x0, 
  tp_flags = 2148293632, 
  tp_doc = 0x7ffff7d42600 "type(object_or_name, bases, dict)\ntype(object) -> the object's type\ntype(name, bases, dict) -> a new type", 
  tp_traverse = 0x7ffff7be5060 <type_traverse>, 
  tp_clear = 0x7ffff7c67f20 <type_clear>, 
  tp_richcompare = 0x7ffff7bf6e50 <object_richcompare>, 
  tp_weaklistoffset = 368, tp_iter = 0x0, tp_iternext = 0x0, 
  tp_methods = 0x7ffff7dbdee0, tp_members = 0x7ffff7dbdda0, 
  tp_getset = 0x7ffff7dbdc20, tp_base = 0x7ffff7dd5620 <PyBaseObject_Type>, 
  tp_dict = 0x7ffff78c2700, tp_descr_get = 0x0, tp_descr_set = 0x0, 
  tp_dictoffset = 264, tp_init = 0x7ffff7c45a60 <type_init>, 
  tp_alloc = 0x7ffff7bc31a0 <PyType_GenericAlloc>, 
  tp_new = 0x7ffff7bd7b00 <type_new>, 
  tp_free = 0x7ffff7bc5500 <PyObject_GC_Del>, 
  tp_is_gc = 0x7ffff7bc6370 <type_is_gc>, tp_bases = 0x7ffff78bf0a0, 
  tp_mro = 0x7ffff78c2e80, tp_cache = 0x0, tp_subclasses = 0x7ffff78299c0, 
  tp_weaklist = 0x7ffff78c0db0, tp_del = 0x0, tp_version_tag = 3, 
  tp_finalize = 0x0, tp_vectorcall = 0x0}

The code invokes that, expecting to get the IntegerMatrix constructor, and instead invokes the type() function.  The code generated by Cython in the python 3.8 and 3.9 cases differs only in strings that contain "3.8" in the first case and "3.9" in the second.  In Objects/classobject.c from python 3.9, there is a function named method_vectorcall starting at line 36.  We are going down the "then" branch of the conditional starting at line 48.  We set the first argument, which should be the class object, to "self", which is set on line 43:

    PyObject *self = PyMethod_GET_SELF(method);

GDB shows that self is the type() function.  The "method" object is:
$2 = {ob_base = {ob_refcnt = 1, ob_type = 0x7ffff7ddcb60 <PyMethod_Type>}, 
  im_func = <cython_function_or_method at remote 0x7ffff5c78f40>, 
  im_self = <type at remote 0x7ffff7dd52e0>, im_weakreflist = 0x0, 
  vectorcall = 0x7ffff7be5ed0 <method_vectorcall>}

where im_self is, indeed, the type() function.  The only place I can find where im_self is set is line 114 of Objects/classobject.c, in PyMethod_New:

PyObject *
PyMethod_New(PyObject *func, PyObject *self)
{
    if (self == NULL) {
        PyErr_BadInternalCall();
        return NULL;
    }
    PyMethodObject *im = PyObject_GC_New(PyMethodObject, &PyMethod_Type);
    if (im == NULL) {
        return NULL;
    }
    im->im_weakreflist = NULL;
    Py_INCREF(func);
    im->im_func = func;
    Py_INCREF(self);
    im->im_self = self;  // <-------- This is the only place I see where im_self is set.
    im->vectorcall = method_vectorcall;
    _PyObject_GC_TRACK(im);
    return (PyObject *)im;
}

I'm not sure how to follow the trail back from here.  I'm pretty certain that setting a breakpoint on PyMethod_New is going to drive me to madness.

This sounds like: https://bugzilla.redhat.com/show_bug.cgi?id=1793612#c4

Let me try to see whether this is fixed on Cython master.

It is.

class C:
    @classmethod
    def classmethod1(cls):
        pass

    @classmethod
    def classmethod2(cls):
        cls.classmethod1()


Cython 0.29.15:

$ python -c "import reproducer; reproducer.C.classmethod2()"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "reproducer.pyx", line 8, in reproducer.C.classmethod2
    cls.classmethod1()
AttributeError: type object 'type' has no attribute 'classmethod1'


Cython master:

$ python -c "import reproducer; reproducer.C.classmethod2()"
(no output, no error)



OK, let me fins the commit that fixed this.

That is most likely https://github.com/cython/cython/pull/3106

Thanks, Miro.  I'm glad you recognized that bug.

This comment is mass posted to all bugs blocking the Python 3.9 tracker, sorry if it is not 100 % relevant. When in doubt, please ask.


The Python 3.9 rebuild is in progress in a Koji side tag.

If you fix this bug, please don't rebuild the package in regular rawhide, but do it in the side tag with:

    $ fedpkg build --target=f33-python

The rebuild is progressing slowly and it is possible this package won't have all the required build dependencies yet. If that's the case, please just leave the fix committed and pushed and we will eventually rebuild it for you.

You are not asked to go and try rebuild all the missing dependencies yourself. If you know there is a bootstrap loop in the dependencies, let me know and we can untangle it together.

If you want to test your fix or reproduce the failure, you can still use the Copr repo mentioned in the initial comment of this bug: https://copr.fedorainfracloud.org/coprs/g/python/python3.9/

Python 3.9 update: The f33-python side tag is currently being merged.

New builds in f33-python are no longer possible, but python3 is not yet updated to Python 3.9 in rawhide. You can check when Python is Python 3.9 with:

    $ koji wait-repo f33-build --build python3.9-3.9.0~b1-3.fc3

And build the packages normally after that.

This was actually a cython bug and has since been fixed.  This package has been built in Rawhide post-merge.  Closing.