Bug 2030621

FYI it prevents us from upgrading the Koji builders to F35, because Koji runs on mod_wsgi, and it has a fedora-messaging plugin which uses Twisted which uses attrs.

> However, if you try to run it in mod_wsgi...

Thanks for the code snippet, but *how* do you run it exactly?

From our conversation on IRC:

$ cat /etc/httpd/conf.d/essai.conf 
Alias /essai /var/www/cgi-bin/essai/essai.py
 
<Directory "/var/www/cgi-bin/essai">
    Options ExecCGI
    SetHandler wsgi-script
    Require all granted
</Directory>


$ cat /var/www/cgi-bin/essai/essai.py 
import attr
 
@attr.s
class TestClass:
    foo = attr.ib()
 
 
print(attr.s, repr(attr.s), attr.define, repr(attr.define))
 
def application(environ, start_response):
    response = "Debugging.\n"
    response = response.encode()
    headers = [
        ('Content-Length', str(len(response))),
        ('Content-Type', "text/plain"),
    ]
    start_response('200 OK', headers)
    TestClass("debug")
    return [response]


$ sudo systemctl restart httpd

Go to http://localhost/essai and observe the Internal Server Error.

$ sudo cat /var/log/httpd/error_log
...
[Thu Dec 09 11:46:22.823867 2021] [wsgi:error] [pid 667962:tid 668079] <function attrs at 0x7f61fc29e3b0> <function attrs at 0x7f61fc29e3b0> <function define at 0x7f61fc0c5240> <function define at 0x7f61fc0c5240>
[Thu Dec 09 11:46:22.823946 2021] [wsgi:error] [pid 667962:tid 668079] [client 127.0.0.1:33668] mod_wsgi (pid=667962): Exception occurred processing WSGI script '/var/www/cgi-bin/essai/essai.py'.
[Thu Dec 09 11:46:22.824269 2021] [wsgi:error] [pid 667962:tid 668079] [client 127.0.0.1:33668] Traceback (most recent call last):
[Thu Dec 09 11:46:22.824284 2021] [wsgi:error] [pid 667962:tid 668079] [client 127.0.0.1:33668]   File "/var/www/cgi-bin/essai/essai.py", line 18, in application
[Thu Dec 09 11:46:22.824287 2021] [wsgi:error] [pid 667962:tid 668079] [client 127.0.0.1:33668]     TestClass("debug")
[Thu Dec 09 11:46:22.824295 2021] [wsgi:error] [pid 667962:tid 668079] [client 127.0.0.1:33668] TypeError: TestClass() takes no arguments

Here's a simpler reproducer that does not involve attrs:

def tweak_init(cls):
    def new_init(inst, name):
        inst.text = f"Hello, {name}\n"

    cls.__init__ = new_init
    return cls

@tweak_init
class Greeting:
    pass


def application(environ, start_response):
    response = "Debugging.\n"
    response = response.encode()
    headers = [
        ('Content-Length', str(len(response))),
        ('Content-Type', "text/plain"),
    ]
    start_response('200 OK', headers)
    Greeting("debug")
    return [response]

Interesting:

(Pdb) Greeting
<class '_mod_wsgi_bd7b5f024fe2438c2a653d79710eba24.Greeting'>
(Pdb) Greeting.__init__
<function tweak_init.<locals>.new_init at 0x7ff79c0d7640>
(Pdb) Greeting.__init__(object(), "a")
*** AttributeError: 'object' object has no attribute 'text'
(Pdb) Greeting("a")
*** TypeError: Greeting() takes no arguments
(Pdb) Greeting.__call__("a")
*** TypeError: Greeting() takes no arguments


Seems like the __init__ attribute is set, but __call__ keep calling the previous one.

This behaves similarly:


class Greeting:
    pass

def new_init(inst, name):
    inst.text = f"Hello, {name}\n"
 
Greeting.__init__ = new_init



(Pdb) Greeting("a")
*** TypeError: Greeting() takes no arguments

(Pdb) Greeting.__init__(object(), "a")
*** AttributeError: 'object' object has no attribute 'text'

Inspect things it takes the arg:

(Pdb) import inspect
(Pdb) inspect.signature(Greeting)
<Signature (name)>
(Pdb) Greeting("a")
*** TypeError: Greeting() takes no arguments



Setting any __init__ before replacing it works:

class Greeting():
    __init__ = None  # <- define whatever __init__ here and it works

def new_init(inst, name):
    inst.text = f"Hello, {name}\n"

Greeting.__init__ = new_init

This is useful: https://modwsgi.readthedocs.io/en/develop/user-guides/debugging-techniques.html#python-interactive-debugger

I don't understand how mod_wsgi is involved the issue. Moreover, the following code works properly with Python 3.9 and 3.10, and it doesn't seem to be a Python 3.10 regression:
---
def new_init(inst, name):
    inst.text = f"Hello, {name}\n"
    print("new_init")

def tweak_init(cls):
    cls.__init__ = new_init
    return cls

@tweak_init
class Greeting:
    pass

class Greeting2:
    pass
Greeting2.__init__ = new_init

print(Greeting("name"))
print(Greeting2("name"))
---

Python 3.9 and 3.10 output:
---
new_init
<__main__.Greeting object at 0x7fd5db4b4eb0>
new_init
<__main__.Greeting2 object at 0x7fd5db4b4eb0>
---

It breaks in mod_wsgi only.

For the record, Python 3.10.1 does not change this.

This is sub-interpreter related:


>>> import _testcapi
>>> code = r"""class Greeting():
...     pass
... 
... def new_init(inst, name):
...     inst.text = f"Hello, {name}\n"
... 
... Greeting.__init__ = new_init
... 
... Greeting("Miro")"""
>>> _testcapi.run_in_subinterp(code)
Traceback (most recent call last):
  File "<string>", line 9, in <module>
TypeError: Greeting() takes no arguments
-1

Reported upstream as https://bugs.python.org/issue46034

> Reported upstream as https://bugs.python.org/issue46034

I confirm that it's a Python 3.10 regression, I marked issue46034 as a duplicate of: https://bugs.python.org/issue46006

For the mod_wsgi case, a workaround is to avoid subinterpreters by changing the mod_wsgi configuration. Sorry, I'm not sure when and how mod_wsgi decides to use subinterpreters or not. It seems like "WSGIApplicationGroup %{GLOBAL}" directive should be used somehow to run all code in the same interpreter.

See for example:

* https://modwsgi.readthedocs.io/en/develop/user-guides/processes-and-threading.html#python-sub-interpreters
* https://modwsgi.readthedocs.io/en/develop/user-guides/checking-your-installation.html#sub-interpreter-being-used

If I provide you a build with this fix applied https://github.com/python/cpython/pull/30425 -- will you be able to test it?

Rawhide pull request: https://src.fedoraproject.org/rpms/python3.10/pull-request/90
Fedora 35 pull request: https://src.fedoraproject.org/rpms/python3.10/pull-request/91

If the scratch build succeeds there, you can download it and test it. Thanks

I guess I never added myself to cc here. ;) 

So, I pulled the scratch build and it seems to work in stg koji. Many thanks!

FEDORA-2022-23b8acee83 has been submitted as an update to Fedora 36. https://bodhi.fedoraproject.org/updates/FEDORA-2022-23b8acee83

FEDORA-2022-23b8acee83 has been pushed to the Fedora 36 stable repository.
If problem still persists, please make note of it in this bug report.

Reopening for F35.

FEDORA-2022-2303439072 has been submitted as an update to Fedora 35. https://bodhi.fedoraproject.org/updates/FEDORA-2022-2303439072

FEDORA-2022-2303439072 has been pushed to the Fedora 35 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2022-2303439072`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-2303439072

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

FEDORA-2022-2303439072 has been pushed to the Fedora 35 stable repository.
If problem still persists, please make note of it in this bug report.