Bug 2034962

Summary: Regression: Broken subinterpreters affects Kodi and YouTube addon and others
Product: [Fedora] Fedora Reporter: Michael Cronenworth <mike>
Component: python3.10Assignee: Miro Hrončok <mhroncok>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 35CC: cstratak, kjell.m.randa, mathieu, mhroncok, python-maint, python-sig, thedatum+bz, thrnciar, torsava, vstinner
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: python3.10-3.10.1-3.fc36 python3.10-3.10.1-3.fc35 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-01-16 01:18:45 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Michael Cronenworth 2021-12-22 15:15:23 UTC
Description of problem:
Kodi crashes at start.

kodi.log snippet:
ERROR <general>: Exception ignored deletion of interned string failed
ERROR <general>: :
ERROR <general>: Traceback (most recent call last):
ERROR <general>:   File "/usr/lib64/python3.10/sqlite3/dbapi2.py", line 83, in register_adapters_and_converters
ERROR <general>:
ERROR <general>: register_converter("timestamp", convert_timestamp)
ERROR <general>:
ERROR <general>: KeyError
ERROR <general>: :
ERROR <general>: 'timepart_full'
ERROR <general>:


Version-Release number of selected component (if applicable):
python3.10-3.10.1-1.fc35


How reproducible: Always


Steps to Reproduce:
1. Install RPMFusion repo
2. Install Kodi
3. Install YouTube add-on inside Kodi

Actual results:
Kodi crashes after add-on install and then at Kodi start.


Expected results:
No crash.


Additional info:
Upstream is working on the problem.
https://bugs.python.org/issue46070

Downgrading to Python 3.10.0 or uninstalling the YouTube add-on is a temporary workaround.

Comment 1 Michael Cronenworth 2021-12-22 15:20:42 UTC
There is a patch in the Arch bug tracking system that users report fixes it, but I have not tested it.

https://bugs.archlinux.org/task/73002
https://bugs.archlinux.org/task/73002?getfile=20955

Comment 2 Victor Stinner 2022-01-04 15:02:38 UTC
The "Exception ignored deletion of interned string failed" error comes from the unicode_dealloc() function when trying to deleted a "mortal" interned string.
            
    case SSTATE_INTERNED_MORTAL:                                    
    {                                                               
        struct _Py_unicode_state *state = get_unicode_state();      
        /* Revive the dead object temporarily. PyDict_DelItem() removes two
           references (key and value) which were ignored by         
           PyUnicode_InternInPlace(). Use refcnt=3 rather than refcnt=2
           to prevent calling unicode_dealloc() again. Adjust refcnt after
           PyDict_DelItem(). */                                     
        assert(Py_REFCNT(unicode) == 0);                            
        Py_SET_REFCNT(unicode, 3);                                  
        if (PyDict_DelItem(state->interned, unicode) != 0) {        
            _PyErr_WriteUnraisableMsg("deletion of interned string failed",
                                      NULL);                        
        }                                                           
        assert(Py_REFCNT(unicode) == 1);                            
        Py_SET_REFCNT(unicode, 0);                                  
        break;                                                      
    }                                                               

It seems like Kodi uses Python subinterpreters (maybe to execute pluggins).

Well, this issue is now tracked upstream: https://bugs.python.org/issue46070

Comment 3 Victor Stinner 2022-01-05 16:02:04 UTC
Michael: I analyzed https://bugs.python.org/issue46070 and it seems like it's a Python 3.8 regression. Can you confirm that Kodi works on Python 3.7 and older, but crash on Python 3.8 and newer?

Does someone know if Kodi always ran pluggins in subintepreters, or it's a recent Kodi change?

Comment 4 Michael Cronenworth 2022-01-05 16:12:05 UTC
I've provided this information in the OP.

3.10.0 - no crash
3.10.1 - crash

This is exclusively a 3.10.1 issue. Kodi has not changed in any way.

Comment 5 Kjell Randa 2022-01-10 13:45:22 UTC
I see the same on Fedora 35 witk a Kodi installation compiled locally.

Downgrading python from

python3-3.10.1-2.fc35.x86_64
to
python3-3.10.0-1.fc35.x86_64

and Kodi seems to run just fine without the crash at startup.

Kodi information:
Starting Kodi (19.2 (19.2.0) Git:20220104-nogitfound). Platform: Linux x86 64-bit
2022-01-10 14:34:23.768 T:40472    INFO <general>: Using Release Kodi x64
2022-01-10 14:34:23.768 T:40472    INFO <general>: Kodi compiled 2022-01-04 by GCC 11.2.1 for Linux x86 64-bit version 5.15.4 (331524)
2022-01-10 14:34:23.768 T:40472    INFO <general>: Running on Fedora Linux 35 (Thirty Five), kernel: Linux x86 64-bit version 5.15.12-200.fc35.x86_64
2022-01-10 14:34:23.768 T:40472    INFO <general>: FFmpeg version/source: 4.4.1
2022-01-10 14:34:23.768 T:40472    INFO <general>: Host CPU: Intel(R) Core(TM) i5-7260U CPU @ 2.20GHz, 4 cores available

Comment 6 Miro Hrončok 2022-01-10 14:10:51 UTC
If I provide a build with this fix applied: https://github.com/python/cpython/pull/30453 -- will you be able to test it?

Comment 7 Michael Cronenworth 2022-01-10 14:13:18 UTC
I can test a build. Scratch build, whatever build. Thanks, Miro.

Comment 8 Miro Hrončok 2022-01-10 14:27:57 UTC
Rawhide pull request: https://src.fedoraproject.org/rpms/python3.10/pull-request/90
Fedora 35 pull request: https://src.fedoraproject.org/rpms/python3.10/pull-request/91

If the scratch build succeeds there, you can download it and test it. Thanks

Comment 9 Michael Cronenworth 2022-01-10 15:18:52 UTC
Initial testing looks good on F35 + x86_64. No crash on a fresh Kodi / YouTube plugin install. Thanks, Miro.

Comment 10 Victor Stinner 2022-01-10 16:02:52 UTC
It seems like there is a second bug which is not fixed yet, Python 3.9 and newer are affected: https://bugs.python.org/issue46070#msg409991

I may have to revert the https://github.com/python/cpython/commit/7247407c35330f3f6292f1d40606b7ba6afd5700 change. I'm not sure why, but the bug was only seen on Windows. Right now, the bug can only be reproduced with "import _sre" on Windows with Python 3.9.

Comment 11 Kjell Randa 2022-01-10 16:09:51 UTC
Installed the needed 3.10.1-3.fc35.x86_64.rpm packages on my F35 x86_64 and Kodi starts without crashing.
Thanks for your work.

Comment 12 Miro Hrončok 2022-01-10 21:43:20 UTC
(In reply to Victor Stinner from comment #10)
> It seems like there is a second bug which is not fixed yet, Python 3.9 and
> newer are affected: https://bugs.python.org/issue46070#msg409991
> 
> I may have to revert the
> https://github.com/python/cpython/commit/
> 7247407c35330f3f6292f1d40606b7ba6afd5700 change. I'm not sure why, but the
> bug was only seen on Windows. Right now, the bug can only be reproduced with
> "import _sre" on Windows with Python 3.9.

So, Linux is not affected and I can proceed with the backport as is, correct?

Comment 13 Victor Stinner 2022-01-11 10:20:02 UTC
Kjell Randa confirmed that the interned string revert fix the Kodi crash, so it's enough to close this issue.

I still care about https://bugs.python.org/issue46070 because in theory, the regression can cause crash on Linux as well. It seems like in practice only Python 3.9 on Windows is affected. This part is handlded upstream, there is no need to track it downstream.

Comment 14 Fedora Update System 2022-01-12 19:55:14 UTC
FEDORA-2022-23b8acee83 has been submitted as an update to Fedora 36. https://bodhi.fedoraproject.org/updates/FEDORA-2022-23b8acee83

Comment 15 Fedora Update System 2022-01-12 19:58:21 UTC
FEDORA-2022-23b8acee83 has been pushed to the Fedora 36 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 16 Miro Hrončok 2022-01-12 20:02:18 UTC
Reopening for F35.

Comment 17 Fedora Update System 2022-01-12 20:03:40 UTC
FEDORA-2022-2303439072 has been submitted as an update to Fedora 35. https://bodhi.fedoraproject.org/updates/FEDORA-2022-2303439072

Comment 18 Fedora Update System 2022-01-13 01:12:01 UTC
FEDORA-2022-2303439072 has been pushed to the Fedora 35 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2022-2303439072`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-2303439072

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 19 Victor Stinner 2022-01-13 21:29:23 UTC
While it's not directly related to this bug, FYI I fixed https://bugs.python.org/issue46070

Comment 20 Fedora Update System 2022-01-16 01:18:45 UTC
FEDORA-2022-2303439072 has been pushed to the Fedora 35 stable repository.
If problem still persists, please make note of it in this bug report.