Bug 2231751

Summary: Python 3.12 ctypes regression
Product: [Fedora] Fedora Reporter: Nathan Scott <nathans>
Component: python3.12Assignee: Victor Stinner <vstinner>
Status: NEW --- QA Contact:
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 39CC: cstratak, mhroncok, python-maint, python-packagers-sig, thrnciar, torsava, vstinner
Target Milestone: ---Keywords: Regression, Upgrades
Target Release: ---   
Hardware: All   
OS: Linux   
URL: https://performancecopilot.github.io/qa-reports/reports/20230812_213907-b3cc0288/#suites/3d55d380e43a65924702ab7b74fce31b/8e114a031acf49ef/
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Nathan Scott 2023-08-13 22:43:09 UTC
We are observing a significant number of new failures in the PCP test suite on rawhide with the recent python update there.

It's unfortunately taken us this long to report due to problems with python3.12 making python3-pyodbc uninstallable, which caused dependent PCP sub-packages to become uninstallable, on rawhide.  We've had to (temporarily?) disable PCP metrics based on that module (pcp-pmda-mssql) in order to observe the remaining fallout.

Anyway, the ctypes issues seem to have two similar but slightly different signatures:

>     desc.contents.type, PM_TYPE_U64)
>     ^^^^^^^^^^^^^^^^^^
> AttributeError: 'dict' object has no attribute 'type'


and

>     if desc.contents.indom == pmapi.c_api.PM_INDOM_NULL:
>        ^^^^^^^^^^^^^
> ValueError: Unexpected NULL pointer in _objects


This ctypes structure for variable 'desc' (pmDesc) is here:
https://github.com/performancecopilot/pcp/blob/83cf926e507ab1302c18663daa4ce7d2129b99a1/src/python/pcp/pmapi.py#L600

These failure modes can be observed from daily PCP QA:
https://github.com/performancecopilot/pcp/actions
(click on a current "QA" action, then "fedora-rawhide-container", then the "QA" step therein - there's many examples of the above two failure signatures if you need a reproducible test case)

And the test cases are available in rawhide via pcp-testsuite package,
which puts failing cases (such as qa/991) below /var/lib/pcp/testsuite

Please let me know if any further information is needed.  FWIW there's been no change to PCP python wrapper library recently, and these tests pass on every other version of python (incl. python2) so we're 100% certain this is directly related to the python3.12 upgrade.

Reproducible: Always

Steps to Reproduce:
Run PCP python tools e.g. via pcp-testsuite
Actual Results:  
Numerous tests fail due to crashes in python ctypes code.

Expected Results:  
Tests pass, python tools using ctypes do not fail.

Comment 1 Fedora Release Engineering 2023-08-16 08:07:09 UTC
This bug appears to have been reported against 'rawhide' during the Fedora Linux 39 development cycle.
Changing version to 39.