Bug 2012513
| Summary: | python3.10-3.10.0-2.fc36 breaks anaconda/pykickstart on Cloud composes | ||||||
|---|---|---|---|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | Kevin Fenzi <kevin> | ||||
| Component: | anaconda | Assignee: | Anaconda Maintenance Team <anaconda-maint-list> | ||||
| Status: | CLOSED RAWHIDE | QA Contact: | Fedora Extras Quality Assurance <extras-qa> | ||||
| Severity: | unspecified | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | rawhide | CC: | anaconda-maint-list, awilliam, cstratak, jonathan, kellin, mhroncok, python-maint, python-sig, thrnciar, torsava, vanmeeuwen+fedora, vponcova, vslavik, vstinner, w | ||||
| Target Milestone: | --- | ||||||
| Target Release: | --- | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | anaconda-36.6-1.fc36 | Doc Type: | If docs needed, set a value | ||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2021-10-12 15:45:43 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
|
Description
Kevin Fenzi
2021-10-09 23:27:21 UTC
Created attachment 1831291 [details]
traceback in journal on Cloud-Base image build failure in f36
> I untagged python3.10-3.10.0-2.fc36 and re-ran the compose and it completed ok.
If you did, better also untag python-pip-21.2.3-3.fc36 and the newer python-pip-21.2.3-4.fc36, as python-pip-21.2.3-3.fc36 and python3.10-3.10.0-2.fc36 was a joined update and untagging just one of it will break things.
I will have a look early next week why this fails.
This works: $ python3 Python 3.10.0 (default, Oct 5 2021, 00:00:00) [GCC 11.2.1 20210728 (Red Hat 11.2.1-1)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from pykickstart import version >>> version.returnClassForVersion() <class 'f35.F35Handler'> ----------- Does this script run in the process? https://github.com/rhinstaller/anaconda/blob/master/dracut/python-deps Try replacing: sitedir = sysconfig.get_path('purelib') With: sitedir = sysconfig.get_path('purelib', scheme='rpm_prefix') And let me know if that changes anything. (In reply to Miro Hrončok from comment #2) > > I untagged python3.10-3.10.0-2.fc36 and re-ran the compose and it completed ok. > > If you did, better also untag python-pip-21.2.3-3.fc36 and the newer > python-pip-21.2.3-4.fc36, as python-pip-21.2.3-3.fc36 and > python3.10-3.10.0-2.fc36 was a joined update and untagging just one of it > will break things. Done. I believe the script in https://github.com/rhinstaller/anaconda/blob/master/dracut/python-deps tries to figure out what Python files are needed to be in initramfs by something (what?), but instead of going through Python directories in sys.path, it goes trough: - directory that should be used to install pure Python modules - directory where standard Python library was installed Note that until python3.10-3.10.0-2.fc36 this somewhat worked, but it was never the right thing to do. If I understand the script, it tries to locate installed Python modules, not a place to install them. A possible real fix is to go through sys.path entries instead. If the entry startswith sys.prefix and is an existing directory, it should be included in this search. Note however that alsoNeeded = {sitedir+"/requests/__init__.py": libdir+"/stringprep.py"} will need to be adapted to use the rpm's sitelib and not the "general" one. Possible quick workarounds: 1. Use sysconfig.get_path('purelib', scheme='rpm_prefix') - only works on Fedora 36+, but can be try-excepted for KeyError 2. set RPM_BUILD_ROOT=/ when invoking the script to fool sysconfig to think we are building RPM packages Let me know if you want a pull request for the proper fix. We broke it, we can fix it. The context about change in python3.10-3.10.0-2.fc36: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org/thread/AAGUFQZ4RZDU7KUN4HA43KQJCMSFR3GW/ Output of (sorted) /usr/lib/dracut/modules.d/80anaconda/python-deps /usr/lib/dracut/modules.d/80anaconda/parse-kickstart /usr/lib/dracut/modules.d/80anaconda/driver_updates.py with python3.10-3.10.0-1.fc36 -> python3.10-3.10.0-2.fc36: --- before 2021-10-11 11:26:38.411721326 +0200 +++ after 2021-10-11 11:26:42.643743186 +0200 @@ -5,8 +5,6 @@ /usr/lib/python3.10/site-packages/charset_normalizer/api.py /usr/lib/python3.10/site-packages/charset_normalizer/assets/__init__.py /usr/lib/python3.10/site-packages/charset_normalizer/cd.py -/usr/lib/python3.10/site-packages/charset_normalizer/cli/__init__.py -/usr/lib/python3.10/site-packages/charset_normalizer/cli/normalizer.py /usr/lib/python3.10/site-packages/charset_normalizer/constant.py /usr/lib/python3.10/site-packages/charset_normalizer/legacy.py /usr/lib/python3.10/site-packages/charset_normalizer/md.py @@ -14,8 +12,6 @@ /usr/lib/python3.10/site-packages/charset_normalizer/utils.py /usr/lib/python3.10/site-packages/charset_normalizer/version.py /usr/lib/python3.10/site-packages/idna/__init__.py -/usr/lib/python3.10/site-packages/idna/codec.py -/usr/lib/python3.10/site-packages/idna/compat.py /usr/lib/python3.10/site-packages/idna/core.py /usr/lib/python3.10/site-packages/idna/idnadata.py /usr/lib/python3.10/site-packages/idna/intranges.py @@ -101,46 +97,6 @@ /usr/lib/python3.10/site-packages/pykickstart/errors.py /usr/lib/python3.10/site-packages/pykickstart/handlers/__init__.py /usr/lib/python3.10/site-packages/pykickstart/handlers/control.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/f10.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/f11.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/f12.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/f13.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/f14.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/f15.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/f16.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/f17.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/f18.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/f19.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/f20.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/f21.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/f22.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/f23.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/f24.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/f25.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/f26.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/f27.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/f28.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/f29.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/f30.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/f31.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/f32.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/f33.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/f34.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/f35.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/f7.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/f8.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/f9.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/fc3.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/fc4.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/fc5.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/fc6.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/rhel3.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/rhel4.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/rhel5.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/rhel6.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/rhel7.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/rhel8.py -/usr/lib/python3.10/site-packages/pykickstart/handlers/rhel9.py /usr/lib/python3.10/site-packages/pykickstart/i18n.py /usr/lib/python3.10/site-packages/pykickstart/ko.py /usr/lib/python3.10/site-packages/pykickstart/load.py @@ -158,7 +114,6 @@ /usr/lib/python3.10/site-packages/requests/compat.py /usr/lib/python3.10/site-packages/requests/cookies.py /usr/lib/python3.10/site-packages/requests/exceptions.py -/usr/lib/python3.10/site-packages/requests/help.py /usr/lib/python3.10/site-packages/requests/hooks.py /usr/lib/python3.10/site-packages/requests/models.py /usr/lib/python3.10/site-packages/requests/packages.py @@ -174,13 +129,7 @@ /usr/lib/python3.10/site-packages/urllib3/connectionpool.py /usr/lib/python3.10/site-packages/urllib3/contrib/__init__.py /usr/lib/python3.10/site-packages/urllib3/contrib/_appengine_environ.py -/usr/lib/python3.10/site-packages/urllib3/contrib/_securetransport/__init__.py -/usr/lib/python3.10/site-packages/urllib3/contrib/_securetransport/bindings.py -/usr/lib/python3.10/site-packages/urllib3/contrib/_securetransport/low_level.py -/usr/lib/python3.10/site-packages/urllib3/contrib/appengine.py -/usr/lib/python3.10/site-packages/urllib3/contrib/ntlmpool.py /usr/lib/python3.10/site-packages/urllib3/contrib/pyopenssl.py -/usr/lib/python3.10/site-packages/urllib3/contrib/securetransport.py /usr/lib/python3.10/site-packages/urllib3/contrib/socks.py /usr/lib/python3.10/site-packages/urllib3/exceptions.py /usr/lib/python3.10/site-packages/urllib3/fields.py Hence, my assumptions are correct, the traceback on the attached screenshot is caused by this. I've also confiremd that:
sitedir = sysconfig.get_path('purelib', scheme='rpm_prefix')
works as a viable workaround.
Hi Miro! Yes, it finds what files are needed in initramfs/dracut to run the scripts passed as args. This is how it runs: https://github.com/rhinstaller/anaconda/blob/3d9f46435b6cdc89f59eda173468428cf44f26d0/dracut/module-setup.sh#L74 I think we'd all be very happy to have a patch that solves this properly, so if you know how to do this, yes please! In the meantime, I'm adding the workaround as a PR to ask/nack/whatever: https://github.com/rhinstaller/anaconda/pull/3644 (In reply to Miro Hrončok from comment #5) > Let me know if you want a pull request for the proper fix. We broke it, we > can fix it. I have it ready, writing a commit message now... Kevin, if I give you the fix as a scratchbuild, is it possible to test the compose? Alas, not easily. Adam might have something with openqa that could do it though? Never mind, the fix was merged to anaconda upstream and I was told we will have that in rawhide in a couple minutes. Could you please re-tag the latest python3.10 and python-pip builds back? That would be python3.10-3.10.0-2.fc36 and python-pip-21.2.3-4.fc36. (In reply to Miro Hrončok from comment #13) > Could you please re-tag the latest python3.10 and python-pip builds back? > That would be python3.10-3.10.0-2.fc36 and python-pip-21.2.3-4.fc36. This was done. "Alas, not easily. Adam might have something with openqa that could do it though?" For the record, I can do this *for stable or branched releases*, for Rawhide it's substantially harder as we don't have things set up to run the updates tests on Rawhide, only nightly compose tests. |