Note: This bug is displayed in read-only format because
the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Created attachment 1266893[details]
backtrace of hanging anaconda process
The issue started to appear as a race condition hit when running our local kickstart tests [2] after Gtk rebase during 7.4 development [1], when PyGIWarnings for importing without version specified started to appear. The warnings should be handled (disappear) in scope of bug 1433943 so I think there is high chance the freezing issue will disappear with the warnings, but I am filing this bug for the record as we might try to fix the root cause or refer to the bug when we hit the issue later/again. As can be seen from the attached backtrace, Thread 1 and Thread 4 are in a deadlock.
Thread 1 (main) is holding import lock when importing datetime spoke which triggers PyGIWarning on import from gi repository. The warning is passed to anaconda logging handler which is waiting for logging lock held by Thread 4.
Thread 4 (storage initialization thread) is logging from blivet, a Size object being part of the message. Formatting of the Size object value involves translation which triggers import of locale.normalize in a function of gettext python module which is waiting to acquire import lock held by thread 1.
(Thread 3 is waiting for the logging lock, Thread 2 is waiting for Thread 4 to join)
We are using Python 2.7. For newer versions of Python the issue would be mitigated or fixed by two changes in Python:
- removing lazy/local import "from locale import normalize" from gettext.py (seems to be have been there only for the reason of locale name conflict) - https://github.com/python/cpython/commit/31e87203248047aa99ddbb4165b3b20c758196d8
- per-module import lock in Python 3.3 - https://github.com/python/cpython/blob/d5adb7f65d30afd00921e6c22e9e2b8c323c058d/Doc/whatsnew/3.3.rst#a-finer-grained-import-lock
[1] eg http://download.eng.brq.redhat.com/pub/rhel/nightly/RHEL-7.4-20170323.n.0/compose/Server/x86_64/os/
[2] see https://bugzilla.redhat.com/show_bug.cgi?id=1431618#c0 for details on the environment of kickstart-tests where I am seeing the deadlock
Any advice / recommendation from Python side please?
Comment 3Charalampos Stratakis
2017-03-28 11:04:20 UTC
I don't have free cycles this week to check that, but some days ago I stumbled upon this upstream issue which might be relevant [0]. Could you take a look?
[0] http://bugs.python.org/issue6721
(In reply to Radek Vykydal from comment #0)
> Created attachment 1266893[details]
> backtrace of hanging anaconda process
>
> The issue started to appear as a race condition hit when running our local
> kickstart tests [2] after Gtk rebase during 7.4 development [1], when
> PyGIWarnings for importing without version specified started to appear. The
> warnings should be handled (disappear) in scope of bug 1433943 so I think
> there is high chance the freezing issue will disappear with the warnings,
The warnings should disappear in anaconda-21.48.22.107-1
(https://bugzilla.redhat.com/show_bug.cgi?id=1433943#c3)
Comment 6Samantha N. Bueno
2017-05-10 08:52:29 UTC
*** Bug 1437836 has been marked as a duplicate of this bug. ***
(In reply to Radek Vykydal from comment #5)
> (In reply to Radek Vykydal from comment #0)
> > Created attachment 1266893[details]
> > backtrace of hanging anaconda process
> >
> > The issue started to appear as a race condition hit when running our local
> > kickstart tests [2] after Gtk rebase during 7.4 development [1], when
> > PyGIWarnings for importing without version specified started to appear. The
> > warnings should be handled (disappear) in scope of bug 1433943 so I think
> > there is high chance the freezing issue will disappear with the warnings,
>
> The warnings should disappear in anaconda-21.48.22.107-1
> (https://bugzilla.redhat.com/show_bug.cgi?id=1433943#c3)
Marking as a duplicate of the bug 1433943.
*** This bug has been marked as a duplicate of bug 1433943 ***