Bug 1370484 - RFC: Python 3.3 logging QueueListener drops log messages
Summary: RFC: Python 3.3 logging QueueListener drops log messages
Keywords:
Status: CLOSED EOL
Alias: None
Product: Red Hat Software Collections
Classification: Red Hat
Component: python
Version: rh-python34
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
: 3.0
Assignee: Python Maintainers
QA Contact: BaseOS QE - Apps
URL:
Whiteboard:
Depends On:
Blocks: 1449573
TreeView+ depends on / blocked
 
Reported: 2016-08-26 13:00 UTC by Paulo Andrade
Modified: 2021-03-11 14:40 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1449573 (view as bug list)
Environment:
Last Closed: 2017-03-31 15:02:02 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
test.py (4.34 KB, text/plain)
2016-08-26 13:00 UTC, Paulo Andrade
no flags Details

Description Paulo Andrade 2016-08-26 13:00:07 UTC
Created attachment 1194338 [details]
test.py

The attached script shows what appears to be a
design problem in python3. The bug happens in all
python3 packages I tested (scl, rhel7, rawhide).

  There are two "barrier" like abstractions on
/usr/lib64/python3*/logging/handlers.py in the
_monitor method.

  First _monitor has two loops, what is already
kind of a hint something is not right.

  Second, it has two ways to exit the loop, that
also exit the thread:
1) The _stop threading.Event is "set"
2) The _sentinel object is added to the queue

  The problem is, the documentation says that the
correct way to not loose records, the stop method
must be called, but, the stop method just sets
the _stop object and then adds the _sentinel object
to the queue.

  The loop stops when noticing that _stop is set,
and then enters a second version of the loop, trying
again to see the _sentinel object, but this time
with non blocking read.

  The test case shows the problem, but it also
hints about the race conditions by the fact that
running the test case under "taskset 1" works,
so, to reproduce the issue, run the test under a
multiprocessor environment.

  The proper solution would be to have a proper
locking mechanism, otherwise, the _stop object
should not be used, and rely only in seeing the
_sentinel field; this is what the class
DeterministicQueueListener does in the test case.

Comment 1 Petr Viktorin (pviktori) 2016-09-01 15:18:16 UTC
Thank you for the report and analysis. I've reported an [upstream issue]; let's see if the original author of this code has any comments.


[upstream issue]: http://bugs.python.org/issue27930

Comment 4 Joe Orton 2016-10-12 12:31:00 UTC
Red Hat does not currently plan to provide any further changes to this collection in a Red Hat Software Collections update release.

This software collection is nearing the retirement date (October 2016) after which customers are encouraged either to upgrade to a later release or continue on as self-supported without official Red Hat Support.

Please contact Red Hat Support if you have further questions, or refer to the support lifecycle page for more information. https://access.redhat.com/support/policy/updates/rhscl/

Comment 5 Deepu K S 2016-10-13 15:03:46 UTC
Since the issue is present upstream, I am changing the component to next python version under SCL.
"rh-python34"

Thanks,

Comment 6 Charalampos Stratakis 2016-10-26 12:13:04 UTC
The issue is fixed upstream, so backporting the [patch] should be fairly straightforward.

[patch] https://hg.python.org/cpython/rev/b2ea0ede3753/

Comment 8 Joe Orton 2017-03-31 15:02:02 UTC
In accordance with the Red Hat Software Collections Product Life Cycle, the support period for this collection has ended.

New bug fix, enhancement, and security errata updates, as well as technical support services will no longer be made available for this collection.

Customers are encouraged to upgrade to a later release.

Please contact Red Hat Support if you have further questions, or refer to the support lifecycle page for more information. https://access.redhat.com/support/policy/updates/rhscl/


Note You need to log in before you can comment on or make changes to this bug.