Bug 1755635

Summary: jupyter_client: cannot start manager with stdout redirected to /dev/null
Product: [Fedora] Fedora Reporter: Jerry James <loganjerry>
Component: python-jupyter-clientAssignee: Thomas Spura <tomspur>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: unspecified    
Version: rawhideCC: cbm, mhroncok, python-sig, tomspur
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: python-jupyter-client-5.3.3-1.fc31 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-10-06 00:01:17 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jerry James 2019-09-25 22:00:33 UTC
Description of problem:
The python-ipykernel package is failing many tests in Rawhide, which I discovered when I looked into building a new version.  It looks like jupyter_client cannot start a manager when stdout refers to /dev/null, at least in mock.  Here's a transcript from a mock shell:

$ python3
Python 3.8.0b4 (default, Aug 30 2019, 00:00:00) 
[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> from subprocess import STDOUT
>>> from jupyter_client import manager
/usr/lib/python3.8/site-packages/jupyter_client/manager.py:81: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  if self._kernel_spec is None and self.kernel_name is not '':
>>> KM, KC = manager.start_new_kernel(startup_timeout=120, stdout=open(os.devnull), stderr=STDOUT)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.8/site-packages/jupyter_client/manager.py", line 472, in start_new_kernel
    kc.wait_for_ready(timeout=startup_timeout)
  File "/usr/lib/python3.8/site-packages/jupyter_client/blocking/client.py", line 120, in wait_for_ready
    raise RuntimeError('Kernel died before replying to kernel_info')
RuntimeError: Kernel died before replying to kernel_info

On the other hand, leaving stdout and stderr at their default values works:

>>> KM, KC = manager.start_new_kernel(startup_timeout=120)
/usr/lib/python3.8/site-packages/jupyter_client/manager.py:81: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  if self._kernel_spec is None and self.kernel_name is not '':
>>> KM
<jupyter_client.manager.KernelManager object at 0x7faa7ff28400>
>>> KC
<jupyter_client.blocking.client.BlockingKernelClient object at 0x7faa7db69eb0>

Setting stderr=STDOUT and leaving stdout alone also works.  Redirecting stdout as shown above succeeds on an F-30 machine with python3-jupyter-client-5.2.3-7.fc32.noarch.

Version-Release number of selected component (if applicable):
python3-jupyter-client-5.2.3-10.fc32.noarch

How reproducible:
Always

Steps to Reproduce:
1. Run the python code above
2.
3.

Actual results:
The manager creation step fails when stdout points to /dev/null.

Expected results:
The manager creation step should succeed.

Additional info:

Comment 1 Miro HronĨok 2019-09-26 21:29:28 UTC
RuntimeError: Kernel died before replying to kernel_info

I've seen this once, and it was an undebuggable nightmare and than it just stopped: https://github.com/ipython/ipykernel/issues/406

Comment 2 Jerry James 2019-09-27 15:08:35 UTC
Interesting.  Well, it seems to be gone again with the latest builds....

Comment 3 Fedora Update System 2019-09-27 15:13:23 UTC
FEDORA-2019-65876aef60 has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2019-65876aef60

Comment 4 Fedora Update System 2019-09-28 00:53:32 UTC
Macaulay2-1.14.0.1-1.fc31, Singular-4.1.1p3-8.fc31, cddlib-0.94j-4.fc31, eclib-20190909-1.fc31, flint-2.5.2-28.fc31, frobby-0.9.0-20.fc31, giac-1.5.0.63-2.fc31, latte-integrale-1.7.5-4.fc31, linbox-1.5.2-8.fc31, normaliz-3.8.0-2.fc31, ntl-11.4.0-1.fc31, polymake-3.5-4.fc31, pynac-0.7.24-4.fc31, python-ipykernel-5.1.2-1.fc31, python-jupyter-client-5.3.3-1.fc31, python-jupyter-core-4.5.0-1.fc31, sagemath-8.8-6.fc31 has been pushed to the Fedora 31 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-65876aef60

Comment 5 Fedora Update System 2019-10-06 00:01:17 UTC
Macaulay2-1.14.0.1-1.fc31, Singular-4.1.1p3-8.fc31, cddlib-0.94j-4.fc31, eclib-20190909-1.fc31, flint-2.5.2-28.fc31, frobby-0.9.0-20.fc31, giac-1.5.0.63-2.fc31, latte-integrale-1.7.5-4.fc31, linbox-1.5.2-8.fc31, normaliz-3.8.0-2.fc31, ntl-11.4.0-1.fc31, polymake-3.5-4.fc31, pynac-0.7.24-4.fc31, python-ipykernel-5.1.2-1.fc31, python-jupyter-client-5.3.3-1.fc31, python-jupyter-core-4.5.0-1.fc31, sagemath-8.8-6.fc31 has been pushed to the Fedora 31 stable repository. If problems still persist, please make note of it in this bug report.