Bug 1654325 - Can't open any new terminals after running oc cluster up using moby-engine
Summary: Can't open any new terminals after running oc cluster up using moby-engine
Keywords:
Status: CLOSED NEXTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: moby-engine
Version: 29
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Olivier Lemasle
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-11-28 14:29 UTC by Jason Montleon
Modified: 2019-07-12 19:47 UTC (History)
3 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2019-07-12 19:47:37 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Jason Montleon 2018-11-28 14:29:22 UTC
Description of problem:
Run oc cluster up using the moby-engine

Version-Release number of selected component (if applicable):
rpm -q moby-engine
moby-engine-18.06.0-1.ce.git0ffa825.fc29.x86_64

$ ~/bin/oc version
oc v3.11.0+0cbc58b
kubernetes v1.11.0+d4cacc0
features: Basic-Auth GSSAPI Kerberos SPNEGO

Server https://172.17.0.1:8443
kubernetes v1.11.0+d4cacc0

How reproducible:
Seems always

Steps to Reproduce:
1. Install moby-engine and start the docker service
2. Bring up an openshift cluster using oc cluster up, something like, "oc cluster up --routing-suffix=172.17.0.1.nip.io --public-hostname=172.17.0.1 --base-dir=/tmp/openshift.local.clusterup --tag=v3.11 --image=docker.io/openshift/origin-\${component}:\${version} --enable=service-catalog,router,registry,web-console,persistent-volumes,sample-templates,rhel-imagestreams"
3. Try to open a new terminal, whether xterm, terminator, whatever. 

Actual results:
Terminal won't open. If you try to run xterm from an already open terminal you'll see

$ xterm
xterm: Error 32, errno 13: Permission denied
Reason: get_pty: not enough ptys

Expected results:
Running oc cluster up doesn't break stuff

Additional info:
This doesn't happen using  docker-2:1.13.1-62.git9cb56fd.fc29.x86_64

As long as I have a terminal open I can do chmod a+rw /dev/pts/ptmx to work around the problem.

Comment 1 Jason Montleon 2018-11-28 15:00:57 UTC
From: https://github.com/opencontainers/runc/issues/80

"When a container is launched with a bind mount between /dev (on the host) and /dev (in the container), all subsequent attempts to open /dev/ptmx fail.

setupPtmx() in libcontainer/rootfs_linux.go will unconditionally remove /dev/ptmx and then symlink it to /dev/pts/ptmx , which is a character file that is untouchable by unprivileged users. This prevents unprivileged programs which rely on getpt from running on the host (such as X terminals, screen, and tmux)."

It looks like this was originally fixed in https://github.com/opencontainers/runc/pull/96.

The again in August of this year with https://github.com/opencontainers/runc/issues/1866

Your runc in the package is from July so doesn't appear to have this change.

I patched the included runc and am running a build now. I'll report back when it's done whether it fixes the problem.

Comment 2 Jason Montleon 2018-11-28 15:14:11 UTC
Applying this patch appears to correct the problem and /dev/ptmx is no longer replaced with a link after running oc cluster up.

Comment 3 Jason Montleon 2018-12-03 15:11:49 UTC
This is still broken in moby-engine-18.06.0-1.ce.git0ffa825.fc29.x86_64.

Retrieving the patch and some minor modifications to the RPM spec should be sufficient to fix this if updating runc to a newer version isn't desired.

wget https://github.com/opencontainers/runc/pull/1867.patch -O revert_libcontainer_rootfs_linux_cleaup.patch

diff --git a/moby-engine.spec.orig b/moby-engine.spec
index 46aeb28..d55b7c7 100644
--- a/moby-engine.spec.orig
+++ b/moby-engine.spec
@@ -50,6 +50,7 @@ Source1: %{git_runc}/archive/%{commit_runc}.tar.gz
 Source2: %{git_containerd}/archive/%{commit_containerd}.tar.gz
 Source3: %{git_libnetwork}/archive/%{commit_libnetwork}.tar.gz
 Source4: %{git_tini}/archive/%{commit_tini}.tar.gz
+Patch0: revert_libcontainer_rootfs_linux_cleaup.patch
 URL: https://www.%{origname}.com
 
 BuildRequires: btrfs-progs-devel
@@ -135,7 +136,7 @@ Requires: nano
 This package installs %{summary}.
 
 %prep
-%autosetup -Sgit -n %{service_name}-ce-%{commit_moby}
+%setup -n %{service_name}-ce-%{commit_moby}
 
 # correct rpmlint errors for bash completion
 pushd components/cli
@@ -144,6 +145,9 @@ popd
 
 # untar runc
 tar zxf %{SOURCE1}
+pushd runc-%{commit_runc}
+%patch0 -p1
+popd
 
 # untar containerd
 tar zxf %{SOURCE2}

Comment 4 David Michael 2018-12-03 17:32:26 UTC
Can you try the 18.06.1 version in rawhide?  That version's runc snapshot looks like it is after the issue was fixed the first time, but before it was broken again.

https://dl.fedoraproject.org/pub/fedora/linux/development/rawhide/Everything/x86_64/os/Packages/m/moby-engine-18.06.1-1.ce.gite68fc7a.fc30.x86_64.rpm

Comment 5 Jason Montleon 2018-12-03 17:52:19 UTC
Thanks, the Fedora 30 package fixes it!

Sorry if my last comment is confusing. I didn't look closely and it looks like when I did a dnf update this morning it reinstalled the same package I initially reported against. I thought I had picked up a new version.

Comment 6 Olivier Lemasle 2019-07-12 19:47:37 UTC
Closing as the requester wrote the issue is fixed in Fedora 30.


Note You need to log in before you can comment on or make changes to this bug.