Bug 1293502

Summary: Docker spawns a docker-proxy process for each port in a container
Product: Red Hat Enterprise Linux 7 Reporter: wibrown <wibrown>
Component: dockerAssignee: Antonio Murdaca <amurdaca>
Status: CLOSED ERRATA QA Contact: atomic-bugs <atomic-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.1CC: amurdaca, lsm5, lsu, mpatel
Target Milestone: rcKeywords: Extras
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-03-31 23:23:11 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description wibrown@redhat.com 2015-12-22 00:08:52 UTC
Description of problem:
Docker spawns a docker-proxy process for each port in a container.

Consider the usage of asterisk in a container. This required the opening of hundreds, if not thousands of ports.

This can cause the docker host to crash.

docker run -it --rm=true --name porttest -p 3000-3010 busybox

In another shell:

ps ax | grep docker-proxy | egrep '30[0-9]{2}' | wc

Should yield 11 processes. If this is increased, such as:

docker run -it --rm=true --name porttest -p 3000-4000 busybox

Your docker host may destabilise, crash or freeze.

How reproducible:
Always

Actual results:
Docker host crashes from the number of spawned processes

Expected results:
Docker host should not crash or freeze, and there should be one docker-proxy for each running container.

Comment 2 Mrunal Patel 2015-12-22 17:10:56 UTC
Wasn't there an option to use iptables instead of the userland port proxy?

Comment 3 Antonio Murdaca 2015-12-22 19:26:34 UTC
Enabling hairpin NAT via the daemon flag "--userland-proxy=false" prevents the daemon from spawning docker-proxy processes. This way the port mapping is achieved via just iptables (as Mrunal said).

Are there any reasons you can't disable userland-proxy? Could you try setting "--userland-proxy=false" and see if the daemon still has issues?

Comment 5 wibrown@redhat.com 2016-01-20 02:13:26 UTC
I believe that using the iptables rules prevents container to container communication.

Comment 6 Antonio Murdaca 2016-01-20 09:22:49 UTC
could you tell me how you reproduce the blocked container to container communication?

Comment 7 wibrown@redhat.com 2016-01-21 00:18:20 UTC
Here is the information I was sent:

"""
Well, to explain it better, take a host that's on 192.168.0.1, running a Docker container listening on port 5000 (in the container) that's exposed on port 800 on the host.  Assume that the first container's Docker IP is 172.17.0.1.  Then, take another Docker container on the same host trying to connect to that exposed port;

APP=`docker run -d -p 800:5000/tcp training/webapp python app.py`
curl http://192.168.0.1:800/   # works, of course
docker run --rm -it image curl http://192.168.0.1:800/   # does not work
docker run --rm -it image curl http://172.17.0.1:5000/   # does work
docker stop $APP
docker rm $APP
"""

I haven't tried this myself yet however.

Comment 8 Luwen Su 2016-03-20 13:11:57 UTC
In docker-1.9.1-23.el7.x86_64, move to verified.

Just to be noted,
It's better not try `docker run -it --rm=true --name porttest -p 3000-4000 busybox` in  remote machine which used network is not strong enough.

Comment 10 errata-xmlrpc 2016-03-31 23:23:11 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-0536.html