Bug 1461574 - docker run --read-only : fails when user namespaces enabled
docker run --read-only : fails when user namespaces enabled
Status: VERIFIED
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: docker (Show other bugs)
7.4
Unspecified Unspecified
medium Severity medium
: rc
: ---
Assigned To: Tom Sweeney
atomic-bugs@redhat.com
: Extras
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2017-06-14 16:06 EDT by Ed Santiago
Modified: 2017-12-07 10:48 EST (History)
3 users (show)

See Also:
Fixed In Version: docker-latest-1.13.1-36.git9a813fa.el7
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Ed Santiago 2017-06-14 16:06:16 EDT
Setup: RHEL 7.4, docker daemon running with--userns-remap=default:

    # docker run --read-only centos date
    /usr/bin/docker-current: Error response from daemon: Cannot use the --read-only option when user namespaces are enabled.
    See '/usr/bin/docker-current run --help'.

It succeeds if you add --userns=host to docker run, but it's not clear why this is required.

Full setup details:

    # echo 100 >/proc/sys/user/max_user_namespaces
    # for i in uid gid; do echo "dockremap:100000:65536" > /etc/sub$i;done
    # vi /etc/sysconfig/docker
    [ add --userns-remap=default to OPTIONS ]
    # systemctl stop docker
    # rm -rf /var/lib/docker
    # docker-storage-setup --reset
    # docker-storage-setup
    # systemctl start docker
Comment 2 Daniel Walsh 2017-06-15 07:39:29 EDT
Tom could you figure out what is going on here?
Comment 3 Tom Sweeney 2017-06-21 15:15:15 EDT
The docker daemon code was returning this error whenever the --read-only switch was used and user namespaces were enable unless as Ed noted the '--userns=host' parameter was included.  I found this in git commit 301bfbdd21c7795c41611bf8a8c31a0136b91bde and earlier.  The most recent docker daemon code no longer includes that if statement.

This can be seen:  http://pastebin.test.redhat.com/496452

I'm trying to figure out the easiest way to get the latest docker/moby installed on a 7.4 test system as it's not yet in the repos apparently.
Comment 4 Ed Santiago 2017-06-21 17:38:45 EDT
This is the commit that removed it: 

  https://github.com/moby/moby/commit/6062ae5742e49ec1a79073c327f3d1343c218a12

It's from August 2016, tagged in v1.13.0-rc1, so the obvious test is to try the reproducer on docker-latest. Unfortunately I can't seem to get userns to work at all on docker-latest-1.13.1-19.1.git19ea2d3.el7. I will need to pursue this tomorrow.
Comment 5 Tom Sweeney 2017-06-22 08:34:45 EDT
I installed docker-latest-1.13.1-19.1.git19ea2d3.el7 on my test machine and it's no longer failing with the --read-only error.  I did run into an error that I think is unrelated.  I'd trouble pulling centos, but if I changed to rhel, it worked.

Also before installing the docker, I first did:

yum remove docker*
yum remove oci-*

Then I installed docker-latest.  I've found things go a little south if you don't first remove the images.

[root@rhelbz ~]# docker run --read-only centos date
Unable to find image 'centos:latest' locally
Trying to pull repository registry.access.redhat.com/centos ... 
Trying to pull repository docker.io/library/centos ... 
latest: Pulling from docker.io/library/centos
d5e46245fe40: Pulling fs layer 
/usr/bin/docker-latest: error pulling image configuration: Get https://dseasb33srnrn.cloudfront.net/registry-v2/docker/registry/v2/blobs/sha256/3b/3bee3060bfc81c061ce7069df35ce090593bda584d4ef464bc0f38086c11371d/data?Expires=1498135692&Signature=Htg2XOo1U3IkVPOgRRVDfDO9xR1lO5l5lD-nqjd0KqW5u3HWJNLzMWwJ9q4ZSSjJqo1PYK3OwrOjccVApHQQHlvoXsi7P7LGwTmXmffLWHqGsHP1bITXJIJ6oUMoQLriNTEwJra4zRZbkhhQHv9Ie4NtGqHHbHdaK5n-fdQFcco_&Key-Pair-Id=APKAJECH5M7VWIS5YZ6Q: dial tcp: lookup dseasb33srnrn.cloudfront.net on 192.168.122.1:53: read udp 192.168.122.108:40182->192.168.122.1:53: i/o timeout.
See '/usr/bin/docker-latest run --help'.
[root@rhelbz ~]# docker run --read-only rhel7 date
Thu Jun 22 12:30:38 UTC 2017
Comment 6 Tom Sweeney 2017-06-22 09:39:34 EDT
And fwiw, I just tried the centos variant and it now works:

[root@rhelbz ~]# docker run --read-only centos date
Unable to find image 'centos:latest' locally
Trying to pull repository registry.access.redhat.com/centos ... 
Trying to pull repository docker.io/library/centos ... 
latest: Pulling from docker.io/library/centos
d5e46245fe40: Pull complete 
Digest: sha256:aebf12af704307dfa0079b3babdca8d7e8ff6564696882bcb5d11f1d461f9ee9
Status: Downloaded newer image for docker.io/centos:latest
Thu Jun 22 13:38:37 UTC 2017

Looks like a network/registry issue earlier.
Comment 7 Ed Santiago 2017-06-22 11:13:18 EDT
Kernel regression in 3.10.0-681 breaks userns. Bug 1463072 , bug 1462124
Comment 8 Tom Sweeney 2017-06-22 16:28:29 EDT
Ed, based on what we've found/seen, ok to close out this Bugzilla?  I think it's covered in one or both of the bugzilla's that you found.
Comment 9 Ed Santiago 2017-06-22 16:46:59 EDT
Tough call. I don't think I can decide this.

My hope yesterday was to confirm that docker-1.13 fixes this, and then ask the     team if we can live with the limitation in 1.12. The kernel bug throws a monkey wrench because right now no testing is possible.

I would like to leave this open until such time as the kernel bug is fixed. At that time, I would like to double-confirm that docker-latest, run with user namespaces enabled on a RHEL 7.4 system, running a container with --read-only works as expected. I can't in good conscience close this when it's not working.
Comment 10 Tom Sweeney 2017-06-22 17:01:32 EDT
Valid points, let's keep it open then.

Note You need to log in before you can comment on or make changes to this bug.