Bug 2106319 - Docker / Moby-Engine: can't build image with with BuildKit enabled because of SELinux
Summary: Docker / Moby-Engine: can't build image with with BuildKit enabled because of...
Keywords:
Status: CLOSED EOL
Alias: None
Product: Fedora
Classification: Fedora
Component: moby-engine
Version: 36
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Olivier Lemasle
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-07-12 11:24 UTC by Manuel Colmenero
Modified: 2023-05-25 18:17 UTC (History)
5 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2023-05-25 18:17:13 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Manuel Colmenero 2022-07-12 11:24:18 UTC
Description of problem:
Docker / Moby-Engine: can't build image with with BuildKit enabled (DOCKER_BUILDKIT=1) because of SELinux. No changes to the overlay file system are allowed by SELinux policies.

Building images does work with BuildKit disabled (DOCKER_BUILDKIT=0), and also with `setenforce 0`. I did not change the default Docker directories or their SELinux labels.

I also tried restoring the labels to no avail with:
restorecon -R -v /var/lib/containers
restorecon -R -v /var/lib/docker

Version-Release number of selected component (if applicable):

Fedora release 36

=== Docker version output: ===
Client:
 Version:           20.10.17
 API version:       1.41
 Go version:        go1.18.3
 Git commit:        aa7e414
 Built:             Tue Jun 21 20:56:14 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server:
 Engine:
  Version:          20.10.17
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.18.3
  Git commit:       f756502
  Built:            Tue Jun 21 20:56:14 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.6
  GitCommit:
 runc:
  Version:          1.1.1
  GitCommit:        77a38de-dirty
 docker-init:
  Version:          0.19.0
  GitCommit:
===================

=== Docker info output: ===
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  compose: Docker Compose (Docker Inc., v2.2.3)

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 0
 Server Version: 20.10.17
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: journald
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: /usr/libexec/docker/docker-init
 containerd version:
 runc version: 77a38de-dirty
 init version:
 Security Options:
  seccomp
   Profile: default
  selinux
  cgroupns
 Kernel Version: 5.18.10-200.fc36.x86_64
 Operating System: Fedora Linux 36 (Workstation Edition)
 OSType: linux
 Architecture: x86_64
 CPUs: 16
 Total Memory: 15.46GiB
 Name: freaky-beast
 ID: TA6T:YGPA:RUN6:7VZV:ZVUN:ZUCR:4SH3:UX4W:EDVV:L6JU:2UNW:6RCK
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: true
===================

How reproducible:


Steps to Reproduce:
1. Create a test Dockerfile with following contents:

FROM alpine:latest
RUN echo "foo" > /bar

2. Try to build an image with the given Dockerfile:

DOCKER_BUILDKIT=1 docker build . -t foobar

Actual results:

Docker exists with an error:
 > [2/2] RUN echo "foo" > /bar:
#5 0.214 /bin/sh: can't create /bar: Permission denied

Expected results:
Image is build and stored in the local cache

Additional info:
The system journal alerts that `sh` was prevented from writing because of SELinux (message in German because of system locale, omitted here).

Output of `ausearch -c 'sh' --raw`:
type=AVC msg=audit(1657624135.143:2224): avc:  denied  { write } for  pid=43348 comm="sh" name="/" dev="overlay" ino=526123 scontext=system_u:system_r:container_t:s0:c273,c313 tcontext=system_u:object_r:container_ro_file_t:s0 tclass=dir permissive=0

Comment 1 Maxwell G 2022-07-12 23:30:14 UTC
Does the solution in https://bugzilla.redhat.com/show_bug.cgi?id=2072908#c7 fix your problem?

Comment 2 Manuel Colmenero 2022-07-14 13:33:04 UTC
Hi, thanks for taking the time to answer.

No, it seems to be a different problem. After a fresh reboot I reproduce the issue and then run `audit2allow -b`:

#============= container_t ==============
allow container_t container_ro_file_t:dir write;

Running it with the `-w` flag, doesn't say exactly which directory it trying to write:

$ audit2allow -bw
type=AVC msg=audit(1657804704.662:464): avc:  denied  { write } for  pid=4275 comm="sh" name="/" dev="overlay" ino=3130955 scontext=system_u:system_r:container_t:s0:c517,c548 tcontext=system_u:object_r:container_ro_file_t:s0 tclass=dir permissive=0
        Was caused by:
                Missing type enforcement (TE) allow rule.

                You can use audit2allow to generate a loadable module to allow this access.

Maybe a misslabeling?

Grep-ing for that label in the file contexts:

$ grep container_ro_file_t /etc/selinux/targeted/contexts/files/file_contexts
/var/lib/docker/.*/config\.env  system_u:object_r:container_ro_file_t:s0
/var/lib/docker/init(/.*)?      system_u:object_r:container_ro_file_t:s0
/var/lib/containerd/[^/]*/sandboxes(/.*)?       system_u:object_r:container_ro_file_t:s0
/var/lib/containerd/[^/]*/snapshots(/.*)?       system_u:object_r:container_ro_file_t:s0
/var/lib/docker/overlay(/.*)?   system_u:object_r:container_ro_file_t:s0
/var/lib/ocid/sandboxes(/.*)?   system_u:object_r:container_ro_file_t:s0
/var/lib/docker-latest/.*/config\.env   system_u:object_r:container_ro_file_t:s0
/var/lib/docker/overlay2(/.*)?  system_u:object_r:container_ro_file_t:s0
/var/lib/kata-containers(/.*)?  system_u:object_r:container_ro_file_t:s0
/var/cache/kata-containers(/.*)?        system_u:object_r:container_ro_file_t:s0
/var/lib/containers/overlay(/.*)?       system_u:object_r:container_ro_file_t:s0
/var/lib/docker-latest/init(/.*)?       system_u:object_r:container_ro_file_t:s0
/var/lib/docker/containers/.*/hosts     system_u:object_r:container_ro_file_t:s0
/var/lib/docker/containers/.*/hostname  system_u:object_r:container_ro_file_t:s0
/var/lib/containers/overlay2(/.*)?      system_u:object_r:container_ro_file_t:s0
/var/lib/docker-latest/overlay(/.*)?    system_u:object_r:container_ro_file_t:s0
/var/lib/docker-latest/overlay2(/.*)?   system_u:object_r:container_ro_file_t:s0
/var/lib/containers/overlay-images(/.*)?        system_u:object_r:container_ro_file_t:s0
/var/lib/containers/overlay-layers(/.*)?        system_u:object_r:container_ro_file_t:s0
/var/lib/docker-latest/containers/.*/hosts      system_u:object_r:container_ro_file_t:s0
/var/lib/docker-latest/containers/.*/hostname   system_u:object_r:container_ro_file_t:s0
/var/lib/containers/overlay2-images(/.*)?       system_u:object_r:container_ro_file_t:s0
/var/lib/containers/overlay2-layers(/.*)?       system_u:object_r:container_ro_file_t:s0
/var/lib/containers/storage/overlay(/.*)?       system_u:object_r:container_ro_file_t:s0
/var/lib/containers/storage/overlay2(/.*)?      system_u:object_r:container_ro_file_t:s0
/var/lib/containers/storage/overlay-images(/.*)?        system_u:object_r:container_ro_file_t:s0
/var/lib/containers/storage/overlay-layers(/.*)?        system_u:object_r:container_ro_file_t:s0
/var/lib/containers/storage/overlay2-images(/.*)?       system_u:object_r:container_ro_file_t:s0
/var/lib/containers/storage/overlay2-layers(/.*)?       system_u:object_r:container_ro_file_t:s0

Seems to me like /var/lib/docker/overlay2 should be `container_file_t` and not `container_ro_file_t`? But then again why did Docker work without BuildKit enabled?

Comment 3 Manuel Colmenero 2022-07-14 13:45:59 UTC
To anwser my own question, changing this line:

/var/lib/docker/overlay2(/.*)?  system_u:object_r:container_ro_file_t:s0

To this one:

/var/lib/docker/overlay2(/.*)?  system_u:object_r:container_file_t:s0

And then running

restorecon -Rv /var/lib/docker

Did fix the issue. But the question remains, why did Docker work without BuildKit enabled? Are the overlay directories wrongly labeled? Should be `container_file_t` instead of `container_ro_file_t` by default?

Thanks again,
Manu

Comment 4 Yann Soubeyrand 2023-03-24 14:10:57 UTC
Hello,

I confirm this bug is still present in Fedora 38 beta.

Comment 5 John 2023-04-16 17:40:46 UTC
Encountered issue on F37, see discussion in https://bugzilla.redhat.com/show_bug.cgi?id=2093501

Comment 6 John 2023-04-18 02:13:09 UTC
Issue is blocking if we want to upgrade past moby-engine 20.10 due to the way of generating the docs. https://github.com/docker/cli/tree/master/man, what is needed to get this fixed?

Comment 7 Ben Cotton 2023-04-25 17:35:13 UTC
This message is a reminder that Fedora Linux 36 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora Linux 36 on 2023-05-16.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
'version' of '36'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, change the 'version' 
to a later Fedora Linux version. Note that the version field may be hidden.
Click the "Show advanced fields" button if you do not see it.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora Linux 36 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora Linux, you are encouraged to change the 'version' to a later version
prior to this bug being closed.

Comment 8 Ludek Smid 2023-05-25 18:17:13 UTC
Fedora Linux 36 entered end-of-life (EOL) status on 2023-05-16.

Fedora Linux 36 is no longer maintained, which means that it
will not receive any further security or bug fix updates. As a result we
are closing this bug.

If you can reproduce this bug against a currently maintained version of Fedora Linux
please feel free to reopen this bug against that version. Note that the version
field may be hidden. Click the "Show advanced fields" button if you do not see
the version field.

If you are unable to reopen this bug, please file a new report against an
active release.

Thank you for reporting this bug and we are sorry it could not be fixed.


Note You need to log in before you can comment on or make changes to this bug.